A method for enhancing the processing of stacked images

Abstract

The present invention provides a method for enhancing and processing stacked images, which relates to the field of image enhancement technology. The method comprises the following steps: collecting and preprocessing stacked images to construct a training sample set containing risk level and tilt angle labels; constructing an encoder-classifier two-stage model, wherein the encoder extracts features through multi-scale feature extraction and enhancement, spatial perception attention enhancement, dynamic feature pyramid fusion, and a deformation gradient enhancement residual module, and the classifier outputs risk level and tilt angle; dynamically adjusting classification and regression loss weights during training, generating adversarial enhancement samples to enhance robustness, and adaptively clipping gradients based on channel attention enhancement vectors. The method can effectively eliminate local illumination interference, fuse multi-scale features, enhance sensitivity to sparse risk areas, and improve the adaptive enhancement processing effect of stacked images in complex environments.

Description

Technical Field

[0001] The present invention relates to the field of image enhancement technology, and in particular to a stacked image enhancement processing method. Background Art

[0002] With the continuous development of construction engineering, stacked load structures are increasingly being used in various fields, including construction and warehousing. During the stacking process, stacked objects often deform due to uneven stacking, environmental fluctuations, or improper operation, leading to tilting or deformation, significantly increasing the risk of structural collapse. Therefore, effectively processing stacked load images under these changing conditions has become a key issue in stacked load risk detection.

[0003] Prior art proposes a fusion module and a multi-scale feature fusion convolutional neural network. This neural network consists of a previous layer, a fusion layer, and a cascade layer. The previous layer operates on the input image through the fusion layer, and then merges the results of the operation through the cascade layer for output. This prior art attempts to improve the model's ability to capture image details by fusing dilated convolution with a multi-scale feature extraction network with a bottleneck structure. However, the global mean-variance normalization preprocessing method it uses is difficult to cope with the local highlights or shadows commonly found in stacked images, resulting in feature distortion in the image data input to the model. In particular, key details such as cracks are weakened during the normalization process.

[0004] Furthermore, the channel attention enhancement mechanisms widely used in existing technologies (such as the Squeeze-and-Excitation module) focus only on the response weights between feature channels, but fail to optimize for the spatial sparsity of overloading risk areas (such as regional deformation caused by local overloading). This results in insufficient model response to features in risk-concentrated areas. At the training strategy level, existing multi-task learning methods use fixed weight allocation. When faced with imbalanced datasets, they are unable to dynamically increase the weight of high-risk categories in the loss function, resulting in a significant increase in the model's prediction bias for small sample risk levels. In particular, in the tilt angle regression task, the continuous variation features between adjacent frames are not effectively utilized, making it impossible to effectively enhance the overloading image. Summary of the Invention

[0005] In view of this, an embodiment of the present invention provides a stacked image enhancement processing method to eliminate or improve one or more defects existing in the prior art, and solve the problems in the prior art that conventional image normalization methods cannot effectively eliminate local highlight or shadow interference and easily weaken subtle features, traditional convolutional neural networks are difficult to simultaneously capture fine features and overall structural features, the attention mechanism ignores the influence of spatial sparsity, and the multi-task learning method uses a fixed task weight distribution, resulting in poor model adaptability to imbalanced data sets.

[0006] In one aspect, the present invention provides a method for enhancing a stacked image, the method comprising:

[0007] Preprocessing the images of the pile load collected under different environmental conditions and adding labels to construct a training sample set; the labels include risk level and tilt angle;

[0008] An initial model of an encoder-classifier two-stage architecture is constructed; the initial model takes the stacked image as input, and the encoder extracts and fuses the multi-scale features of the stacked image through a multi-scale feature enhancement module; attention is enhanced through a spatial perception attention enhancement module; attention-enhanced feature tensors of different levels are adaptively weighted and fused through a dynamic feature pyramid fusion module; deformation feature expression is enhanced through a deformation gradient enhancement residual module, and the output residual enhancement feature tensor is handed over to the classifier to output a risk level class probability vector and a tilt angle estimate;

[0009] The initial model is trained using the training sample set, and a joint loss of risk level classification loss and tilt angle regression loss is constructed based on dynamic task weights for iterative optimization. Every time a preset number of iterations is reached, environmental interference items are added to the load image to generate adversarial enhancement samples that are added to the training sample set. Based on the channel attention enhancement vector calculated in the spatial perception attention enhancement module, the gradient update parameters are adaptively clipped to obtain a load risk detection model.

[0010] Acquire the image of the pile load to be detected; input the preprocessed image of the pile load to be detected into the pile load risk detection model to obtain the risk level and tilt angle of the image of the pile load to be detected.

[0011] In some embodiments of the present invention, preprocessing of the stacked images collected under different environmental conditions includes:

[0012] Calculating a local mean and a local standard deviation of the stacked image based on the edge region;

[0013] Converting the stacked image into a grayscale image; calculating the local variance of the grayscale image to obtain a texture enhancement mask;

[0014] The stacked image is subjected to normalized enhancement processing based on the local mean and the local standard deviation, and is combined with the texture enhancement mask to obtain a preprocessed stacked image tensor.

[0015] In some embodiments of the present invention, extracting and fusing multi-scale features of the stacked image by a multi-scale feature enhancement module includes:

[0016] The multi-scale feature enhancement module adopts a dual-pathway feature pyramid structure including a texture path and a geometric path;

[0017] The texture path uses multi-layer dilated convolution to extract texture features of the stacked image;

[0018] The geometric path uses deformable convolutional layers and pooling layers to extract geometric features of the stacked image;

[0019] The texture features and the geometric features are fused to obtain a multi-scale enhanced feature tensor.

[0020] In some embodiments of the present invention, attention enhancement is performed by a spatial perception attention enhancement module, including:

[0021] Performing global average pooling and nonlinear transformation processing on the multi-scale enhanced feature tensor output by the multi-scale feature enhancement module to obtain a global weight factor;

[0022] Based on the Laplacian operator, the second-order gradient map of each channel of the multi-scale enhanced feature tensor is calculated and interacted across channels to generate a gradient weight factor;

[0023] Add the global weight factor to the gradient weight factor and generate a channel attention enhancement vector through a Sigmoid activation function;

[0024] The channel attention enhancement vector and the multi-scale enhancement feature tensor are multiplied, and the multi-scale enhancement feature tensor is residually connected to obtain the attention enhancement feature tensor.

[0025] In some embodiments of the present invention, adaptive weighted fusion of attention-enhanced feature tensors at different levels is performed through a dynamic feature pyramid fusion module, including:

[0026] Dividing the fused feature tensor output by the spatial perception attention enhancement module into a multi-level pyramid; upsampling the features of adjacent levels to obtain upsampled features;

[0027] The target layer feature is concatenated with the upsampled feature to obtain a joint feature; the cross-layer spatial dependency of each pixel position in the joint feature is extracted; the cross-layer spatial dependency is normalized by the layer dimension using the Softmax function to obtain a spatial weight;

[0028] The spatial weight and the upsampled feature are weighted and summed to obtain a fused feature tensor.

[0029] In some embodiments of the present invention, the deformation feature expression is enhanced by a deformation gradient enhancement residual module, and the residual enhancement feature tensor is output, including:

[0030] Based on a horizontal gradient operator and a vertical gradient operator, a horizontal gradient feature map, a vertical gradient feature map, and a gradient amplitude of the fused feature tensor output by the dynamic feature pyramid fusion module are calculated; the horizontal gradient feature map and the vertical gradient feature map are merged into a three-dimensional tensor; and a deformation-sensitive mask is calculated based on the gradient amplitude;

[0031] The filtered three-dimensional tensor is multiplied by the deformation-sensitive mask to obtain a gradient-enhanced feature; the gradient-enhanced feature is convolved and connected with the residual of the fused feature tensor to obtain the residual-enhanced feature tensor.

[0032] In some embodiments of the present invention, each time a preset number of iterations is reached, an environmental interference term is added to the stacked image to generate an adversarial enhancement sample, including:

[0033] Constructing a physical model based on physical characteristics of various environments, wherein the environments include at least a rainy and foggy environment and a dusty environment;

[0034] The environmental interference term is calculated based on the physical model and added to the loaded image to generate the adversarial enhancement sample.

[0035] In some embodiments of the present invention, adaptively clipping the gradient based on the channel attention enhancement vector calculated in the spatial perception attention enhancement module includes:

[0036] Calculating the gradient magnitude based on a horizontal gradient operator and a vertical gradient operator; calculating a weighted average of the channel attention enhancement vector and the gradient magnitude to obtain an attention-gradient joint expected value;

[0037] Generate a gradient clipping threshold based on a preset base threshold and the attention-gradient joint expected value;

[0038] Calculate the L2 norm of the gradient of each parameter of the initial model and compare it with the gradient clipping threshold. If the L2 norm is greater than the gradient clipping threshold, perform gradient clipping based on the gradient clipping threshold; otherwise, retain the original gradient.

[0039] In some embodiments of the present invention, a joint loss of risk level classification loss and tilt angle regression loss is constructed based on dynamic task weights, including:

[0040] In each training batch, the mean of the risk level probability vectors of all samples is calculated, and the Sigmoid function is used to map and normalize them to obtain the risk weight factor;

[0041] Obtain the tilt angle prediction value within a preset time period, calculate the change in tilt angle between adjacent frames, and calculate the continuity weight factor;

[0042] Construct the classification loss of the risk level class probability vector and the risk level in the label, construct the regression loss of the tilt angle estimate and the tilt angle in the label, and weight the classification loss and the regression loss based on the risk weight factor and the continuity weight factor to obtain a joint loss.

[0043] In some embodiments of the present invention, iterative optimization is performed based on a joint loss, including:

[0044] The parameters of the initial model were updated using back propagation of the AdamW optimizer, and the initial learning rate was set to 0.001.

[0045] The present invention provides a method for enhancing a stacked image, which has the following beneficial effects:

[0046] Heap images are collected and preprocessed using the illumination-texture decoupling normalization method to construct a training sample set containing risk level and tilt angle labels. This method effectively separates the brightness component and texture features in the image, thereby enhancing local structural details, avoiding interference from local highlights or shadows, and more accurately preserving key features such as small cracks, thereby enhancing the recognizability of risk areas.

[0047] A two-stage encoder-classifier model is constructed. The encoder extracts features through multi-scale feature extraction and enhancement, spatial perception attention enhancement, dynamic feature pyramid fusion, and deformation gradient enhancement residual modules, while the classifier outputs risk level and tilt angle. The multi-scale feature enhancement module uses a dual-pathway feature pyramid structure consisting of a texture path and a geometry path. It independently extracts and fuses image features from the texture and geometry paths, solving the problem that traditional convolutional neural networks cannot effectively capture both image details and overall stacking morphology. It effectively preserves the multi-scale features in the stacked image and improves the ability to detect subtle cracks and stacking morphology changes. The spatial perception attention enhancement module uses a channel attention mechanism based on spatial perception, combined with the Laplacian operator to calculate the gradient information of the feature tensor, thereby significantly enhancing the response of risk areas in the image. By focusing on the spatial sparsity of the image, it solves the problem of conventional methods' insufficient response to sparse risk areas when processing stacked images, thereby improving classification accuracy.

[0048] During training, task weights are dynamically adjusted based on the risk level and continuity of the task, optimizing the model's adaptability to different risk levels in real time. Furthermore, adversarial enhancement samples driven by physical models enhance the model's robustness in complex environments, ensuring its stability under environmental disturbances such as rain, fog, and dust, while also adaptively clipping gradients based on channel-wise attention enhancement vectors.

[0049] The method provided by the present invention ultimately achieves high-precision and robust detection of overloading risks, effectively preventing structural collapse accidents.

[0050] Additional advantages, objects, and features of the present invention will be set forth in part in the following description and will become apparent to those skilled in the art upon examination of the following or may be learned from practice of the present invention. The objects and other advantages of the present invention may be realized and obtained by the structures particularly pointed out in the description and drawings.

[0051] Those skilled in the art will understand that the purposes and advantages that can be achieved by the present invention are not limited to the above specific descriptions, and the above and other purposes that can be achieved by the present invention will be more clearly understood based on the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of this application, and do not constitute a limitation of the present invention. In the drawings:

[0053] Figure 1 A schematic diagram of the steps of a stacked image enhancement processing method according to an embodiment of the present invention;

[0054] Figure 2 This is an experimental comparison diagram of the effects of different normalization methods on risk identification accuracy in one embodiment of the present invention;

[0055] Figure 3 This is an experimental comparison chart of the robustness of different models under adversarial enhancement samples in one embodiment of the present invention;

[0056] FIG4( a ) is a comparison diagram of feature extraction capabilities of different feature enhancement modules according to an embodiment of the present invention;

[0057] FIG4( b ) is a comparison diagram of key area response strengths of different feature enhancement modules according to an embodiment of the present invention;

[0058] FIG5( a ) is a comparison chart of the risk classification accuracy using a fixed weight strategy and a dynamic task weight strategy in one embodiment of the present invention;

[0059] FIG5( b ) is a comparison diagram of the tilt angle prediction errors using a fixed weight strategy and a dynamic task weight strategy in one embodiment of the present invention. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the exemplary embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0061] It should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, the accompanying drawings only show structures and / or processing steps closely related to the solutions according to the present invention, while other details that are not closely related to the present invention are omitted.

[0062] It should be emphasized that the term "include / comprises" when used herein refers to the existence of features, elements, steps or components, but does not exclude the existence or addition of one or more other features, elements, steps or components.

[0063] It should also be noted that, unless otherwise specified, the term "connection" herein may refer not only to a direct connection but also to an indirect connection involving an intermediate.

[0064] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0065] It should be emphasized here that the step marks mentioned below do not limit the order of the steps, but it should be understood that the steps can be executed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be executed simultaneously.

[0066] In order to solve the problems in the prior art that conventional image normalization methods cannot effectively eliminate local highlight or shadow interference and easily weaken subtle features, traditional convolutional neural networks are difficult to capture fine features and overall structural features at the same time, the attention mechanism ignores the influence of spatial sparsity, and the multi-task learning method adopts fixed task weight distribution, resulting in poor adaptability of the model to unbalanced data sets, the present invention provides a stacked image enhancement processing method, such as Figure 1 As shown, the method includes the following steps S101 to S104:

[0067] Step S101: Preprocess the images of the pile load collected under different environmental conditions and add labels to construct a training sample set. The labels include risk level and tilt angle;

[0068] Step S102: Construct an initial model using a two-stage encoder-classifier architecture. The initial model takes the pile image as input. In the encoder, the multi-scale feature enhancement module extracts and fuses the pile image's multi-scale features. The spatial perception attention enhancement module performs attention enhancement. The dynamic feature pyramid fusion module adaptively weights and fuses the attention-enhanced feature tensors at different levels. The deformation gradient enhancement residual module enhances the deformation feature representation and outputs a residual-enhanced feature tensor. The residual-enhanced feature tensor is then input into the classifier, which outputs a risk level probability vector and tilt angle estimate for the pile image.

[0069] Step S103: The initial model is trained using the training sample set. Based on the dynamic task weights, a joint loss consisting of a risk level classification loss and a tilt angle regression loss is constructed to iteratively optimize the initial model. Each time a preset number of iterations is reached, environmental interference items are added to the load image to generate adversarial enhancement samples, which are added to the training sample set. Based on the channel attention enhancement vector calculated in the spatial perception attention enhancement module, the gradient is adaptively clipped to update the parameters of the initial model until the preset training termination condition is met, resulting in a load risk detection model.

[0070] Step S104: Acquire the image of the pile load to be detected; input the pre-processed image of the pile load to be detected into the pile load risk detection model to obtain the risk level and tilt angle of the image of the pile load to be detected.

[0071] In step S101, a multi-dimensional and multi-modal data acquisition strategy is used to ensure that loading images covering different environments, weather conditions, materials, and risk stages are obtained, providing a high-quality data set for subsequent model training.

[0072] In some embodiments, a high-definition industrial camera array is deployed in the loading scene, and loading images are collected by combining fixed camera positions with mobile inspection equipment.

[0073] In some embodiments, a time series acquisition mode is used to target the dynamic changing characteristics of the pile structure. Preferably, continuous shooting is performed at a frequency of 5 frames per minute for 72 hours to fully record the entire process from the stable state to the deformation development of the pile, especially focusing on high-risk areas such as the pressure zone at the bottom of the pile and the junction of the inclined surfaces.

[0074] Each loading image is manually annotated, and corresponding risk level and tilt angle labels are added according to the loading status shown in the loading image.

[0075] In some embodiments, the risk levels include low risk, medium risk, and high risk.

[0076] In some embodiments, the tilt angle is determined by measuring an angle of deviation of the stacked object relative to a horizontal plane.

[0077] In some embodiments, the stack image data exhibits characteristics such as uneven illumination, significant material texture variations, and irregular stack geometry. Conventional normalization methods typically employ global mean-variance normalization, which cannot effectively eliminate localized interference from highlights or shadows and can easily weaken important features such as small cracks in the image. When processing the details of the stack structure, minute changes crucial for risk assessment are often overlooked. Therefore, the present invention employs a normalization method based on illumination-texture decoupling, aiming to separate the brightness component and texture features in the stack image, thereby enhancing the details of the local structure in the image and more effectively preserving critical crack and structural information.

[0078] Specifically, the Sobel operator is applied to the stacked image to calculate the edge gradient modulus of each pixel to highlight the areas with significant structural changes in the image.

[0079] In a preset window, such as a 5×5 window, the edge gradient modulus is used as the weight to calculate the local mean of the stacked image, so that the mean is more biased towards the edge area and the weight of the highlight area is reduced, as shown in formula (1):

[0080] ; (1)

[0081] in, represents the local mean; Represents pixels The intensity value of the stacking image at ; Indicated in pixels Centered window; Represents the pixels extracted by the Sobel operator The edge gradient modulus at ; represents the L2 norm.

[0082] In the same window, the local standard deviation of the stacked image is calculated to standardize the local area and eliminate the influence of light intensity differences.

[0083] The stacked image is converted to a grayscale image, and the local variance is calculated within a preset window, such as a 3×3 window. A larger variance indicates a richer texture. A texture enhancement mask is calculated based on the local variance to enhance low-variance areas (weak texture areas) and highlight subtle features, as shown in formula (2):

[0084] ; (2)

[0085] in, represents the texture enhancement mask; In order to control the hyperparameters of texture enhancement strength and solve the problem of material surface texture weakening, it is preferred that ; represents a 3×3 local variance operation; Represents a grayscale image.

[0086] The illumination normalization enhancement processing of the piled image is performed based on the local mean and local standard deviation to eliminate the influence of global and local illumination unevenness. Then, it is combined with the texture enhancement mask through element-level multiplication to obtain the preprocessed piled image tensor to enhance key details such as cracks and deformations, as shown in formula (3):

[0087] ; (3)

[0088] in, Represents the normalized enhanced image tensor; represents the original stacked image; is a constant to avoid the situation where the denominator is zero; is element-wise multiplication.

[0089] In step S102, an initial model is constructed, which adopts an encoder-classifier two-stage architecture, wherein the encoder includes a multi-scale feature enhancement module, a spatial perception attention enhancement module, a dynamic feature pyramid fusion module and a deformation gradient enhancement residual module.

[0090] In some embodiments, there is a close relationship between the risk of stacking and multi-scale features such as material crack depth and stacking inclination. In traditional technologies, convolutional neural networks process input images through fixed convolution kernels, which makes it difficult to capture features of different scales. When faced with stacking images, it is impossible to effectively distinguish the above-mentioned scale differences, making it difficult to simultaneously capture fine crack features and structural information of the overall stacking morphology. Therefore, the multi-scale feature enhancement module of the present invention adopts a dual-path feature pyramid structure containing a texture path and a geometric path to better extract and process features of different scales.

[0091] Specifically, the texture pathway uses multiple layers of dilated convolutions, preferably three layers of 3×3 dilated convolutions stacked in sequence. The geometry pathway uses deformable convolution and pooling layers, preferably a 7×7 deformable convolution kernel. The sampling points are adaptively adjusted by learning offsets to accommodate the irregular geometry of the heap. A maximum pooling layer is then used to reduce the resolution of the feature map, extracting the macroscopic features of the heap outline while reducing computational effort.

[0092] Considering that the feature map resolution of the geometric path is lower than that of the texture path due to the pooling operation, an upsampling function is used for adjustment, as shown in formula (4):

[0093] ; (4)

[0094] in, represents the upsampling function; Represents a geometric pathway; Indicates bilinear interpolation upsampling; Represents a 1×1 convolution.

[0095] The texture features output by the texture pathway and the geometric features output by the geometric pathway are spliced ​​along the channel axis to obtain a multi-scale enhanced feature tensor, as shown in formula (5):

[0096] ; (5)

[0097] in, Represents the multi-scale enhanced feature tensor; Indicates splicing along the channel axis; Represents a texture pass.

[0098] In some embodiments, risk areas in a stacked image often exhibit spatial sparsity. In particular, in the case of local overload or uneven force, risk areas may be concentrated in certain regions. While the conventional Squeeze-and-Excitation module can enhance inter-channel feature responses, it ignores the impact of spatial position on image importance and cannot effectively improve the response of risk areas when dealing with spatial sparsity, which can easily lead to insufficient sensitivity of the model in specific regions. Therefore, the present invention designs a spatially aware attention enhancement module to enhance the attention of the feature tensor.

[0099] Specifically, the multi-scale enhanced feature tensor output by the multi-scale feature enhancement module is globally average pooled to obtain a global feature vector that reflects the overall response strength of each channel. The global feature vector is then nonlinearly transformed through a fully connected layer to generate a global weight factor.

[0100] The Laplacian operator is applied to the multi-scale enhanced feature tensor to calculate the second-order gradient map for each channel, highlighting rapidly changing areas in the image, such as the edges of cracks in the pile. Through convolution, the second-order gradient map is interactively transmitted across channels to generate a gradient weight factor with the same dimension as the global weight factor.

[0101] The global weight factor is added to the gradient weight factor, and the channel attention enhancement vector is generated through the Sigmoid activation function. The larger the value of the channel attention enhancement vector, the more important the risk feature of the corresponding channel is, as shown in formula (6):

[0102] ; (6)

[0103] in, represents the channel attention enhancement vector; Represents the Sigmoid activation function; represents a fully connected layer; represents global average pooling; Represents convolution processing; represents the Laplace operator.

[0104] Multiply the channel attention enhancement vector and the multi-scale enhancement feature tensor to enhance the response of the key channel, and residually connect the multi-scale enhancement feature tensor to retain the original features, and finally obtain the attention enhancement feature tensor, as shown in formula (7):

[0105] ; (7)

[0106] in, Represents the multi-scale enhanced feature tensor; Represents channel multiplication.

[0107] In some embodiments, in deep neural networks, deep features often lose image details, while shallow features, while retaining more details, lack sufficient semantic information. Traditional feature pyramid networks use a fixed weighted sum approach for feature fusion, but for stacked images of different sizes, this fixed weighted sum strategy often leads to unsatisfactory fusion effects and fails to achieve efficient information transfer between different levels. Therefore, the present invention designs a learnable spatial weight fusion mechanism that achieves dynamic fusion by calculating the spatial weights of features at different levels.

[0108] Specifically, the fused feature tensor output by the spatial-aware attention enhancement module is divided into a multi-level pyramid. For example, three layers are used, with resolutions of 1 / 4, 1 / 8, and 1 / 16 of the input image, respectively. Shallow layers retain more details, while deeper layers contain stronger semantic information. Features from adjacent layers are upsampled to achieve a uniform resolution.

[0109] The features of the target layer are concatenated with the features of the adjacent layer after upsampling (i.e., after resolution alignment) along the channel axis to obtain a joint feature containing cross-layer information. The joint feature is convolved to extract the cross-layer spatial dependency of each pixel position. The cross-layer spatial dependency is then normalized by the Softmax function according to the layer dimension to obtain the pixel-by-pixel spatial weight, as shown in formula (8):

[0110] ; (8)

[0111] in, Representation level The spatial weight of Indicates along Dimensionally normalized Softmax function; Represents 3×3 convolution; Representation level The attention-enhanced feature tensor of ; Represents the resolution adjustment function; Representation level The attention-enhanced feature tensor of .

[0112] Based on the spatial weights and the features of the adjacent layers after upsampling, a weighted sum is performed to obtain the fused feature tensor, as shown in formula (9):

[0113] ; (9)

[0114] in, Representation level The fused feature tensor of .

[0115] In some embodiments, deformation characteristics in load images are important indicators for risk assessment. However, conventional residual modules lack sufficient sensitivity to small displacements, making them unable to effectively capture subtle changes caused by material compression. Furthermore, the common fixed convolution kernel design makes it difficult to accurately capture the gradual evolution of deformation. Therefore, the present invention designs a deformation gradient enhanced residual module that extracts image deformation information through a gradient operator.

[0116] Specifically, the horizontal gradient operator and the vertical gradient operator are applied to the fused feature tensor output by the dynamic feature pyramid fusion module to obtain the horizontal gradient feature map and the vertical gradient feature map. The horizontal gradient feature map and the vertical gradient feature map are spliced ​​through channels and merged into a three-dimensional tensor to retain the complete deformation direction information.

[0117] The gradient amplitude is calculated based on the horizontal gradient operator and the vertical gradient operator to reflect the deformation intensity of each pixel position, as shown in formula (10):

[0118] ; (10)

[0119] in, represents the gradient amplitude; represents the horizontal gradient operator; Represents the vertical gradient operator.

[0120] The deformation sensitive mask is calculated based on the gradient amplitude, as shown in formula (11):

[0121] ; (11)

[0122] in, represents a deformation-sensitive mask; is the deformation sensitive mask influencing factor, which controls the enhancement strength of the deformation area. Preferably, .

[0123] The three-dimensional tensor is filtered using the ReLU activation function to retain the positive response in the deformation direction, and then multiplied by the deformation-sensitive mask to further enhance the features of the high deformation area to obtain the gradient enhancement feature, as shown in formula (12):

[0124] ; (12)

[0125] in, represents the gradient enhancement feature; Represents the ReLU activation function; Indicates channel splicing; Represents channel multiplication.

[0126] The gradient enhancement feature is convolved and then connected with the fusion feature tensor residual to obtain the residual enhancement feature tensor, as shown in formula (13):

[0127] ; (13)

[0128] in, represents the residual enhanced feature tensor; Represents a 3×3 convolution.

[0129] In some embodiments, the classifier includes multiple fully connected layers and two output layers, wherein one output layer is used to output a risk level class probability vector, and the other output layer is used to output a tilt angle estimation value.

[0130] The residual enhanced feature tensor output by the encoder is input into the classifier. Neurons in multiple fully connected layers learn the features in the feature tensor and gradually extract high-level features related to the load risk level and tilt angle. These features are then input to the output layer. The output layer corresponding to the risk level uses a softmax activation function to map the input features to a probability distribution. The probability value for each category represents the likelihood that the image belongs to that risk level. The output layer corresponding to the tilt angle uses a linear activation function to directly output a continuous value representing the predicted tilt angle.

[0131] In step S103, the initial model is trained using the training sample set. Exemplarily, in the initialization stage, all convolutional layers are initialized using He normal distribution, the fully connected layers are initialized using Xavier uniform distribution, and the learning rate is set to 0.001.

[0132] First, a multi-task collaborative training mechanism is constructed.

[0133] In each training batch, the mean of the risk levels of all samples is calculated. The mean is mapped and normalized using the Sigmoid function to obtain the risk weight factor, as shown in formula (14):

[0134] ; (14)

[0135] in, represents the risk weight factor; Represents the Sigmoid function; Indicates the current batch The mean risk level.

[0136] Maintain a sliding window. For example, the length of the sliding window is Record the estimated tilt angle of the most recent T frames and calculate the change between adjacent frames. Calculate the continuity weight factor based on the change between adjacent frames, as shown in formula (15):

[0137] ; (15)

[0138] in, represents the continuity weight factor; Indicates the change between adjacent frames; Indicates the sliding window length.

[0139] Construct the classification loss of the risk level class probability vector and the risk level in the label, construct the regression loss of the tilt angle estimation value and the tilt angle in the label, and weight the classification loss and regression loss based on the dynamically adjusted loss weight (risk weight factor and continuity weight factor) to obtain the joint loss, as shown in formula (16):

[0140] ; (16)

[0141] in, represents the classification loss; represents the regression loss; represents the cross entropy classification loss; represents the predicted risk level class probability vector; Indicates the true risk level in the label; represents the mean square error loss; represents the estimated value of the predicted tilt angle; Indicates the actual tilt angle of the label.

[0142] With the goal of minimizing the joint loss, the initial model is iteratively optimized. For example, the AdamW optimizer is used to back-propagate and update the parameters.

[0143] In some embodiments, monitoring of stacked images in real-world environments is susceptible to interference from rain, fog, dust, and other factors. Traditional data augmentation methods are generally unable to simulate these environmental changes, resulting in insufficient robustness of the model when used in real-world scenarios. Conventional image augmentation techniques augment data through random cropping, rotation, and other operations, but these methods fail to effectively account for environmental degradation factors, particularly image degradation under complex weather conditions such as rain, fog, and dust, thereby affecting the model's ability to recognize images in complex environments. Therefore, the present invention enhances the diversity of training samples by simulating environmental interference.

[0144] Specifically, a physical model is constructed based on the physical characteristics of various environments (such as rainy and foggy environments, dusty environments, etc.).

[0145] Simulated rain and fog environment: randomly generated The coordinates of the raindrop center are calculated, and the distance between each pixel and the center of all raindrops is calculated. The rain streak intensity is obtained by Gaussian function attenuation. Then, the sine function is combined to simulate the periodic arrangement of raindrops to generate a rain streak pattern with physical realism. Therefore, the rain streak generation function is shown in formula (17):

[0146] ; (17)

[0147] in, represents the rain streak generating function; Indicates the number of randomly generated raindrops; Represents pixel coordinates; Indicates the raindrop center; Indicates the raindrop influence radius; represents the spatial frequency, preferably, ; Represents the raindrop falling angle parameter, used to simulate rain and fog effects.

[0148] Simulate dust environment: generate mean 0 and variance Gaussian noise (such as 0.001) is superimposed on the loading image to simulate the scattering effect of dust particles in the air on light.

[0149] Based on the simulated rain and fog environment and dust environment, the rain and fog effects and dust effects are obtained by setting adjustable parameters, and a physical model is constructed, as shown in formula (18):

[0150] ; (18)

[0151] in, Represents a physical model; represents the environmental parameter vector; represents the rain and fog effect, where is the rain and fog effect intensity parameter, reflecting the intensity of the impact of rain and fog on the image. Preferably, ; represents the dust effect, where is the dust effect intensity parameter, reflecting the intensity of the impact of dust on the image. Preferably, .

[0152] The environmental interference term is calculated based on the physical model and added to the loaded image to generate adversarial enhancement samples, as shown in formula (19):

[0153] ; (19)

[0154] in, Represents the generated adversarial enhancement samples, which are used to enhance the robustness of the model to environmental interference; is the perturbation intensity, which controls the intensity of the generated adversarial enhancement samples. Preferably, .

[0155] In some embodiments, during the above-mentioned iterative optimization of the initial model, the generated adversarial enhancement samples are called every 5 iteration cycles, and the adversarial enhancement samples are added to the original training sample set to train the initial model to enhance the robustness of the model.

[0156] In some embodiments, the risk areas in the stacked image are usually very sparse, especially when there is local overload or uneven force, the performance of the risk area is more localized, which often leads to the gradient explosion problem. During the training process, the conventional global gradient clipping method will destroy the feature distribution, resulting in the loss of important features. Therefore, the present invention designs an adaptive gradient clipping strategy based on the spatial attention map, which dynamically adjusts the gradient clipping threshold according to the channel attention enhancement vector calculated in the spatial perception attention enhancement module, allowing larger gradient changes in the risk area, while strictly constraining the gradient in the background area, thereby balancing the stability of training and the efficiency of feature learning.

[0157] Specifically, based on formula (6), the channel attention enhancement vector is obtained ,The larger the value of the channel attention enhancement vector, the higher the probability that the corresponding channel contains risk features.

[0158] The gradient amplitude is calculated based on formula (10), which reflects the intensity of feature change at each spatial position.

[0159] The channel dimension and spatial dimension are weighted averaged to obtain the joint expected value of attention-gradient.

[0160] The gradient clipping threshold is generated based on the preset basic threshold and the attention-gradient joint expected value, as shown in formula (20):

[0161] ; (20)

[0162] in, Represents the gradient clipping threshold; represents the basic threshold, preferably, ; Represents the sensitivity coefficient, which controls the sensitivity of the gradient clipping strategy. Preferably, ; Represents the expected operation.

[0163] The L2 norm of the gradient of each parameter of the initial model is calculated and compared with the gradient clipping threshold. If the L2 norm is greater than the gradient clipping threshold, it means that the gradient is too large (may contain noise or come from the background area), then the gradient is clipped based on the gradient clipping threshold and the gradient amplitude is scaled to the threshold level; otherwise, the original gradient is retained to ensure that the effective gradient update in the risk area is not suppressed, as shown in formula (21):

[0164] ; (twenty one)

[0165] in, represents the clipped gradient; represents the original gradient.

[0166] In some embodiments, the training termination condition adopts a dual judgment criterion, including:

[0167] Monitor the validation set indicators and stop the iteration when the risk classification accuracy fluctuates less than 0.5% for 10 consecutive iteration cycles.

[0168] Alternatively, if the training set loss continues to decrease while the validation set loss increases, and the early stopping counter reaches 20 iterations, training is forced to terminate and roll back to the model corresponding to the optimal parameters.

[0169] Based on the training method of step S103, a load risk detection model is finally obtained, which is used to detect the risk level and tilt angle of the load.

[0170] In step S104 , the load image to be detected is pre-processed and then input into the load risk detection model trained in steps S101 to S103 to obtain the risk level and tilt angle of the load image to be detected.

[0171] In some embodiments, an alarm threshold is preset. For example, when the risk level probability value is greater than 0.7 or the tilt angle estimation value is greater than 25 degrees, a risk alarm is generated.

[0172] Corresponding to the above method, the present invention also provides a stacking risk detection system, which implements the steps of the above method when executed.

[0173] The present invention will be further described below based on specific experimental data.

[0174] In order to evaluate the effects of different image normalization methods on the accuracy of heap load risk classification, the experiment compared global mean normalization, histogram equalization, contrast-constrained adaptive histogram equalization (CLAHE) and the illumination-texture decoupling normalization method proposed in this paper. Figure 2As shown in the figure, the impact of different normalization methods on the risk identification accuracy is demonstrated. Among them, the column height represents the average classification accuracy, the error bar shows the data fluctuation range, and the scatter points represent the original data distribution of each experiment. The experimental results show that the illumination-texture decoupling normalization method of the present invention is significantly better than the traditional technology. The column height is significantly higher and the error range is narrower, indicating that the innovative edge area local mean calculation and texture enhancement mask mechanism of the present invention effectively eliminates highlight interference and strengthens key detail features such as cracks, enabling the model to more accurately identify risk areas.

[0175] In order to analyze the performance stability of different models in the anti-disturbance environment, the disturbance intensity is gradually increased to simulate the rain, fog, dust and other interferences in the real scene, including ResNet-50, VGG-16, conventional CNN model and the stacking risk detection model trained by the present invention. Figure 3 As shown, the horizontal axis represents the disturbance intensity, and the vertical axis represents the classification accuracy. Marked points of different shapes represent the actual test results of different models under various disturbance intensities, and the smooth curve shows the overall trend. The stacking risk detection model (triangle mark) trained by the present invention always maintains the highest accuracy with the smallest decline, especially in a strong disturbance environment. This verifies the effectiveness of the physical model-driven adversarial enhancement sample generation method and the attention-guided gradient clipping strategy in the technology of the present invention. The former enhances the model's adaptability by accurately simulating environmental degradation factors, while the latter ensures that feature learning in key risk areas is not disturbed. The gently declining trend of the broken line proves that the technology of the present invention has excellent environmental robustness.

[0176] To demonstrate the superiority of the multi-scale feature enhancement module designed in this invention (which employs a dual-pathway feature pyramid structure consisting of texture and geometry pathways) in multi-scale feature extraction and enhancement, experiments compared the feature extraction capabilities of conventional convolution, deformable convolution, and the multi-scale feature enhancement module of this invention, focusing on analyzing performance at different scales (0.1-5 mm). Figure 4(a) shows a comparison of the feature extraction capabilities of different feature enhancement modules in one embodiment of the invention; Figure 4(b) shows a comparison of the response intensity of key regions of different feature enhancement modules in one embodiment of the invention. The experimental results show that the multi-scale feature enhancement module of this invention maintains optimal feature extraction capabilities at key scales, such as microcracks (0.1 mm scale) and overall pile morphology (5 mm scale), demonstrating the significant advantages of the dual-pathway design. Furthermore, the curves for the response intensity indicators of key risk areas (bearing zones and inclined surfaces) of the multi-scale feature enhancement module of this invention consistently rank at the top, indicating that the synergistic mechanism of dilated and deformable convolution effectively enhances feature perception capabilities in risk-sensitive areas.

[0177] To verify the adaptability of the dynamic task weight allocator to high-risk scenarios, the performance of a fixed weight strategy and the dynamic task weight strategy of the present invention were compared under low, medium, and high risk levels. The dynamic task weight strategy of the present invention includes: the weight of the classification task (risk level classification) is dynamically adjusted via a risk weight factor (the Sigmoid normalization of the current batch's risk level mean), and the weight of the regression task (tilt angle estimation) is dynamically optimized via a continuity weight factor (the change in tilt angle between adjacent frames). Figure 5(a) shows a comparison of the risk classification accuracy using the fixed weight strategy and the dynamic task weight strategy in one embodiment of the present invention; Figure 5(b) shows a comparison of the tilt angle prediction error using the fixed weight strategy and the dynamic task weight strategy in one embodiment of the present invention. The experimental results show that as the risk level increases, the classification accuracy of the dynamic task weight strategy of the present invention remains stable at a high level, while the tilt angle prediction error shows a significant downward trend. This demonstrates that the dynamic weight mechanism successfully addresses the prediction bias caused by the scarcity of high-risk samples, substantially improving the decision-making reliability of the model in critical risk scenarios.

[0178] Corresponding to the above method, the present invention also provides an electronic device, which includes a computer device, the computer device includes a processor and a memory, the memory stores computer instructions, and the processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the electronic device implements the steps of the method described above.

[0179] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program implements the steps of the aforementioned edge computing server deployment method. The computer-readable storage medium can be a tangible storage medium, such as a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, floppy disk, hard disk, removable storage disk, CD-ROM, or any other form of storage medium known in the art.

[0180] It should be understood by those skilled in the art that the various exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether to implement the system in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention. When implemented in hardware, it may be, for example, an electronic circuit, an application-specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments used to perform the required tasks. The programs or code segments can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via a data signal carried in a carrier wave.

[0181] It should be understood that the present invention is not limited to the specific configurations and processes described above and illustrated in the figures. For the sake of brevity, a detailed description of known methods is omitted. In the above embodiments, several specific steps are described and illustrated as examples. However, the method of the present invention is not limited to the specific steps described and illustrated. Those skilled in the art may make various changes, modifications, and additions, or change the order of the steps after understanding the spirit of the present invention.

[0182] In the present invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or replace features of other embodiments.

[0183] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A method for enhancing a stacked image, characterized in that: The method comprises: Preprocessing the images of the pile load collected under different environmental conditions and adding labels to construct a training sample set; the labels include risk level and tilt angle; An initial model of an encoder-classifier two-stage architecture is constructed; the initial model takes the stacked image as input, and the encoder extracts and fuses the multi-scale features of the stacked image through a multi-scale feature enhancement module; attention is enhanced through a spatial perception attention enhancement module; attention-enhanced feature tensors of different levels are adaptively weighted and fused through a dynamic feature pyramid fusion module; deformation feature expression is enhanced through a deformation gradient enhancement residual module, and the output residual enhancement feature tensor is handed over to the classifier to output a risk level class probability vector and a tilt angle estimate; The initial model is trained using the training sample set, and a joint loss of risk level classification loss and tilt angle regression loss is constructed based on dynamic task weights for iterative optimization. Every time a preset number of iterations is reached, environmental interference items are added to the load image to generate adversarial enhancement samples that are added to the training sample set. Based on the channel attention enhancement vector calculated in the spatial perception attention enhancement module, the gradient update parameters are adaptively clipped to obtain a load risk detection model. Acquire the image of the pile load to be detected; input the preprocessed image of the pile load to be detected into the pile load risk detection model to obtain the risk level and tilt angle of the image of the pile load to be detected.

2. The method for enhancing the image according to claim 1, characterized in that: Preprocessing of the pile load images collected under different environmental conditions, including: Calculating a local mean and a local standard deviation of the stacked image based on the edge region; Converting the stacked image into a grayscale image; calculating the local variance of the grayscale image to obtain a texture enhancement mask; The stacked image is subjected to normalized enhancement processing based on the local mean and the local standard deviation, and is combined with the texture enhancement mask to obtain a preprocessed stacked image tensor.

3. The method for enhancing the image according to claim 1, wherein: The multi-scale features of the stacked image are extracted and fused by a multi-scale feature enhancement module, including: The multi-scale feature enhancement module adopts a dual-pathway feature pyramid structure including a texture path and a geometric path; The texture path uses multi-layer dilated convolution to extract texture features of the stacked image; The geometric path uses deformable convolutional layers and pooling layers to extract geometric features of the stacked image; The texture features and the geometric features are fused to obtain a multi-scale enhanced feature tensor.

4. The method for enhancing the image according to claim 1, wherein: Attention enhancement is performed through the spatial perception attention enhancement module, including: Performing global average pooling and nonlinear transformation processing on the multi-scale enhanced feature tensor output by the multi-scale feature enhancement module to obtain a global weight factor; Based on the Laplacian operator, the second-order gradient map of each channel of the multi-scale enhanced feature tensor is calculated and interacted across channels to generate a gradient weight factor; Add the global weight factor to the gradient weight factor and generate a channel attention enhancement vector through a Sigmoid activation function; The channel attention enhancement vector and the multi-scale enhancement feature tensor are multiplied, and the multi-scale enhancement feature tensor is residually connected to obtain the attention enhancement feature tensor.

5. The method for enhancing the image according to claim 1, wherein: The dynamic feature pyramid fusion module performs adaptive weighted fusion of attention-enhanced feature tensors at different levels, including: Dividing the fused feature tensor output by the spatial perception attention enhancement module into a multi-level pyramid; upsampling the features of adjacent levels to obtain upsampled features; The target layer feature is concatenated with the upsampled feature to obtain a joint feature; the cross-layer spatial dependency of each pixel position in the joint feature is extracted; the cross-layer spatial dependency is normalized by the layer dimension using the Softmax function to obtain a spatial weight; The spatial weight and the upsampled feature are weighted and summed to obtain a fused feature tensor.

6. The method for enhancing the image according to claim 1, wherein: The deformation feature expression is enhanced by the deformation gradient enhanced residual module, and the residual enhanced feature tensor is output, including: Based on a horizontal gradient operator and a vertical gradient operator, a horizontal gradient feature map, a vertical gradient feature map, and a gradient amplitude of the fused feature tensor output by the dynamic feature pyramid fusion module are calculated; the horizontal gradient feature map and the vertical gradient feature map are merged into a three-dimensional tensor; and a deformation-sensitive mask is calculated based on the gradient amplitude; The filtered three-dimensional tensor is multiplied by the deformation-sensitive mask to obtain a gradient-enhanced feature; the gradient-enhanced feature is convolved and connected with the residual of the fused feature tensor to obtain the residual-enhanced feature tensor.

7. The method for enhancing the image according to claim 1, characterized in that: Whenever a preset number of iterations is reached, an environmental interference term is added to the stacked image to generate an adversarial enhancement sample, including: Constructing a physical model based on physical characteristics of various environments, wherein the environments include at least a rainy and foggy environment and a dusty environment; The environmental interference term is calculated based on the physical model and added to the loaded image to generate the adversarial enhancement sample.

8. The method for enhancing the image according to claim 1, wherein: Adaptively clipping gradients based on the channel attention enhancement vector calculated in the spatial perception attention enhancement module, including: Calculating a gradient magnitude based on a horizontal gradient operator and a vertical gradient operator; calculating a weighted average of the channel attention enhancement vector and the gradient magnitude to obtain an attention-gradient joint expected value; Generate a gradient clipping threshold based on a preset base threshold and the attention-gradient joint expected value; Calculate the L2 norm of the gradient of each parameter of the initial model and compare it with the gradient clipping threshold. If the L2 norm is greater than the gradient clipping threshold, perform gradient clipping based on the gradient clipping threshold; otherwise, retain the original gradient.

9. The method for enhancing the image according to claim 1, wherein: Based on the dynamic task weights, a joint loss of risk level classification loss and tilt angle regression loss is constructed, including: In each training batch, the mean of the risk level probability vectors of all samples is calculated, and the Sigmoid function is used to map and normalize them to obtain the risk weight factor; Obtain the tilt angle prediction value within a preset time period, calculate the change in tilt angle between adjacent frames, and calculate the continuity weight factor; Construct the classification loss of the risk level class probability vector and the risk level in the label, construct the regression loss of the tilt angle estimate and the tilt angle in the label, and weight the classification loss and the regression loss based on the risk weight factor and the continuity weight factor to obtain a joint loss.

10. The method for enhancing the image according to claim 1, wherein: Iterative optimization based on joint loss, including: The parameters of the initial model were updated using back propagation of the AdamW optimizer, and the initial learning rate was set to 0.001.

Images