Container damage detection method

Abstract

The invention discloses a container damage detection method, which comprises the following steps of: acquiring a container image or video data sample, and constructing an initial data set; performing data enhancement preprocessing on the container image or video data sample in the initial data set to obtain a final data set; using the final data set to train an improved Res2Net model, and generating a container damage detection model; the improved Res2Net model comprises an improved residual network model, an additional path fusion enhancement module, a self-adaptive activation function and an attention mechanism; the improved residual network model comprises introduction of jump connection and addition of input to output; and for a to-be-detected container, acquiring an image or a video of the to-be-detected container, and inputting the image or the video into the container damage detection model to obtain a damage detection result.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision and object recognition, and in particular to a container damage detection method. Background Art

[0002] In recent years, China's port container transportation industry has developed rapidly. In fully automated terminals, information perception systems, operation control systems, dispatching and command systems, and other systems are becoming increasingly sophisticated, but the automatic container inspection module has not yet been fully developed.

[0003] Damaged container inspections are an inevitable task for containers entering port terminals, primarily to prevent disputes between shipping companies and terminals. Common types of damage to port containers include dents, convexities, holes, breakage, distortion, scratches, broken markings, damaged container doors, and deformed container door railings. Currently, there are no corresponding research results based on vision technology to detect and identify container damage, but damage state detection has a wide range of applications. Due to the small proportion of damaged containers in the total number of port containers, the sample size under these working conditions is small, and the location, size, and type of damage are uncertain. In addition, due to changes in background, they are easily missed and misdetected. Traditional methods and previous neural network frameworks are not very effective in detecting such conditions. Summary of the Invention

[0004] The purpose of the present invention is to provide a container damage detection method to address the problems existing in the above-mentioned prior art.

[0005] The technical solution to achieve the purpose of the present invention is: a container damage detection method, the method comprising the following steps:

[0006] Step 1: Collect container image or video data samples to build an initial dataset;

[0007] Step 2: performing data enhancement preprocessing on the container image or video data samples in the initial dataset to obtain a final dataset;

[0008] Step 3: Using the final dataset to train an improved Res2Net model to generate a container damage detection model; the improved Res2Net model includes an improved residual network model, adding a path fusion enhancement module, an adaptive activation function, and an attention mechanism; the improved residual network model includes introducing a skip connection to add the input to the output;

[0009] Step 4: For the container to be inspected, collect its image or video and input it into the container damage detection model to obtain the damage detection result.

[0010] Furthermore, in step 3, the improved residual network model is located in the convolutional layer of the Res2Net model for extracting features from image or video data samples, and the path fusion enhancement module, adaptive activation function, and CA attention mechanism are located in the pooling layer of the Res2Net model.

[0011] Furthermore, the improved residual network model in step 3 includes five layers of residual modules, which represent features at multiple scales and increase the receptive field range of each network layer;

[0012] The first-layer residual module uses three 3*3 convolution kernels, with 64 channels and a stride of 2. After standardization and pooling, it is input into the second-layer residual module; the second-layer residual module sets 3 Res2Net modules, increasing the number of channels from 64 to 256; the third-layer residual module sets 4 Res2Net modules, increasing the number of channels to 512; the fourth-layer residual module sets 23 Res2Net modules, increasing the number of channels to 1024; the last layer sets 3 Res2Net modules, increasing the number of channels to 2048.

[0013] Furthermore, the residual module includes multiple groups of convolution kernels, each group of convolution kernels includes a serialized 1*1 convolution kernel, a 3*3 convolution kernel and a 1*1 convolution kernel; the input of the residual module is: a plurality of groups of input feature maps, which are obtained by dividing the image or video data samples in the final data set into a plurality of groups;

[0014] A set of convolution kernels first extracts features from a set of input feature maps, then combines the output feature maps with another set of input feature maps and extracts features through another set of convolution kernels; repeats this process several times until all input feature maps are processed; finally, the output feature maps of all groups are spliced ​​together and feature fusion is performed through a 1×1 convolution kernel.

[0015] Furthermore, the path fusion enhancement module in step 3 enhances FPN2-FPN5 generated by the Mask-RCNN algorithm using FPN, specifically including: converting a feature map FPN2 output by FPN into FPN21 through maximum pooling, and convolving FPN21 with another feature map FPN3 to obtain FPN31, where the convolution is a 3×3 convolution kernel; similarly, convolving FPN31 with another feature map FPN4 to obtain FPN41, and convolving FPN41 with another feature map FPN5 to obtain FPN51;

[0016] Except for the first layer, FPN21, which directly outputs as output 1, the following three layers fuse the input and output of the previous layers as output 2, output 3, and output 4, respectively; output 2 is obtained by convolution fusion of FPN3 and FPN31, output 3 is obtained by convolution fusion of FPN4 and FPN41, and output 4 is obtained by convolution fusion of FPN5 and FPN51.

[0017] Furthermore, the adaptive activation function in step 3 is:

[0018]

[0019] Among them, the adaptive activation function is an L-layer structure, Φ l is the function matrix of the lth layer, x is the given input vector, and KAN(x) is the output of the adaptive activation function for the given input vector x;

[0020]

[0021] Furthermore, the learning of each sub-activation function, i.e., the local B-spline basis function, includes:

[0022] (1) Residual activation function

[0023] The sub-activation function is expressed as:

[0024] φ(x)=w(b(x)+spline(x))

[0025] Here, the subactivation function is represented by φ(x), b(x) represents the basis function, and spline(x) represents the spline function;

[0026] in,

[0027] b(x)=x / (1+e -x )

[0028] spline(x) is parameterized as a linear combination of B-splines B i (x):

[0029]

[0030] Where c i is a trainable matrix, w is a weight matrix, which is used to control the overall size of the sub-activation function; B i (x) represents the i-th B-spline function, i ranges from 1 to N, and N is the total number of B-spline functions;

[0031] (2) Initialization

[0032] Initialize each sub-activation function to the spline curve spline(x)≈0;

[0033] w is initialized according to the Xavier initialization criterion, where the Xavier initialization criterion is as follows:

[0034]

[0035] In the formula, each element in the weight matrix w is initialized to obey the mean 0 and the variance is Normal distribution, where n in is the number of input nodes, n out is the number of output nodes;

[0036] (3) Update

[0037] The update rule of the sub-activation function is as follows:

[0038]

[0039] Where φ'(x) is the updated sub-activation function, C represents the updated function m The order norm is used to measure the magnitude of the derivative of the function φ'(x) at each order. G represents the size of the spline grid, that is, the number of spline functions in each layer. m is the order of the derivative of the function. β is the maximum value less than or equal to m. sup represents the supremum of the set.

[0040] In another aspect, a container damage detection system is provided, comprising:

[0041] The first module is used to collect container image or video data samples and build an initial data set;

[0042] The second module is used to perform data enhancement preprocessing on the container image or video data samples in the initial data set to obtain the final data set;

[0043] The third module is configured to train an improved Res2Net model using the final dataset to generate a container damage detection model; the improved Res2Net model includes an improved residual network model, which adds a path fusion enhancement module, an adaptive activation function, and an attention mechanism; the improved residual network model includes introducing a skip connection to add the input to the output;

[0044] The fourth module is used to collect images or videos of the container to be inspected and input them into the container damage detection model to obtain damage detection results.

[0045] Compared with the prior art, the present invention has the following significant advantages:

[0046] (1) Improved detection accuracy and efficiency: By combining an improved residual network, data augmentation technology, adaptive activation function, CA attention mechanism, and path fusion enhancement, the present invention can significantly improve the accuracy of container damage detection. The combined use of these technologies reduces false detections and missed detections, enabling the model to quickly and accurately inspect the surface condition of containers in unmanned operation mode.

[0047] (2) Enhanced model robustness: By introducing a variety of data enhancement methods, such as random Gaussian blur, image flipping, contrast variation, etc., the model’s adaptability to different environments and conditions is enhanced. This enables the model to maintain high performance stability in the face of changing practical application scenarios.

[0048] (3) Adaptability to various damage types: The technology can identify and classify various types of container damage, including but not limited to dents, bulges, holes, cracks, distortions, scratches, etc. This broad damage recognition capability is achieved through sophisticated image processing and pattern recognition technology, ensuring that all types of damage can be effectively detected and improving the comprehensiveness of the inspection.

[0049] The present invention is further described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 Schematic diagram of the principle of the container damage detection method of the present invention.

[0051] Figure 2 Schematic diagram of the improved residual network model.

[0052] Figure 3 Schematic diagram of the path fusion enhancement module.

[0053] Figure 4 Schematic diagram of the adaptive activation function of MLP.

[0054] Figure 5 Schematic diagram of the CA attention mechanism structure.

[0055] Figure 6 This is a comparison chart of the visualization results of container damage detection, where Figure 6 (a1) to (a3) ​​are the visualization results of damage detection of containers in scenarios 1, 2, and 3 using the original residual network. Figure 6 (b1) to (b3) are respectively the visualization results of damage detection of containers in scenarios 1, 2 and 3 using the method of the present invention. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0057] It should be noted that if there are descriptions involving "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of such features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in this field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0058] Container body damage inspection is an inevitable inspection task for containers entering the port terminal. Since damaged containers account for a small proportion of the total number of containers in the port, the sample size is small under this working condition, and the location, size and category of the damage are uncertain. In addition, due to changes in the background, it is easy to be missed and misdetected. Traditional methods and previous neural network frameworks are not very effective in detecting such conditions. To solve this problem, the present invention optimizes the Res2Net model, improves the residual network, and adds data enhancement, path fusion enhancement, adaptive activation function and CA attention mechanism. Multi-scale features are added, the utilization of position information is improved, the loss of upsampling information is reduced, and the efficiency of existing samples is improved. It aims to reduce omissions and false detections in damage identification, replace manual inspection, and be able to quickly and accurately inspect the surface condition of containers in unmanned operation mode.

[0059] In one embodiment, combined Figure 1 , provides a container damage detection method, the method comprising the following steps:

[0060] Step 1: Collect container image or video data samples to build an initial dataset;

[0061] Step 2: performing data enhancement preprocessing on the container image or video data samples in the initial dataset to obtain a final dataset;

[0062] Step 3: Using the final dataset to train an improved Res2Net model to generate a container damage detection model; the improved Res2Net model includes an improved residual network model, adding a path fusion enhancement module, an adaptive activation function, and an attention mechanism; the improved residual network model includes introducing a skip connection to add the input to the output;

[0063] Step 4: For the container to be inspected, collect its image or video and input it into the container damage detection model to obtain the damage detection result.

[0064] Furthermore, in one embodiment, the data enhancement in step 2 is specifically:

[0065] 50% of the data samples in the initial dataset are flipped horizontally and vertically, and random Gaussian blur is introduced.

[0066] or:

[0067] The following enhancement operations are set: (1) horizontal flip; (2) vertical flip; (3) random Gaussian blur introduction; (4) image scaling; (5) image contrast change; (6) image sharpening; (7) input pixel value inversion, that is, the original value is v and the change value is 255-v; (8) random deformation of the image; (9) image movement to the left, right, up or down; (10) image rotation clockwise or counterclockwise;

[0068] Randomly select any of the 10 enhancement operations mentioned above to process image or video data samples.

[0069] Here, in order to identify container damage, cracks, etc., data augmentation was performed on the existing samples to increase the number of training samples and improve the generalization ability of the network. At the same time, noisy data was introduced to improve the robustness of the model.

[0070] Furthermore, in one embodiment, in step 3, the improved residual network model is located in the convolution layer of the Res2Net model, for performing feature extraction on image or video data samples, and the path fusion enhancement module, adaptive activation function and attention mechanism are located in the pooling layer of the Res2Net model.

[0071] Here, in some embodiments, the improved residual network model in step 3 includes five layers of residual modules, which represent features at multiple scales and increase the receptive field range of each network layer; the architecture is shown in Table 1.

[0072] Table 1 Architecture

[0073]

[0074] The first-layer residual module uses three 3*3 convolution kernels, with 64 channels and a stride of 2. After standardization and pooling, it is input into the second-layer residual module; the second-layer residual module sets 3 Res2Net modules, increasing the number of channels from 64 to 256; the third-layer residual module sets 4 Res2Net modules, increasing the number of channels to 512; the fourth-layer residual module sets 23 Res2Net modules, increasing the number of channels to 1024; the last layer sets 3 Res2Net modules, increasing the number of channels to 2048.

[0075] Here, in some embodiments, combined Figure 2 The residual module includes multiple groups of convolution kernels, each group of convolution kernels includes a serialized 1*1 convolution kernel, a 3*3 convolution kernel and a 1*1 convolution kernel; the input of the residual module is: a plurality of groups of input feature maps, which are obtained by dividing the image or video data samples in the final data set into a plurality of groups;

[0076] A set of convolution kernels first extracts features from a set of input feature maps, then combines the output feature maps with another set of input feature maps and extracts features through another set of convolution kernels; repeats this process several times until all input feature maps are processed; finally, the output feature maps of all groups are spliced ​​together and feature fusion is performed through a 1×1 convolution kernel.

[0077] Here, the improved residual network of the present invention has the following characteristics:

[0078] (1) Alleviate the vanishing gradient problem: The vanishing gradient problem in deep networks is one of the main reasons for the increased difficulty of training. Residual networks introduce skip connections, which allow gradients to propagate directly to shallower layers. This allows strong gradients to be maintained in very deep networks, which helps network training.

[0079] (2) Accelerated convergence: Due to the presence of skip connections, residual networks can converge faster during training. This connection method directly adds the input to the output, which can be regarded as a correction to the output of each layer in the network. This allows the network to quickly reduce high-frequency errors in the early stages of training and improve training efficiency.

[0080] (3) Improve network performance: Studies have shown that residual networks can effectively increase network depth without causing overfitting problems. Deeper networks can capture more complex features, thereby achieving better performance in various image recognition tasks.

[0081] Furthermore, in one embodiment, the path fusion enhancement module in step 3 enhances FPN2-FPN5 generated by the Mask-RCNN algorithm using FPN, and combines Figure 3Specifically, it includes: converting a feature map FPN2 output by FPN into FPN21 through maximum pooling, and convolving FPN21 with another feature map FPN3 to obtain FPN31, where the convolution is a 3×3 convolution kernel; similarly, convolving FPN31 with another feature map FPN4 to obtain FPN41, and convolving FPN41 with another feature map FPN5 to obtain FPN51;

[0082] Except for the first layer, FPN21, which directly outputs as output 1, the following three layers fuse the input and output of the previous layers as output 2, output 3, and output 4, respectively; output 2 is obtained by convolution fusion of FPN3 and FPN31, output 3 is obtained by convolution fusion of FPN4 and FPN41, and output 4 is obtained by convolution fusion of FPN5 and FPN51.

[0083] By adopting the solution of this embodiment, candidate areas are extracted through FPN, and path fusion enhancement is introduced to improve the utilization of low-level information and the accuracy of position information without reducing high-level information, thereby enhancing the model's ability to identify damage.

[0084] Furthermore, in one embodiment, the adaptive activation function in step 3 is:

[0085]

[0086] Among them, the adaptive activation function is an L-layer structure, Φ l is the function matrix of the lth layer, x is the given input vector, and KAN(x) is the output of the adaptive activation function for the given input vector x;

[0087] The shape of the adaptive activation function is represented by an array of integers in matrix form:

[0088]

[0089] The learning of each sub-activation function, i.e., local B-spline basis function, includes:

[0090] (1) Residual activation function

[0091] The sub-activation function is expressed as:

[0092] φ(x)=w(b(x)+spline(x))

[0093] Here, the subactivation function is represented by φ(x), b(x) represents the basis function, and spline(x) represents the spline function;

[0094] in,

[0095] b(x)=x / (1+e -x )

[0096] spline(x) is parameterized as a linear combination of B-splines B i (x):

[0097]

[0098] Where c i is a trainable matrix, w is a weight matrix, which is used to control the overall size of the sub-activation function; B i (x) represents the i-th B-spline function, i ranges from 1 to N, and N is the total number of B-spline functions;

[0099] (2) Initialization

[0100] Initialize each sub-activation function to the spline curve spline(x)≈0;

[0101] w is initialized according to the Xavier initialization criterion, where the Xavier initialization criterion is as follows:

[0102]

[0103] In the formula, each element in the weight matrix w is initialized to obey the mean 0 and the variance is Normal distribution, where n in is the number of input nodes, n out is the number of output nodes;

[0104] (3) Update

[0105] The update rule of the sub-activation function is as follows:

[0106]

[0107] Where φ'(x) is the updated sub-activation function, C represents the updated function m The order norm is used to measure the magnitude of the derivative of the function φ'(x) at each order. G represents the size of the spline grid, that is, the number of spline functions in each layer. m is the order of the derivative of the function. β is the maximum value less than or equal to m. sup represents the supremum of the set.

[0108] The specific analysis is as follows:

[0109] MLP is inspired by the universal approximation theorem, but adaptive activation functions such as Figure 4As shown in Figure 2, it is inspired by the Kolmogorov-Anold representation theorem. Like MLP, the adaptive activation function has a fully connected structure. However, while MLP places fixed activation functions on nodes, the adaptive activation function places learnable activation functions on edges. Therefore, the adaptive activation function has no linear weight matrix at all: instead, each weight parameter is replaced by a learnable function parameterized as a spline. The nodes of the adaptive activation function simply sum the input signals without applying any nonlinearity. The specific steps are as follows:

[0110] If f is a continuous function of multiple variables over a bounded domain, then f can be written in terms of single variables and binary addition operations.

[0111] For smooth f:[0,1] n →R.

[0112]

[0113] Suppose there is an input-output pair {x i ,y i}, where we want to find f so that y for all data points i =f(x i ). Formula 3 shows that if a suitable single variable function φ can be found q,p and Φ q , the supervised learning task is completed.

[0114] The present invention designs a neural network with a clear parameterized equation. As shown in Formula 3, since all functions to be learned are single variable functions, each adaptive function can be parameterized as a B-spline curve with learnable coefficients of local B-spline basis functions (see Figure 4 Right). Now, we have a prototype of the adaptive activation function, whose calculation graph is specified by formula 3. The present invention designs a 2-layer adaptive activation function. In which the function φ q,p It has trainable parameters, as described above. In the Kolmogov-Anold theorem, the inner function takes n inputs and outputs 2n+1, while the outer function forms a layer with 2n+1 inputs and 1 output. The shape of the adaptive activation function is represented by an array of integers. Its matrix representation is shown in Equation 3.

[0115] Here, the adaptive activation function proposed in the present invention is applicable to low-dimensional functions, is easy to adjust locally, and can switch between different functions. The adaptive activation function can not only learn features, but also optimize these learned features very accurately.

[0116] Further preferably, in one of the embodiments, the attention mechanism in step 3 adopts the CA attention mechanism, which can effectively improve the accuracy of the model.

[0117] The CA attention mechanism encodes channel relationships and long-range dependencies through precise location information, which is also divided into two steps: coordinate information embedding and coordinate attention generation. Its specific structure is as follows: Figure 5 shown.

[0118] Coordinate information embedding part. Global pooling is often used in channel attention to globally encode spatial information as channel descriptors, making it difficult to preserve position information. To enable the attention module to capture long-range spatial dependencies with precise position information, global pooling is decomposed into a pair of one-dimensional feature encoding operations. Specifically, for input X, pooling kernels of size (H, 1) and (1, W) are first used to encode each channel along the horizontal and vertical coordinate directions. Therefore, the output of the cth channel with height h is expressed as follows.

[0119]

[0120] Similarly, the output of the cth channel with width w is expressed as follows.

[0121]

[0122] The above two transformations aggregate features along two spatial directions and return a pair of direction-aware attention maps. These two transformations also allow the attention module to capture long-range dependencies along one spatial direction and preserve precise position information along the other spatial direction, which helps the network more accurately locate the object of interest.

[0123] Next, in order to better utilize the representation with global receptive field and precise location information generated by the coordinate information embedding module above, a coordinate attention generation operation is designed, which generates an attention map that follows the following three criteria.

[0124] 1. For applications in mobile environments, this conversion should be as simple and efficient as possible;

[0125] 2. It can make full use of the captured location information to accurately locate the area of ​​interest;

[0126] 3. It should also be able to effectively capture the relationship between channels, which is fundamental.

[0127] The coordinate attention generation first concatenates the two feature maps generated by the previous module, and then uses a shared 1x1 convolution to transform F1, which can be expressed as follows:

[0128] f=δ(F1([z h ,z w ]))

[0129] The generated f is the intermediate feature map of spatial information in the horizontal and vertical directions.

[0130] Then split f into two separate tensors f along the spatial dimension h and f w . Then use two 1x1 convolution F h and F w The feature map f h and f w Transform to the same number of channels as input X and obtain the following result.

[0131] g h =σ(F H (f h ))g w =σ(F W (f w ))

[0132] Finally, g h and g w Expanding on this, as the attention weight, the final output of the CA module can be expressed as follows.

[0133]

[0134] At this point, the CA module completes both horizontal and vertical attention, and it is also a channel attention.

[0135] In one embodiment, a container damage detection system is provided, the system comprising:

[0136] The first module is used to collect container image or video data samples and build an initial data set;

[0137] The second module is used to perform data enhancement preprocessing on the container image or video data samples in the initial data set to obtain the final data set;

[0138] The third module is configured to train an improved Res2Net model using the final dataset to generate a container damage detection model; the improved Res2Net model includes an improved residual network model, which adds a path fusion enhancement module, an adaptive activation function, and an attention mechanism; the improved residual network model includes introducing a skip connection to add the input to the output;

[0139] The fourth module is used to collect images or videos of the container to be inspected and input them into the container damage detection model to obtain damage detection results.

[0140] The specific definitions of the container damage detection system can be found in the definitions of the container damage detection method above and will not be further elaborated here. Each module in the aforementioned container damage detection system may be implemented in whole or in part through software, hardware, or a combination thereof. Each module may be embedded in or independent of a processor in a computer device in hardware form, or may be stored in a computer device memory in software form, allowing the processor to invoke and execute the corresponding operations of each module.

[0141] In one embodiment, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the following is achieved:

[0142] Step 1: Collect container image or video data samples to build an initial dataset;

[0143] Step 2: performing data enhancement preprocessing on the container image or video data samples in the initial dataset to obtain a final dataset;

[0144] Step 3: Using the final dataset, train the improved Res2Net model to generate a container damage detection model; the improved Res2Net model includes an improved residual network model, adding a path fusion enhancement module, an adaptive activation function, and an attention mechanism; the improved residual network model includes introducing a skip connection to add the input to the output;

[0145] Step 4: For the container to be inspected, collect its image or video and input it into the container damage detection model to obtain the damage detection result.

[0146] For the specific limitations of each step, please refer to the limitations of the container damage detection method above, which will not be repeated here.

[0147] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the computer program implements:

[0148] Step 1: Collect container image or video data samples to build an initial dataset;

[0149] Step 2: performing data enhancement preprocessing on the container image or video data samples in the initial dataset to obtain a final dataset;

[0150] Step 3: Using the final dataset to train an improved Res2Net model to generate a container damage detection model; the improved Res2Net model includes an improved residual network model, adding a path fusion enhancement module, an adaptive activation function, and an attention mechanism; the improved residual network model includes introducing a skip connection to add the input to the output;

[0151] Step 4: For the container to be inspected, collect its image or video and input it into the container damage detection model to obtain the damage detection result.

[0152] For the specific limitations of each step, please refer to the limitations of the container damage detection method above, which will not be repeated here.

[0153] As a specific example, the present invention is further verified and explained in one of the embodiments.

[0154] This example uses the RoboFlow dataset, which contains 2139 labeled training images. This example uses the AdamW optimizer for training on an RTX 4070super GPU. The experimental environment is shown in Table 2.

[0155] Table 2 Experimental environment

[0156]

[0157] The training epoch is set to 200, and the batch size, initial learning rate, weight decay, and dropout rate are set to 32, 3×10, and 4×10, respectively. -4 , 1×10 -4 and 0.1.

[0158] The ablation experiment is carried out below.

[0159] This example compares the functions of each module of the improved residual network model of the present invention, and the results are shown in Table 3.

[0160] Table 3 Comparison results

[0161]

[0162] It can be seen that the adaptive activation function module and CA attention module added in the present invention significantly improve the accuracy of the overall model.

[0163] Comparison of the visualization results of container damage detection using the original residual network and the method of the present invention is shown in the figure. Figure 6 It can be clearly seen that the method of the present invention has a significant improvement in detection accuracy.

[0164] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only illustrative of the principles of the present invention. Without departing from the spirit and scope of the present invention, any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included in the scope of protection of the present invention.

Claims

1. A container damage detection method, characterized in that: The method comprises the following steps: Step 1: Collect container image or video data samples to build an initial data set; Step 2, performing data enhancement preprocessing on the container image or video data samples in the initial data set to obtain a final data set; Step 3, using the final data set to train the improved Res2Net model to generate a container damage detection model; the improved Res2Net model includes an improved residual network model, adding a path fusion enhancement module, an adaptive activation function and an attention mechanism; the improved residual network model includes introducing a jump connection to add the input to the output; Step 4: For the container to be inspected, collect its image or video and input it into the container damage detection model to obtain the damage detection result.

2. The container damage detection method according to claim 1, characterized in that: The data enhancement described in step 2 is specifically as follows: flipping 50% of the data samples in the initial data set in the horizontal and vertical directions, and introducing random Gaussian blur.

3. The container damage detection method according to claim 1, characterized in that: The data enhancement described in step 2 specifically includes: The following enhancement operations are set: (1) horizontal flip; (2) vertical flip; (3) random Gaussian blur introduction; (4) image scaling; (5) image contrast change; (6) image sharpening; (7) input pixel value inversion, that is, the original value is v and the change value is 255-v; (8) random deformation of the image; (9) image movement to the left, right, up or down; (10) image rotation clockwise or counterclockwise; Randomly select any of the above 10 enhancement operations to process image or video data samples.

4. The container damage detection method according to claim 1, characterized in that: In step 3, the improved residual network model is located in the convolution layer of the Res2Net model, which is used to extract features from image or video data samples, and the path fusion enhancement module, adaptive activation function and attention mechanism are located in the pooling layer of the Res2Net model.

5. The container damage detection method according to claim 1, characterized in that: The improved residual network model in step 3 includes five layers of residual modules, which represent features at multiple scales and increase the receptive field range of each network layer; The first-layer residual module uses three 3*3 convolution kernels, with 64 channels and a step size of 2. After standardization and pooling, it is input into the second-layer residual module; the second-layer residual module sets 3 Res2Net modules, increasing the number of channels from 64 to 256; the third-layer residual module sets 4 Res2Net modules, increasing the number of channels to 512; the fourth-layer residual module sets 23 Res2Net modules, increasing the number of channels to 1024; the last layer sets 3 Res2Net modules, increasing the number of channels to 2048.

6. The container damage detection method according to claim 5, characterized in that: The residual module includes multiple groups of convolution kernels, each group of convolution kernels includes a serialized 1*1 convolution kernel, a 3*3 convolution kernel and a 1*1 convolution kernel; the input of the residual module is: a plurality of groups of input feature maps, which are obtained by dividing the image or video data samples in the final data set into a plurality of groups; A set of convolution kernels first extracts features from a set of input feature maps, then combines the output feature maps with another set of input feature maps and extracts features through another set of convolution kernels; This process is repeated several times until all input feature maps are processed; finally, the output feature maps of all groups are concatenated together and feature fusion is performed through a 1×1 convolution kernel.

7. The container damage detection method according to claim 1, characterized in that: The path fusion enhancement module in step 3 enhances FPN2-FPN5 generated by the Mask-RCNN algorithm using FPN, specifically including: converting a feature map FPN2 output by FPN into FPN21 through maximum pooling, and convolving FPN21 with another feature map FPN3 to obtain FPN31, where the convolution is a 3×3 convolution kernel; similarly, convolving FPN31 with another feature map FPN4 to obtain FPN41, and convolving FPN41 with another feature map FPN5 to obtain FPN51; Except for the first layer, i.e., FPN21, which directly outputs as output 1, the following three layers fuse the inputs and outputs of the previous layers as output 2, output 3, and output 4, respectively; output 2 is obtained by convolution fusion of FPN3 and FPN31, output 3 is obtained by convolution fusion of FPN4 and FPN41, and output 4 is obtained by convolution fusion of FPN5 and FPN51.

8. The container damage detection method according to claim 1, characterized in that: The adaptive activation function in step 3 is: Among them, the adaptive activation function is an L-layer structure, Φ l is the function matrix of the lth layer, x is the given input vector, and KAN(x) is the output of the adaptive activation function corresponding to the given input vector x; The matrix form of the adaptive activation function is: In the formula, φ l,nl,n (·) is Φ l The nth l The sub-activation function of the nth row and column, nl represents the number of points to be activated, n represents the number of features, i.e. the tensor matrix, X l is the input of the lth layer, X l+1 is the input of the l+1th layer.

9. The container damage detection method according to claim 8, characterized in that: The learning of each sub-activation function, i.e., local B-spline basis function, includes: (1) Residual activation function The sub-activation function is expressed as: φ(x)=w(b(x)+spline(x)) Here, the sub-activation function is represented by φ(x), b(x) represents the basis function, and spline(x) represents the spline function; in, b(x)=x / (1+e -x ) spline(x) is parameterized as a linear combination of B-splines: In the formula, c i is a trainable matrix, w is a weight matrix, which is used to control the overall size of the sub-activation function; B i (x) represents the i-th B-spline function, i ranges from 1 to N, and N is the total number of B-spline functions; (2) Initialization Initialize each sub-activation function to the spline curve spline(x)≈0; w is initialized according to the Xavier initialization criterion, where the Xavier initialization criterion is as follows: In the formula, each element in the weight matrix w is initialized to obey the mean of 0 and the variance is The normal distribution of in is the number of input nodes, n out is the number of output nodes; (3) Update The update rule of the sub-activation function is as follows: Where φ'(x) is the updated sub-activation function, C represents the updated function m The order norm is used to measure the magnitude of the derivative of the function φ'(x) at each order. G represents the size of the spline grid, that is, the number of spline functions in each layer. m is the order of the derivative of the function. β is the maximum value less than or equal to m. sup represents the supremum of the set.

10. A container damage detection system based on the method according to any one of claims 1 to 9, characterized in that: The system comprises: The first module is used to collect container image or video data samples and build an initial data set; The second module is used to perform data enhancement preprocessing on the container image or video data samples in the initial data set to obtain a final data set; The third module is used to train the improved Res2Net model using the final data set to generate a container damage detection model; the improved Res2Net model includes an improved residual network model, adding a path fusion enhancement module, an adaptive activation function and an attention mechanism; the improved residual network model includes introducing a jump connection to add the input to the output; The fourth module is used to collect images or videos of the container to be inspected and input them into the container damage detection model to obtain damage detection results.

Images