A method for detecting the cleaning effect of concrete curing boxes based on classification weights

Abstract

The present application discloses a detection method for the cleaning effect of a concrete curing box based on classification weights, which comprises: obtaining an image of a concrete curing box to be detected; passing the image to be detected through a convolutional neural network to obtain a feature map; passing the feature map through first and second attention mechanism modules respectively to obtain first and second attention feature maps; passing the first and second attention feature maps through a Softmax function to obtain first and second classification weight score maps; performing positional point multiplication on the first and second classification weight score maps and the first and second attention feature maps respectively to obtain first and second classification weight feature maps; fusing the first and second classification weight feature maps to obtain a classification feature map; and passing the classification feature map through a classifier to obtain a classification result.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and more specifically, to a detection method for the cleaning effect of a concrete curing box based on classification weights, a detection system for the cleaning effect of a concrete curing box based on classification weights, and electronic equipment. Background Art

[0002] Concrete curing boxes are generally used to place concrete test blocks in the box for testing. After the test blocks are tested, the existing concrete curing boxes need to be cleaned, especially the surface of the support frame for placing the concrete test blocks in the inner cavity of the concrete curing box, otherwise the subsequent experimental results will be affected.

[0003] The cleaning effect of the current concrete curing box basically needs to rely on manual observation. However, due to the large size and depth of the concrete curing box and the lack of a good light source, the manual observation effect is not good.

[0004] Therefore, a technical solution for detecting the cleaning effect of a concrete curing box is desired.

[0005] At present, deep learning and neural networks have been widely used in computer vision, natural language processing, text signal processing and other fields. In addition, deep learning and neural networks have also shown a level close to or even beyond that of humans in image classification, object detection, semantic segmentation, text translation and other fields.

[0006] In recent years, the development of deep learning and neural networks has provided new solutions and solutions for the cleaning effect detection of concrete curing boxes. Summary of the invention

[0007] In order to solve the above technical problems, the present application is proposed. The embodiments of the present application provide a detection method of the cleaning effect of a concrete curing box based on classification weights, a detection system and an electronic device for the cleaning effect of a concrete curing box based on classification weights, which extract features and classify images of the inner wall of the concrete curing box and the surface of the support frame based on deep learning computer vision technology to detect the cleaning effect of the inner wall of the concrete curing box and the surface of the support frame.

[0008] According to one aspect of the present application, a method for detecting the cleaning effect of a concrete curing box based on classification weights is provided, which comprises:

[0009] Acquire an image of the concrete curing box to be detected, wherein the image of the concrete curing box to be detected includes an inner wall portion and a surface portion of a support frame of the concrete curing box to be detected;

[0010] Passing the image of the concrete curing box to be detected through a convolutional neural network to obtain a feature map;

[0011] Passing the feature map through a first attention mechanism module and a second attention mechanism module respectively to obtain a first attention feature map and a second attention feature map;

[0012] Pass the first attention feature map through a Softmax function to calculate the classification weight of each position in the first attention feature map to obtain a first classification weight score map;

[0013] Pass the second attention feature map through a Softmax function to calculate the classification weight of each position in the second attention feature map to obtain a second classification weight score map;

[0014] Multiply the first classification weight score map by the first attention feature map at pixel positions to obtain a first classification weight feature map;

[0015] Performing pixel-wise multiplication of the second classification weight score map and the second attention feature map to obtain a second classification weight feature map;

[0016] fusing the first classification weight feature map and the second classification weight feature map to obtain a classification feature map; and

[0017] The classification feature map is passed through a classifier to obtain a classification result, and the classification result is used to indicate whether the cleaning effect of the inner wall of the concrete curing box to be inspected and the surface of the support frame is qualified.

[0018] In the above-mentioned detection method of the cleaning effect of the concrete curing box based on classification weights, the feature map is passed through the first attention mechanism module and the second attention mechanism module respectively to obtain the first attention feature map and the second attention feature map, including: passing the first feature map through multiple first convolutional layers to obtain the first convolutional feature map; performing pixel-wise multiplication of the first convolutional feature map and the first feature map to obtain the first attention feature map; passing the second feature map through multiple second convolutional layers to obtain the second convolutional feature map; and performing pixel-wise multiplication of the second convolutional feature map and the second feature map to obtain the second attention feature map.

[0019] In the above-mentioned detection method of the cleaning effect of the concrete curing box based on classification weight, the first attention feature map is passed through the Softmax function to calculate the classification weight of each position in the first attention feature map to obtain a first classification weight score map, including: calculating the classification weight of each position in the first attention feature map by the following formula to obtain the first classification weight score map, the formula is: classification weight = exp(Ai) / ∑exp(Ai), Ai is the characteristic value of each pixel position in the first attention feature map; wherein, the second attention feature map is passed through the Softmax function to calculate the classification weight of each position in the second attention feature map to obtain a second classification weight score map, including: calculating the classification weight of each position in the second attention feature map by the following formula to obtain the second classification weight score map, the formula is: classification weight = exp(Bi) / ∑exp(Bi), Bi is the characteristic value of each pixel position in the second attention feature map.

[0020] In the above-mentioned detection method of the cleaning effect of the concrete curing box based on classification weights, the first classification weight feature map and the second classification weight feature map are fused to obtain a classification feature map, including: calculating the weighted sum of the first classification weight feature map and the second classification weight feature map by pixel position to obtain the classification feature map.

[0021] In the above-mentioned detection method of the cleaning effect of the concrete curing box based on classification weights, the classification feature map is passed through a classifier to obtain a classification result, including: passing the classification feature map through an encoder to obtain a classification feature vector, the encoder includes one or more fully connected layers; and, passing the classification feature vector through a Softmax classification function to obtain the classification result.

[0022] In the above-mentioned method for detecting the cleaning effect of a concrete curing box based on classification weights, the convolutional neural network is a deep residual network.

[0023] According to another aspect of the present application, a system for detecting the cleaning effect of a concrete curing box based on classification weights is provided, comprising:

[0024] An image acquisition unit to be detected is used to acquire an image of the concrete curing box to be detected, wherein the image of the concrete curing box to be detected includes an inner wall portion and a surface portion of a support frame of the concrete curing box to be detected;

[0025] A feature map generating unit, configured to obtain a feature map by passing the image of the concrete curing box to be detected obtained by the image acquisition unit through a convolutional neural network;

[0026] An attention feature map generating unit, configured to pass the first feature map obtained by the feature map generating unit through a first attention mechanism module and a second attention mechanism module respectively, so as to obtain a first attention feature map and a second attention feature map;

[0027] A first classification weight score map generating unit, configured to calculate the classification weight of each position in the first attention feature map through a Softmax function on the first attention feature map obtained by the attention feature map generating unit, so as to obtain a first classification weight score map;

[0028] A second classification weight score map generating unit, configured to pass the second attention feature map obtained by the attention feature map generating unit through a Softmax function to calculate the classification weight of each position in the second attention feature map, so as to obtain a second classification weight score map;

[0029] A first classification weight feature map generating unit, configured to perform a pixel-wise multiplication of the first classification weight score map obtained by the first classification weight score map generating unit and the first attention feature map obtained by the attention feature map generating unit to obtain a first classification weight feature map;

[0030] A second classification weight feature map generating unit, configured to perform a pixel-wise multiplication of the second classification weight score map obtained by the second classification weight score map generating unit and the second attention feature map obtained by the attention feature map generating unit to obtain a second classification weight feature map;

[0031] a feature map fusion unit, configured to fuse the first classification weight feature map obtained by the first classification weight feature map generation unit with the second classification weight feature map obtained by the second classification weight feature map generation unit to obtain a classification feature map; and

[0032] The classification result generating unit is used to pass the classification feature map obtained by the feature map fusion unit through a classifier to obtain a classification result, wherein the classification result is used to indicate whether the cleaning effect of the inner wall of the concrete curing box to be inspected and the surface of the support frame is qualified.

[0033] In the above-mentioned detection system of the cleaning effect of the concrete curing box based on classification weights, the attention feature map generation unit includes: a first convolution feature map feature map generation subunit, used to pass the first feature map through multiple first convolution layers to obtain a first convolution feature map; a first dot multiplication subunit, used to perform pixel-wise dot multiplication of the first convolution feature map and the first feature map to obtain the first attention feature map; a second convolution feature map feature map generation subunit, used to pass the second feature map through multiple second convolution layers to obtain a second convolution feature map; and, a second dot multiplication subunit, used to perform pixel-wise dot multiplication of the second convolution feature map and the second feature map to obtain the second attention feature map.

[0034] In the above-mentioned detection system of the cleaning effect of the concrete curing box based on classification weights, the first classification weight feature map generation unit is further used to: calculate the classification weight of each position in the first attention feature map by the following formula to obtain a first classification weight score map, the formula is: classification weight = exp(Ai) / ∑exp(Ai), Ai is the characteristic value of each pixel position in the first attention feature map; wherein, the second classification weight feature map generation unit is further used to: calculate the classification weight of each position in the second attention feature map by the following formula to obtain a second classification weight score map, the formula is: classification weight = exp(Bi) / ∑exp(Bi), Bi is the characteristic value of each pixel position in the second attention feature map.

[0035] In the above-mentioned detection system of the cleaning effect of the concrete curing box based on classification weights, the feature map fusion unit is further used to: calculate the weighted sum by pixel position between the first classification weight feature map and the second classification weight feature map to obtain the classification feature map.

[0036] In the above-mentioned detection system of the cleaning effect of the concrete curing box based on classification weights, the classification result generation unit includes: an encoding subunit, used to pass the classification feature map through an encoder to obtain a classification feature vector, the encoder includes one or more fully connected layers; and a classification subunit, used to pass the classification feature vector through a Softmax classification function to obtain the classification result.

[0037] In the above-mentioned cleaning effect detection system of the concrete curing box based on classification weights, the convolutional neural network is a deep residual network.

[0038] According to another aspect of the present application, an electronic device is provided, comprising: a processor; and a memory, wherein computer program instructions are stored in the memory, and when the computer program instructions are executed by the processor, the processor executes the above-mentioned method for detecting the cleaning effect of a concrete curing box based on classification weights.

[0039] According to another aspect of the present application, a computer-readable medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the above-mentioned method for detecting the cleaning effect of a concrete curing box based on classification weights.

[0040] According to the method for detecting the cleaning effect of a concrete curing box based on classification weights, the system and electronic device for detecting the cleaning effect of a concrete curing box based on classification weights provided in the present application, computer vision technology based on deep learning is used to extract features and classify images of the inner wall of the concrete curing box and the surface of the support frame to detect the cleaning effect of the inner wall of the concrete curing box and the surface of the support frame. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0042] Figure 1 A schematic diagram of a scenario of a method for detecting a cleaning effect of a concrete curing box based on classification weights according to an embodiment of the present application is illustrated.

[0043] Figure 2 The flowchart of the method for detecting the cleaning effect of a concrete curing box based on classification weights according to an embodiment of the present application is illustrated.

[0044] Figure 3 The figure shows a schematic diagram of the architecture of a method for detecting the cleaning effect of a concrete curing box based on classification weights according to an embodiment of the present application.

[0045] Figure 4 The figure illustrates a flow chart of a method for detecting the cleaning effect of a concrete curing box based on classification weights according to an embodiment of the present application, in which the feature map is passed through a first attention mechanism module and a second attention mechanism module respectively to obtain a first attention feature map and a second attention feature map.

[0046] Figure 5The figure illustrates a flow chart of obtaining a classification result by passing the classification feature map through a classifier in a method for detecting the cleaning effect of a concrete curing box based on classification weights according to an embodiment of the present application.

[0047] Figure 6 The block diagram of a system for detecting the cleaning effect of a concrete curing box based on classification weights according to an embodiment of the present application is illustrated.

[0048] Figure 7 The diagram illustrates a block diagram of an attention feature map generating unit in a system for detecting a cleaning effect of a concrete curing box based on classification weights according to an embodiment of the present application.

[0049] Figure 8 The diagram illustrates a block diagram of a classification result generating unit in a system for detecting a cleaning effect of a concrete curing box based on classification weights according to an embodiment of the present application.

[0050] Fig. 9 A block diagram of an electronic device according to an embodiment of the present application is illustrated. DETAILED DESCRIPTION

[0051] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0052] Scenario Overview

[0053] As mentioned above, a concrete curing box generally places a concrete test block in the box for testing the test block. After the test block is tested, the existing concrete curing box needs to be cleaned, especially the surface of the support frame for placing the concrete test block in the inner cavity of the concrete curing box, otherwise it will affect the subsequent experimental results.

[0054] The cleaning effect of the current concrete curing box basically needs to rely on manual observation. However, due to the large size and depth of the concrete curing box and the lack of a good light source, the manual observation effect is not good.

[0055] Therefore, a technical solution for detecting the cleaning effect of a concrete curing box is desired.

[0056] The applicant of the present application considers using computer vision technology based on deep learning to extract features and classify images of the inner wall of the concrete curing box and the surface of the support frame to detect the cleaning effect of the inner wall of the concrete curing box and the surface of the support frame.

[0057] In practice, the inner wall of the concrete curing box and the surface of the support frame are mainly composed of solidified concrete blocks, which usually do not exist in a single form. Therefore, if these objects of concern can be identified during the image feature extraction process and these objects can be properly classified, it will be very helpful to detect the cleaning effect of the inner wall of the concrete curing box and the surface of the support frame.

[0058] Specifically, in the technical solution of the present application, an image of a concrete curing box is first obtained, the image including the inner wall portion and the surface portion of the support frame of the concrete curing box, and then the image is input into a convolutional neural network to obtain a feature map. Next, the feature map is respectively passed through a first attention mechanism and a second attention mechanism to extract local features that need attention, specifically, local features that may be related to the residual solidified concrete block objects in the concrete curing box, and the first attention map and the second attention map are respectively passed through a softmax function to calculate the classification weight of each position of the attention map, and then multiplied by the first attention map and the second attention map to obtain a first classification weight map and a second classification weight map, and then the first classification weight map and the second classification weight map are fused to obtain a classification feature map.

[0059] In this way, a classification result can be obtained after the obtained classification feature map passes through the classification function, and the classification result is used to represent the cleaning effect of the inner wall of the concrete curing box and the surface of the support frame.

[0060] Based on this, the present application proposes a detection method for the cleaning effect of a concrete curing box based on classification weights, which includes: obtaining an image of a concrete curing box to be detected, wherein the image of the concrete curing box to be detected includes an inner wall portion and a surface portion of a support frame of the concrete curing box to be detected; passing the image of the concrete curing box to be detected through a convolutional neural network to obtain a feature map; passing the feature map through a first attention mechanism module and a second attention mechanism module respectively to obtain a first attention feature map and a second attention feature map; passing the first attention feature map through a Softmax function to calculate the classification weight of each position in the first attention feature map to obtain a first classification weight score map; passing the second attention feature map through a Softmax function to obtain a first classification weight score map. The feature map is passed through a Softmax function to calculate the classification weight of each position in the second attention feature map to obtain a second classification weight score map; the first classification weight score map is dot-multiplied by the first attention feature map at pixel position to obtain a first classification weight feature map; the second classification weight score map is dot-multiplied by the second attention feature map at pixel position to obtain a second classification weight feature map; the first classification weight feature map and the second classification weight feature map are fused to obtain a classification feature map; and the classification feature map is passed through a classifier to obtain a classification result, which is used to indicate whether the cleaning effect of the inner wall of the concrete curing box to be inspected and the surface of the support frame is qualified.

[0061] Figure 1 A schematic diagram of a scenario of a method for detecting a cleaning effect of a concrete curing box based on classification weights according to an embodiment of the present application is illustrated.

[0062] like Figure 1 As shown, in this application scenario, firstly, a camera (for example, Figure 1 C) as shown in the figure, obtaining an image of the concrete curing box to be detected, wherein the image of the concrete curing box to be detected includes the inner wall portion and the surface portion of the support frame of the concrete curing box to be detected; then, the image of the concrete curing box to be detected is input to a server (for example, as shown in the figure) in which a detection algorithm for the cleaning effect of the concrete curing box based on the classification weight is deployed. Figure 1 In S) as shown in , the server can process the image of the concrete curing box to be detected using a detection algorithm for the cleaning effect of the concrete curing box based on the classification weight to generate a detection result indicating whether the cleaning effect of the inner wall and the surface of the support frame of the concrete curing box to be detected is qualified.

[0063] After introducing the basic principles of the present application, various non-limiting embodiments of the present application will be described in detail with reference to the accompanying drawings.

[0064] Exemplary Methods

[0065] Figure 2 FIG. 1 is a flow chart of a method for detecting the cleaning effect of a concrete curing box based on classification weights according to an embodiment of the present application. Figure 2 As shown, according to an embodiment of the present application, a detection method for the cleaning effect of a concrete curing box based on classification weights includes: S110, acquiring an image of a concrete curing box to be detected, wherein the image of the concrete curing box to be detected includes an inner wall portion and a surface portion of a support frame of the concrete curing box to be detected; S120, passing the image of the concrete curing box to be detected through a convolutional neural network to obtain a feature map; S130, passing the feature map through a first attention mechanism module and a second attention mechanism module respectively to obtain a first attention feature map and a second attention feature map; S140, passing the first attention feature map through a Softmax function to calculate the classification weight of each position in the first attention feature map to obtain a first classification weight score map; S150, passing the second attention feature map through a Softmax function to obtain a first classification weight score map; The attention feature map is passed through a Softmax function to calculate the classification weight of each position in the second attention feature map to obtain a second classification weight score map; S160, the first classification weight score map is multiplied by the first attention feature map at pixel position to obtain a first classification weight feature map; S170, the second classification weight score map is multiplied by the second attention feature map at pixel position to obtain a second classification weight feature map; S180, the first classification weight feature map and the second classification weight feature map are fused to obtain a classification feature map; and, S190, the classification feature map is passed through a classifier to obtain a classification result, which is used to indicate whether the cleaning effect of the inner wall of the concrete curing box to be inspected and the surface of the support frame is qualified.

[0066] Figure 3 The figure shows a schematic diagram of the structure of the method for detecting the cleaning effect of the concrete curing box based on the classification weight according to an embodiment of the present application. Figure 3 As shown, in the network architecture, firstly, the acquired concrete curing box image to be detected (for example, Figure 3 The I N0 shown in FIG. 1 is input into a convolutional neural network (e.g., Figure 3 ) to obtain feature maps (e.g., Figure 3 Then, the feature maps are respectively passed through the first attention mechanism module (for example, Figure 3 ATN1 as shown in Figure 1) and a second attention mechanism module (e.g., Figure 3 ) to obtain the first attention feature map (e.g., Figure 3 F1 as shown in the figure) and the second attention feature map (e.g., Figure 3Then, the first attention feature map is passed through a Softmax function to calculate the classification weight of each position in the first attention feature map to obtain a first classification weight score map (for example, Figure 3 Fs1 as shown in FIG, and the second attention feature map is passed through a Softmax function to calculate the classification weight of each position in the second attention feature map to obtain a second classification weight score map (for example, Figure 3 Next, the first classification weight score map is multiplied by the first attention feature map at pixel positions to obtain a first classification weight feature map (e.g., Figure 3 Fw1 as shown in FIG, and performing pixel-wise multiplication of the second classification weight score map and the second attention feature map to obtain a second classification weight feature map (e.g., Figure 3 Then, the first classification weight feature map and the second classification weight feature map are fused to obtain a classification feature map (for example, as Figure 3 Finally, the classification feature map is passed through a classifier to obtain a classification result, and the classification result is used to indicate whether the cleaning effect of the inner wall of the concrete curing box to be inspected and the surface of the support frame is qualified.

[0067] In step S110, an image of the concrete curing box to be inspected is obtained, and the image of the concrete curing box to be inspected includes the inner wall portion and the surface portion of the support frame of the concrete curing box to be inspected. As mentioned above, in practice, the inner wall of the concrete curing box and the surface of the support frame mainly have solidified concrete blocks remaining, and such solidified concrete blocks usually do not exist in a single form. Therefore, if these objects of concern can be identified in the process of extracting image features and these objects are properly classified, it is very helpful to detect the cleaning effect of the inner wall of the concrete curing box and the surface of the support frame.

[0068] Therefore, in the technical solution of the present application, when collecting the image of the concrete curing box to be inspected, the shooting angle of the camera is adjusted so that the image of the concrete curing box to be inspected includes the inner wall portion and the surface portion of the support frame of the concrete curing box to be inspected.

[0069] In step S120, the image of the concrete curing box to be detected is passed through a convolutional neural network to obtain a feature map. That is, the image of the concrete curing box to be detected is processed by a deep convolutional neural network to extract high-dimensional features in the steel bar image to be detected. Those skilled in the art should know that convolutional neural networks have excellent performance in extracting local spatial features of images.

[0070] Preferably, in an embodiment of the present application, the convolutional neural network is implemented as a deep residual network, for example, ResNet 50. Compared with the traditional convolutional neural network, the deep residual network is an optimized network structure proposed on the basis of the traditional convolutional neural network, which mainly solves the problem of gradient vanishing during training. The deep residual network introduces a residual network structure, through which the network layer can be made deeper, and the problem of gradient vanishing will not occur. The residual network draws on the cross-layer connection idea of ​​the high-speed network, which breaks the traditional neural network convention that the input layer of the N-1 layer can only give the N layer as input, so that the output of a certain layer can directly cross several layers as the input of a certain layer behind. Its significance lies in that it provides a new direction for the problem that the error rate of the entire learning model increases instead of decreases due to the superposition of multiple layers of networks.

[0071] In step S130, the feature map is passed through the first attention mechanism module and the second attention mechanism module respectively to obtain a first attention feature map and a second attention feature map. That is, the feature map is passed through the first attention mechanism and the second attention mechanism respectively to extract the local features that need to be paid attention to, that is, the inner wall part and the support frame surface part of the concrete curing box to be detected in the concrete curing box image to be detected.

[0072] Specifically, in a specific example of the present application, the process of passing the feature map through the first attention mechanism module to obtain the first attention feature map includes: first, passing the first feature map through multiple first convolutional layers to obtain the first convolutional feature map, that is, in the technical solution of the present application, the first attention mechanism module includes multiple convolutional layers to extract the area to be paid attention to in the first feature map through multiple convolutional layers. Then, the first convolutional feature map is multiplied by the first feature map at the pixel position to obtain the first attention feature map. That is, the information in the first convolutional feature map is applied to the first feature map so that the area to be paid attention to in the first feature map is strengthened to obtain the first attention feature map.

[0073] Specifically, in this example, the process of passing the feature map through the second attention mechanism module to obtain the second attention feature map includes: first, passing the second feature map through multiple second convolutional layers to obtain a second convolutional feature map, that is, in the technical solution of the present application, the second attention mechanism module includes multiple convolutional layers to extract the area to be paid attention to in the second feature map through multiple convolutional layers. Here, in particular, the multiple second convolutional layers and the multiple second convolutional layers have different network structures. Then, the second convolutional feature map is dot-multiplied with the second feature map by pixel position to obtain the second attention feature map. That is, the information in the second convolutional feature map is applied to the second feature map so that the area to be paid attention to in the first feature map is strengthened to obtain the second attention feature map.

[0074] Figure 4 The figure shows a flow chart of a method for detecting the cleaning effect of a concrete curing box based on classification weights according to an embodiment of the present application, in which the feature graph is passed through a first attention mechanism module and a second attention mechanism module respectively to obtain a first attention feature graph and a second attention feature graph. Figure 4 As shown, the feature map is passed through the first attention mechanism module and the second attention mechanism module respectively to obtain a first attention feature map and a second attention feature map, including: S210, passing the first feature map through multiple first convolutional layers to obtain a first convolutional feature map; S220, performing pixel-wise multiplication of the first convolutional feature map and the first feature map to obtain the first attention feature map; S230, passing the second feature map through multiple second convolutional layers to obtain a second convolutional feature map; and, S240, performing pixel-wise multiplication of the second convolutional feature map and the second feature map to obtain the second attention feature map.

[0075] In step S140, the first attention feature map is passed through the Softmax function to calculate the classification weight of each position in the first attention feature map to obtain a first classification weight score map. It should be understood that the first attention feature map focuses on the local features that need to be paid attention to in the feature map, that is, the features of the inner wall portion of the concrete curing box to be detected or the surface portion of the support frame in the concrete curing box image to be detected. And, as mentioned above, the inner wall of the concrete curing box and the surface of the support frame are mainly solidified concrete blocks, and such solidified concrete blocks usually do not exist in a single form. Therefore, if these objects that need to be paid attention to can be identified in the process of extracting image features and these objects are properly classified, it will be very helpful to detect the cleaning effect of the inner wall of the concrete curing box and the surface of the support frame.

[0076] Accordingly, in the embodiment of the present application, the first attention feature map is passed through the Softmax function to calculate the classification weight of each position in the first attention feature map to obtain a first classification weight score map. That is, the first attention feature map is passed through the Softmax function to calculate the probability value of each pixel position in the first attention feature map belonging to the concrete label.

[0077] More specifically, in an embodiment of the present application, the process of passing the first attention feature map through a Softmax function to calculate the classification weight of each position in the first attention feature map to obtain a first classification weight score map includes: calculating the classification weight of each position in the first attention feature map using the following formula to obtain a first classification weight score map, wherein the formula is: classification weight = exp(Ai) / ∑exp(Ai), where Ai is the characteristic value of each pixel position in the first attention feature map.

[0078] In step S150, the second attention feature map is passed through a Softmax function to calculate the classification weight of each position in the second attention feature map to obtain a second classification weight score map. That is, the second attention feature map is passed through a Softmax function to calculate the probability value of each pixel position in the second attention feature map belonging to the concrete label to obtain the second classification weight score map.

[0079] More specifically, in an embodiment of the present application, the process of passing the second attention feature map through a Softmax function to calculate the classification weight of each position in the second attention feature map to obtain a second classification weight score map includes: calculating the classification weight of each position in the second attention feature map using the following formula to obtain a second classification weight score map, wherein the formula is: classification weight = exp(Bi) / ∑exp(Bi), where Bi is the eigenvalue of each pixel position in the second attention feature map.

[0080] In step S160, the first classification weight score map is multiplied by the first attention feature map at pixel positions to obtain the first classification weight feature map. That is, the information in the first classification weight score map is applied to the first attention feature map to obtain the first classification weight feature map. It should be understood that in the first classification weight feature map, the area belonging to the concrete label is further enhanced.

[0081] In step S170, the second classification weight score map is multiplied by the second attention feature map at pixel positions to obtain a second classification weight feature map. That is, the information in the second classification weight score map is applied to the second attention feature map to obtain the second classification weight feature map. It should be understood that in the second classification weight feature map, the area belonging to the concrete label is further enhanced.

[0082] In step S180, the first classification weight feature map and the second classification weight feature map are fused to obtain a classification feature map. Specifically, in an embodiment of the present application, the first classification weight feature map and the second classification weight feature map are fused by calculating a weighted sum of the first classification weight feature map and the second classification weight feature map according to pixel position to obtain the classification feature map.

[0083] In step S180, the classification feature map is passed through a classifier to obtain a classification result, and the classification result is used to indicate whether the cleaning effect of the inner wall of the concrete curing box and the surface of the support frame to be inspected is qualified. That is, in the technical solution of the present application, the feature extraction stage and the classification stage are decoupled.

[0084] More specifically, in an embodiment of the present application, the classifier includes an encoder, which is used to encode the classification feature map to map the features in the classification feature map to the label space to obtain a classification feature vector. In a specific example, the encoder includes one or more fully connected layers to fully utilize the information at each position in the classification feature map through the one or more fully connected layers. Next, the classification feature vector is input into the classification function to obtain a first probability that the cleaning effect of the inner wall and the support frame surface of the concrete curing box to be detected is qualified and a second probability that the cleaning effect of the inner wall and the support frame surface of the concrete curing box to be detected is unqualified. Then, based on the first probability and the second probability, the classification result is generated, and the classification result is used to indicate whether the cleaning effect of the inner wall and the support frame surface of the concrete curing box to be detected is qualified.

[0085] Figure 5 The figure shows a flow chart of a method for detecting the cleaning effect of a concrete curing box based on classification weights according to an embodiment of the present application, in which the classification feature graph is passed through a classifier to obtain a classification result. Figure 5As shown, in an embodiment of the present application, the classification feature map is passed through a classifier to obtain a classification result, including: S310, passing the classification feature map through an encoder to obtain a classification feature vector, the encoder including one or more fully connected layers; and, S320, passing the classification feature vector through a Softmax classification function to obtain the classification result.

[0086] In summary, a method for detecting the cleaning effect of a concrete curing box based on classification weights according to an embodiment of the present application is explained, which uses deep learning-based computer vision technology to extract features and classify images of the inner wall of the concrete curing box and the surface of the support frame to detect the cleaning effect of the inner wall of the concrete curing box and the surface of the support frame.

[0087] Exemplary Systems

[0088] Figure 6 The block diagram of a system for detecting the cleaning effect of a concrete curing box based on classification weights according to an embodiment of the present application is illustrated.

[0089] like Figure 6As shown, according to an embodiment of the present application, a detection system 600 for the cleaning effect of a concrete curing box based on classification weights includes: a detection image acquisition unit 610, used to acquire an image of a concrete curing box to be detected, wherein the image of the concrete curing box to be detected includes an inner wall portion and a surface portion of a support frame of the concrete curing box to be detected; a feature map generation unit 620, used to pass the image of the concrete curing box to be detected obtained by the detection image acquisition unit 610 through a convolutional neural network to obtain a feature map; an attention feature map generation unit 630, used to pass the first feature map obtained by the feature map generation unit 620 through a first attention mechanism module and a second attention mechanism module respectively to obtain a first attention feature map and a second attention feature map; a first classification weight score map generation unit 640, used to pass the first attention feature map obtained by the attention feature map generation unit 630 through a Softmax function to calculate the classification weight of each position in the first attention feature map to obtain a first classification weight score map; a second classification weight score map generation unit 650, used to pass the second attention feature map obtained by the attention feature map generation unit 630 through a Softmax function to calculate the second attention feature map The first classification weight feature map generating unit 660 is used to perform pixel-wise multiplication of the first classification weight score map obtained by the first classification weight score map generating unit 640 and the first attention feature map obtained by the attention feature map generating unit 630 to obtain the first classification weight feature map; the second classification weight feature map generating unit 670 is used to perform pixel-wise multiplication of the second classification weight score map obtained by the second classification weight score map generating unit 650 and the second attention feature map obtained by the attention feature map generating unit 630 to obtain the second classification weight feature map; the feature map fusion unit 680 is used to fuse the first classification weight feature map obtained by the first classification weight feature map generating unit 660 and the second classification weight feature map obtained by the second classification weight feature map generating unit 670 to obtain a classification feature map; and the classification result generating unit 690 is used to pass the classification feature map obtained by the feature map fusion unit 680 through a classifier to obtain a classification result, and the classification result is used to indicate whether the cleaning effect of the inner wall and the support frame surface of the concrete curing box to be inspected is qualified.

[0090] In one example, in the above detection system 600, as Figure 7As shown, the attention feature map generating unit 630 includes: a first convolution feature map generating subunit 631, used to pass the first feature map through multiple first convolution layers to obtain a first convolution feature map; a first dot multiplication subunit 632, used to perform pixel-wise dot multiplication of the first convolution feature map with the first feature map to obtain the first attention feature map; a second convolution feature map generating subunit 633, used to pass the second feature map through multiple second convolution layers to obtain a second convolution feature map; and, a second dot multiplication subunit 634, used to perform pixel-wise dot multiplication of the second convolution feature map with the second feature map to obtain the second attention feature map.

[0091] In one example, in the above-mentioned detection system 600, the first classification weight feature map generation unit 660 is further used to: calculate the classification weight of each position in the first attention feature map by the following formula to obtain a first classification weight score map, the formula is: classification weight = exp(Ai) / ∑exp(Ai), Ai is the characteristic value of each pixel position in the first attention feature map; wherein, the second classification weight feature map generation unit 670 is further used to: calculate the classification weight of each position in the second attention feature map by the following formula to obtain a second classification weight score map, the formula is: classification weight = exp(Bi) / ∑exp(Bi), Bi is the characteristic value of each pixel position in the second attention feature map.

[0092] In one example, in the above-mentioned detection system 600, the feature map fusion unit 680 is further used to: calculate the weighted sum by pixel position between the first classification weight feature map and the second classification weight feature map to obtain the classification feature map.

[0093] In one example, in the above detection system 600, as Figure 8 As shown, the classification result generating unit 690 includes: an encoding subunit 691, used to pass the classification feature map through an encoder to obtain a classification feature vector, the encoder includes one or more fully connected layers; and a classification subunit 692, used to pass the classification feature vector through a Softmax classification function to obtain the classification result.

[0094] In the above-mentioned cleaning effect detection system of the concrete curing box based on classification weights, the convolutional neural network is a deep residual network.

[0095] Here, those skilled in the art will appreciate that the specific functions and operations of the various units and modules in the above detection system 600 have been described in detail above. Figures 1 to 5The description of the detection method of the cleaning effect of the concrete curing box based on the classification weight has been described in detail, and therefore, its repeated description will be omitted.

[0096] As described above, the detection system 600 according to the embodiment of the present application can be implemented in various terminal devices, such as a server for cleaning effect detection of a concrete curing box. In one example, the detection system 600 according to the embodiment of the present application can be integrated into the terminal device as a software module and / or a hardware module. For example, the detection system 600 can be a software module in the operating system of the terminal device, or can be an application developed for the terminal device; of course, the detection system 600 can also be one of the many hardware modules of the terminal device.

[0097] Alternatively, in another example, the detection system 600 and the terminal device may also be separate devices, and the detection system 600 may be connected to the terminal device via a wired and / or wireless network, and transmit the interactive information in accordance with an agreed data format.

[0098] Exemplary Electronic Devices

[0099] Below, reference Fig. 9 To describe an electronic device according to an embodiment of the present application.

[0100] Fig. 9 A block diagram of an electronic device according to an embodiment of the present application is illustrated.

[0101] like Fig. 9 As shown, the electronic device 10 includes one or more processors 11 and a memory 12 .

[0102] The processor 11 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

[0103] The memory 12 may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may run the program instructions to implement the above-described detection method of the cleaning effect of the concrete curing box based on the classification weight of each embodiment of the present application and / or other desired functions. Various contents such as images of concrete curing boxes to be detected and detection results may also be stored in the computer-readable storage medium.

[0104] In one example, the electronic device 10 may further include: an input device 13 and an output device 14, and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0105] The input device 13 may include, for example, a keyboard, a mouse, etc.

[0106] The output device 14 can output various information to the outside, including detection results, etc. The output device 14 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0107] Of course, to simplify, Fig. 9 Only some of the components related to the present application in the electronic device 10 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device 10 may also include any other appropriate components.

[0108] Exemplary computer program products and computer-readable storage media

[0109] In addition to the above-mentioned method and device, an embodiment of the present application may also be a computer program product, which includes computer program instructions, and when the computer program instructions are executed by a processor, the processor executes the steps of the method for detecting the cleaning effect of a concrete curing box based on classification weights according to various embodiments of the present application described in the above “Exemplary Method” section of this specification.

[0110] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0111] In addition, an embodiment of the present application may also be a computer-readable storage medium having computer program instructions stored thereon, and when the computer program instructions are executed by a processor, the processor executes the steps of the method for detecting the cleaning effect of a concrete curing box based on classification weights according to various embodiments of the present application described in the above “Exemplary Method” section of this specification.

[0112] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0113] The basic principles of the present application are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present application are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present application. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, not for limitation, and the above details do not limit the present application to being implemented by adopting the above specific details.

[0114] The block diagrams of the devices, apparatuses, equipment, and systems involved in this application are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The words "such as" used here refer to the phrase "such as but not limited to", and can be used interchangeably with them.

[0115] It should also be noted that in the apparatus, device and method of the present application, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present application.

[0116] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0117] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A method for detecting the cleaning effect of a concrete curing box based on classification weights, characterized in that: include: Acquire an image of the concrete curing box to be detected, wherein the image of the concrete curing box to be detected includes an inner wall portion and a surface portion of a support frame of the concrete curing box to be detected; Passing the image of the concrete curing box to be detected through a convolutional neural network to obtain a feature map; Passing the feature map through a first attention mechanism module and a second attention mechanism module respectively to obtain a first attention feature map and a second attention feature map; Pass the first attention feature map through a Softmax function to calculate the classification weight of each position in the first attention feature map to obtain a first classification weight score map; Pass the second attention feature map through a Softmax function to calculate the classification weight of each position in the second attention feature map to obtain a second classification weight score map; Performing pixel-wise multiplication of the first classification weight score map and the first attention feature map to obtain a first classification weight feature map; Performing pixel-wise multiplication of the second classification weight score map and the second attention feature map to obtain a second classification weight feature map; fusing the first classification weight feature map and the second classification weight feature map to obtain a classification feature map; as well as The classification feature map is passed through a classifier to obtain a classification result, and the classification result is used to indicate whether the cleaning effect of the inner wall of the concrete curing box to be inspected and the surface of the support frame is qualified.

2. The method for detecting the cleaning effect of a concrete curing box based on classification weight according to claim 1, wherein: Passing the feature map through the first attention mechanism module and the second attention mechanism module respectively to obtain a first attention feature map and a second attention feature map, including: Passing the first feature map through a plurality of first convolutional layers to obtain a first convolutional feature map; Multiply the first convolution feature map by the first feature map at pixel positions to obtain the first attention feature map; Passing the second feature map through a plurality of second convolutional layers to obtain a second convolutional feature map; and Multiply the second convolution feature map by the second feature map at pixel positions to obtain the second attention feature map.

3. The method for detecting the cleaning effect of a concrete curing box based on classification weight according to claim 1, in, The first attention feature map is passed through a Softmax function to calculate the classification weight of each position in the first attention feature map to obtain a first classification weight score map, including: The classification weight of each position in the first attention feature map is calculated by the following formula to obtain a first classification weight score map, wherein the formula is: classification weight = exp(Ai) / ∑exp(Ai), where Ai is the characteristic value of each pixel position in the first attention feature map; The second attention feature map is passed through a Softmax function to calculate the classification weight of each position in the second attention feature map to obtain a second classification weight score map, including: The classification weight of each position in the second attention feature map is calculated by the following formula to obtain a second classification weight score map. The formula is: classification weight = exp(Bi) / ∑exp(Bi), where Bi is the eigenvalue of each pixel position in the second attention feature map.

4. The method for detecting the cleaning effect of a concrete curing box based on classification weights according to claim 1, wherein: Fusion of the first classification weight feature map and the second classification weight feature map to obtain a classification feature map includes: A weighted sum by pixel position between the first classification weight feature map and the second classification weight feature map is calculated to obtain the classification feature map.

5. The method for detecting the cleaning effect of a concrete curing box based on classification weights according to claim 1, wherein: Passing the classification feature map through a classifier to obtain a classification result includes: Passing the classification feature map through an encoder to obtain a classification feature vector, the encoder comprising one or more fully connected layers; and The classification feature vector is passed through a Softmax classification function to obtain the classification result.

6. The method for detecting the cleaning effect of a concrete curing box based on classification weights according to claim 1, wherein: The convolutional neural network is a deep residual network.

7. A detection system for the cleaning effect of a concrete curing box based on classification weights, characterized in that: include: An image acquisition unit to be detected is used to acquire an image of the concrete curing box to be detected, wherein the image of the concrete curing box to be detected includes an inner wall portion and a surface portion of a support frame of the concrete curing box to be detected; A feature map generating unit, configured to obtain a feature map by passing the image of the concrete curing box to be detected obtained by the image acquisition unit through a convolutional neural network; An attention feature map generating unit, configured to pass the first feature map obtained by the feature map generating unit through a first attention mechanism module and a second attention mechanism module respectively, so as to obtain a first attention feature map and a second attention feature map; A first classification weight score map generating unit, configured to calculate the classification weight of each position in the first attention feature map through a Softmax function on the first attention feature map obtained by the attention feature map generating unit, so as to obtain a first classification weight score map; A second classification weight score map generating unit, configured to pass the second attention feature map obtained by the attention feature map generating unit through a Softmax function to calculate the classification weight of each position in the second attention feature map, so as to obtain a second classification weight score map; A first classification weight feature map generating unit, configured to perform a pixel-wise multiplication of the first classification weight score map obtained by the first classification weight score map generating unit and the first attention feature map obtained by the attention feature map generating unit to obtain a first classification weight feature map; A second classification weight feature map generating unit, configured to perform a pixel-wise multiplication of the second classification weight score map obtained by the second classification weight score map generating unit and the second attention feature map obtained by the attention feature map generating unit to obtain a second classification weight feature map; a feature map fusion unit, configured to fuse the first classification weight feature map obtained by the first classification weight feature map generation unit with the second classification weight feature map obtained by the second classification weight feature map generation unit to obtain a classification feature map; as well as The classification result generating unit is used to pass the classification feature map obtained by the feature map fusion unit through a classifier to obtain a classification result, wherein the classification result is used to indicate whether the cleaning effect of the inner wall of the concrete curing box to be inspected and the surface of the support frame is qualified.

8. The cleaning effect detection system of the concrete curing box based on classification weight according to claim 7, wherein: The attention feature map generating unit comprises: A first convolutional feature map feature map generating subunit, configured to pass the first feature map through a plurality of first convolutional layers to obtain a first convolutional feature map; A first dot multiplication subunit, configured to perform pixel-wise dot multiplication of the first convolution feature map and the first feature map to obtain the first attention feature map; A second convolutional feature map generating subunit, configured to pass the second feature map through a plurality of second convolutional layers to obtain a second convolutional feature map; and The second dot multiplication subunit is used to perform pixel-wise dot multiplication of the second convolution feature map with the second feature map to obtain the second attention feature map.

9. The cleaning effect detection system of the concrete curing box based on classification weight according to claim 7, wherein: The first classification weight feature map generating unit is further used to: calculate the classification weight of each position in the first attention feature map by the following formula to obtain a first classification weight score map, wherein the formula is: classification weight=exp(Ai) / ∑exp(Ai), where Ai is the characteristic value of each pixel position in the first attention feature map; Among them, the second classification weight feature map generating unit is further used to: calculate the classification weight of each position in the second attention feature map by the following formula to obtain a second classification weight score map, and the formula is: classification weight = exp(Bi) / ∑exp(Bi), Bi is the characteristic value of each pixel position in the second attention feature map.

10. An electronic device, comprising: processor; as well as A memory, wherein computer program instructions are stored in the memory, and when the computer program instructions are executed by the processor, the processor executes the method for detecting the cleaning effect of a concrete curing box based on classification weights according to any one of claims 1 to 6.

Images