Intelligent construction target counting method, electronic device, and storage medium
By using the lightweight ShuffleNet v2 network and attention mechanism in the deep learning model, the problems of numerous model parameters and large computational cost in rebar counting are solved, and efficient rebar counting on small devices is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-12-15
- Publication Date
- 2026-03-31
AI Technical Summary
Existing deep learning algorithms have many model parameters and large data computation volume in rebar counting, which puts high demands on hardware performance and results in low counting efficiency.
We employ the lightweight ShuffleNet v2 network as the backbone, combining channel attention and spatial attention submodules, fusing features through a path aggregation network, and using the YOLO Head detector for target prediction, thereby reducing model parameters and optimizing feature extraction.
It reduces the computational and storage requirements of the model while improving the accuracy of target detection. It is suitable for deployment on small devices and achieves efficient rebar counting.
Smart Images
Figure CN116152171B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing technology, and more specifically, to an intelligent construction target counting method, electronic device, and storage medium. Background Technology
[0002] Traditional counting methods, typically manual, suffer from low efficiency. With the development of deep learning technology, algorithms such as the ResNeXt101 network and Faster R-CNN can now be used to recognize and count images. For example, counting rebars can be achieved using these algorithms. However, existing models require a large number of parameters, involve significant data processing, and place high demands on hardware performance. Summary of the Invention
[0003] In view of this, the purpose of this application is to provide an intelligent construction target counting method, electronic device and storage medium, which can improve the problems of large data processing volume and high hardware performance requirements of counting.
[0004] To achieve the above technical objectives, the technical solution adopted in this application is as follows:
[0005] In a first aspect, embodiments of this application provide an intelligent construction target counting method, the method comprising:
[0006] Acquire an image to be identified, which is an image obtained by capturing the target to be detected;
[0007] The image to be identified is input into a trained deep learning model to obtain anchor boxes corresponding to each target in the image to be identified, output by the deep learning model. The deep learning model includes an input module, a backbone network, an attention module, an attention module, a feature fusion layer, and an output module. The attention module includes a channel attention submodule and a spatial attention submodule. The backbone is a ShuffleNet v2 network, and the head includes three YOLO head detectors.
[0008] The number of targets in the image to be identified is obtained by counting all anchor boxes in the image to be identified.
[0009] In conjunction with the first aspect, in some optional embodiments, the image to be identified is input into a trained deep learning model to obtain anchor boxes output by the deep learning model corresponding to each target in the image to be identified, including:
[0010] The input image to be recognized, which has three channels, is output to the ShuffleNet v2 network through the Input in the deep learning model.
[0011] The image to be identified is used to extract features through the ShuffleNet v2 network to obtain the first type of image features;
[0012] The first type of image features are convolved using the channel attention submodule and the spatial attention submodule in Att to obtain the second type of image features.
[0013] The first type of image features and the second type of image features are fused using the Neck to obtain the third type of image features;
[0014] The three YOLO Head detectors are used to predict targets in the third type of image features, and the predicted targets are marked in the image to be identified using anchor boxes.
[0015] In conjunction with the first aspect, in some optional implementations, the first type of image features are convolved using the channel attention submodule and the spatial attention submodule in Att to obtain the second type of image features, including:
[0016] The first type of image features are fused by multi-frequency feature fusion through the channel attention submodule to obtain the fused first type of image features.
[0017] The second type of image features are obtained by performing multi-scale deformable convolution on the fused first type of image features through the spatial attention submodule.
[0018] In conjunction with the first aspect, in some optional embodiments, the channel attention submodule performs multi-frequency feature fusion on the first type of image features to obtain fused first type of image features, including:
[0019] The first type of image features are divided into n parts according to channels, denoted as [X 0 ,X 1 ,…,X n-1 ], where n is a value that is divisible by the number of channels of the first type of image features;
[0020] For the partition X i Assign the corresponding two-dimensional DCT frequency components, as follows:
[0021]
[0022] Where i takes any integer value from 0 to n-1; Frei ∈R c For X i After processing, the multi-frequency vector has H as the height of input X and W as the width of input X. The basis functions of the two-dimensional discrete cosine transform; [u,v] refers to X. i The frequency component is a two-dimensional exponent; [h,w] refers to the parameters in the basis function;
[0023] Through compression function F sq (X), the frequency components of n parts X are merged to obtain the first type of image feature after fusion, wherein the compression function is expressed as: F sq (X) = cat([Fre 0 ,Fre 1 ,…,Fre n-1 ]).
[0024] In conjunction with the first aspect, in some optional embodiments, the second type of image features are obtained by performing multi-scale deformable convolution on the fused first type of image features through the spatial attention submodule, including:
[0025] The spatial attention submodule performs parallel convolution on the fused first-class image features using three types of convolution kernels to obtain intermediate image features. The sizes of the three convolution kernels are 1×1, 3×3, and 5×5, respectively.
[0026] The spatial attention submodule performs a 3×3 convolution on the intermediate image features, and then processes the convolved intermediate image features through the sigmoid activation function and outputs them to obtain the second type of image features.
[0027] In conjunction with the first aspect, in some optional embodiments, the Neck includes a path aggregation network PANet, through which the first type of image features and the second type of image features are fused to obtain a third type of image features, including:
[0028] The first type of image features and the second type of image features are fused using the path aggregation network PANet to obtain the third type of image features.
[0029] In conjunction with the first aspect, in some alternative embodiments, the method further includes, before acquiring the image to be identified:
[0030] Obtain a training dataset, which includes multiple images, each of which has a target to be detected and is labeled with a corresponding label.
[0031] The pre-built deep learning model is trained using the training dataset to obtain the trained deep learning model.
[0032] In conjunction with the first aspect, in some alternative implementations, the target includes any one of steel bars, wooden columns, and pipes.
[0033] Secondly, embodiments of this application also provide an electronic device, which includes a processor and a memory coupled to each other. The memory stores a computer program, and when the computer program is executed by the processor, the electronic device performs the above-described method.
[0034] Thirdly, embodiments of this application also provide a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to perform the above-described method.
[0035] The invention employing the above technical solution has the following advantages:
[0036] In the technical solution provided in this application, the image to be identified is input into a trained deep learning model to obtain anchor boxes corresponding to each target in the image. Then, all anchor boxes in the image are counted to obtain the number of targets in the image. The deep learning model includes an input module (Input), a backbone network (Backbone), an attention module (Att), a feature fusion layer (Neck), and an output module (Head). Att includes channel attention submodules and spatial attention submodules. The backbone is a ShuffleNet v2 network, and the Head includes three YOLO Head detectors. Since the backbone network is a lightweight ShuffleNet v2 network, compared to the traditional YOLOv5s backbone network, it helps reduce model parameters and computational load. Furthermore, the introduction of a fusion channel and spatial attention mechanism allows for more effective learning and extraction of image features, which improves the accuracy of model detection. Attached Figure Description
[0037] This application can be further illustrated by the non-limiting embodiments given in the accompanying drawings. It should be understood that the following drawings only illustrate some embodiments of this application and should not be considered as limiting the scope. For those skilled in the art, other related drawings can be obtained from these drawings without any inventive effort.
[0038] Figure 1 This is a flowchart illustrating the intelligent construction target counting method provided in an embodiment of this application.
[0039] Figure 2This is a schematic diagram of the network structure of the deep learning model provided in the embodiments of this application.
[0040] Figure 3 This is a schematic diagram of the network structure of the attention module provided in an embodiment of this application.
[0041] Figure 4 This is a schematic diagram of the network structure of the channel attention submodule provided in an embodiment of this application.
[0042] Figure 5 This is a schematic diagram of the network structure of the spatial attention submodule provided in an embodiment of this application.
[0043] Figure 6 This is a schematic diagram of a reinforcing bar image provided in an embodiment of this application. Detailed Implementation
[0044] The present application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that similar or identical parts are referred to by the same reference numerals in the drawings or description. Implementations not shown or described in the drawings are forms known to those skilled in the art. In the description of this application, terms such as "first" and "second" are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.
[0045] This application provides an electronic device that may include a processing module and a storage module. The storage module stores a computer program, which, when executed by the processing module, enables the electronic device to perform corresponding steps in the intelligent construction target counting method described below.
[0046] Among them, electronic devices can be, but are not limited to, personal computers, smartphones, servers, and other devices.
[0047] Understandably, electronic devices can automatically count the targets to be detected in the image to be recognized, which helps improve the efficiency of target counting. The targets to be detected can be flexibly set according to actual conditions, and are usually elongated structures. For example, the target can be, but is not limited to, steel bars, wooden pillars, pipes, etc. The pipes can be, but are not limited to, steel pipes, PVC pipes, etc., without specific limitations. In this embodiment, steel bars are used as an example for illustration.
[0048] Please refer to Figure 1 This application also provides an intelligent construction target counting method, which can be applied to the aforementioned electronic device. The electronic device executes or implements the steps of the method, enabling automatic counting of the number of reinforcing bars based on photographs of reinforcing bar cross-sections. The intelligent construction target counting method may include the following steps:
[0049] Step 110: Obtain the image to be identified, which is an image obtained by capturing the target to be detected;
[0050] Step 120: Input the image to be identified into the trained deep learning model to obtain the anchor boxes corresponding to each target in the image to be identified, output by the deep learning model. The deep learning model includes an input module, a backbone network, an attention module, an attention module, a feature fusion layer, and an output module. The attention module includes a channel attention submodule and a spatial attention submodule. The backbone is a ShuffleNet v2 network, and the head includes three YOLO head detectors.
[0051] Step 130: Count all anchor boxes in the image to be identified to obtain the number of targets in the image to be identified.
[0052] The steps of the intelligent construction target counting method will be explained in detail below:
[0053] In step 110, the electronic device can use its own camera to capture images of the target to be counted, thereby obtaining an image to be identified. Alternatively, the electronic device can receive images transmitted from other devices as images to be identified. Understandably, the method of acquiring the image to be identified can be flexibly determined according to the actual situation, and no specific limitation is made here.
[0054] In this embodiment, the image to be identified can be an RGB image with three channels.
[0055] In step 120, the network structure of the deep learning model can be referred to Figure 2 That is, a deep learning model consists of five parts: Input, Backbone, Att, Neck, and Head. The deep learning model in this application is an improvement on the YOLOv5 model, which can be called the ShuffleNet-Att-YOLOv5 model, or the improved YOLOv5 model.
[0056] In this embodiment, step 120 may include:
[0057] The input image to be recognized, which has three channels, is output to the ShuffleNet v2 network through the Input in the deep learning model.
[0058] The image to be identified is used to extract features through the ShuffleNet v2 network to obtain the first type of image features;
[0059] The first type of image features are convolved using the channel attention submodule and the spatial attention submodule in Att to obtain the second type of image features.
[0060] The first type of image features and the second type of image features are fused using the Neck to obtain the third type of image features;
[0061] The three YOLO Head detectors are used to predict targets in the third type of image features, and the predicted targets are marked in the image to be identified using anchor boxes.
[0062] Understandably, the image to be recognized input to the Input layer is a three-channel RGB image. The backbone network uses the more lightweight ShuffleNet v2 network to reduce model parameters and computational cost. The Attention module (Att) introduces a channel and spatial attention mechanism to learn the extracted features more deeply, thereby enhancing the model's detection capability. The feature fusion layer (Neck) uses a Path Aggregation Network (PANet) to fuse the effective feature maps output from the Backbone and Att layers, achieving information fusion across different feature layers. PANet better integrates shallow and deep feature information, allowing the network to fully extract features from each layer, thus strengthening feature extraction and obtaining richer feature information. The output module (Head) includes three YOLO Head detectors, which can output feature maps at different scales and perform target prediction.
[0063] In this embodiment, the backbone network uses ShuffleNet v2 network to replace the original YOLOv5 backbone, thereby simplifying and compressing the algorithm model to reduce the required storage space and computing resources.
[0064] ShuffleNet evolved from residual networks, and its core idea is to shuffle different channels. ShuffleNet can significantly reduce model complexity and computation while maintaining a certain level of accuracy. The ShuffleNetV2 convolutional neural network strikes a good balance between speed and accuracy. ShuffleNetV2 retains the channel shuffling, grouped convolution, and depthwise separable convolution (DSC) operations used in ShuffleNetV1, which helps reduce computation and improve accuracy.
[0065] Please refer to the reference. Figure 3 , Figure 4 and Figure 5In this embodiment, the first type of image features are convolved using the channel attention submodule and the spatial attention submodule in Att to obtain the second type of image features, which may include:
[0066] The first type of image features are fused by multi-frequency feature fusion through the channel attention submodule to obtain the fused first type of image features.
[0067] The second type of image features are obtained by performing multi-scale deformable convolution on the fused first type of image features through the spatial attention submodule.
[0068] If the image to be identified is an image obtained from a cross-section of a reinforcing bar, the background of the reinforcing bar image is complex, noisy, and has stains on the end face. The image includes the reinforcing bar to be identified and counted, the sides of the surrounding reinforcing bars, and the complex background. Brighter areas in the complex background and stains on the end face of the reinforcing bars are noise to the target, which are unavoidable interference factors. The YOLOv5 network also has problems such as incomplete attention to the target area and ineffective attention to irrelevant background areas to varying degrees, which increases the difficulty of reinforcing bar detection. At the same time, the reinforcing bars are densely packed, irregularly shaped, and have different edges, such as sharp edges or rounded edges, making existing detection difficult. The attention mechanism in this application embodiment selects partial feature maps by assigning weights to different parts of the feature map, suppressing useless information, so as to select better features.
[0069] In this embodiment, the attention module integrates channel and spatial attention to enhance the identification and utilization of important features and reduce the loss of important detailed features. Functionally, the channel attention submodule improves the preservation of rebar texture information and reduces information loss of important shallow features through multi-frequency feature fusion. Functionally, the spatial attention submodule, for rebars with irregular cross-sections and varied shapes, utilizes a novel parallel structure of multi-scale deformable convolution (DCN) to enable the network to pay more attention to the spatial features of the rebars, thereby improving the effective focus on the target rebar region.
[0070] Depend on Figure 3 As shown, it consists of input features, a channel attention submodule, a spatial attention submodule, and output features. The input feature is F. in This is the first type of image feature; the output feature is F. out This is the second type of image feature.
[0071]
[0072]
[0073] Input features F∈R C*H*W Then through the channel attention submodule Mc New feature F is obtained Mid As in equation (1), F Mid The output feature F is obtained after processing by the spatial attention submodule. out .
[0074] Please refer to Figure 4 The first type of image features are fused using the channel attention submodule to obtain the fused first type of image features, including:
[0075] The first type of image features are divided into n parts according to channels, denoted as [X 0 ,X 1 ,…,X n-1 ]; where n is a value that can be divided evenly by the number of channels of the first type of image features. n can be flexibly determined based on the number of channels of the first type of image features. For example, if the number of channels is 6, then n can be 2 or 3.
[0076] For the partition X i Assign the corresponding two-dimensional (2D) DCT frequency components, as follows:
[0077]
[0078] Where i takes any integer value from 0 to n-1; Fre i ∈R c For X i After processing, the multi-frequency vector has H as the height of input X and W as the width of input X. The basis functions are the two-dimensional (2D) discrete cosine transform; [u,v] refers to X. i The frequency components are two-dimensional exponents; [h,w] are conventional parameters in the basis functions well known to those skilled in the art;
[0079] Through compression function F sq (X), the frequency components of n parts X are merged to obtain the first type of image features after fusion, wherein the compression function is expressed as:
[0080] F sq (X) = cat([Fre 0 ,Fre 1 ,…,Fre n-1 (4)
[0081] In this embodiment, the compression function is consistent with the widely used GAP (Global Average Pooling) operation in terms of output scale, and therefore can be seamlessly replaced by the GAP operation in the SENet (Squeeze and Excitation Network) attention mechanism.
[0082] Existing technologies show that GAP is the lowest frequency of the Discrete Cosine Transform (DCT). Using only GAP is equivalent to discarding other frequency components containing a large amount of information in the feature channels. In this embodiment, multi-frequency attention not only preserves GAP but also uses frequency components other than GAP, which can solve the problem of information loss caused by using a single frequency, allowing the algorithm to focus more on important features and remove redundant features.
[0083] Please refer to Figure 5 The second type of image features are obtained by performing multi-scale deformable convolution on the fused first type of image features through the spatial attention submodule, which may include:
[0084] The spatial attention submodule performs parallel convolution on the fused first-class image features using three types of convolution kernels to obtain intermediate image features. The sizes of the three convolution kernels are 1×1, 3×3, and 5×5, respectively.
[0085] The spatial attention submodule performs a 3×3 convolution on the intermediate image features, and then processes the convolved intermediate image features through the Sigmoid activation function and outputs them to obtain the second type of image features.
[0086] Understandably, in the image of the reinforcing bars to be identified, for reinforcing bars with irregular cross-sections and various shapes, this embodiment uses Deformable Convolutional Networks (DCN) to enhance geometric transformation capabilities. The constructed spatial attention submodule is as follows: Figure 5 As shown. Additionally, the spatial attention submodule selects three convolutional kernel sizes—1×1, 3×3, and 5×5—to mine features at different scales in parallel, which are the intermediate image features. To obtain weight information, the intermediate image features need to be further convolved with a 3×3 kernel, and then activated by a Sigmoid activation function to map the resulting weights to [0,1]. This yields the second type of image features.
[0087] Convolutional networks (CNNs) have an inherent limitation in modeling large, deformable targets because they only sample fixed positions in the input feature map. Deformable convolutions can improve the model's ability to model deformable targets. By using parallel convolutional layers to learn offsets, the sampling points of the convolutional kernel on the input feature map are shifted, allowing them to focus on regions or targets of interest to the user. In other words, an offset is added to the position of each sampling point in the convolutional kernel, enabling arbitrary sampling near the current position rather than being limited to the regular sampling points of conventional convolutions.
[0088] In this embodiment, the Neck includes a path aggregation network PANet. The Neck fuses the first type of image features and the second type of image features to obtain a third type of image features. This includes: using the path aggregation network PANet to perform information fusion on the first type of image features and the second type of image features to obtain the third type of image features.
[0089] Understandably, PANet enables the fusion of information from different feature layers. PANet better integrates feature information from shallow and deep layers, allowing the network to fully extract features from each layer, thereby enhancing feature extraction and obtaining richer feature information.
[0090] After the Neck inputs the fused third-class image features into the three YOLO Head detectors in the output module Head, the Head can predict the bounding box and class based on the transmitted rebar image features. Redundant predicted boxes are eliminated through Non-Maximum Suppression (NMS), and finally, the predicted class with the highest confidence is output, along with the bounding box coordinates, thus obtaining the anchor box corresponding to the target. The Head determines the target's anchor box in the following way:
[0091] First, the rebar image (i.e., the image to be identified) is divided into 3 S*S grids. Each cell in the grid is responsible for detecting the rebar image falling within the center of that grid, and simultaneously outputs multiple predicted candidate boxes (or bounding boxes) and their confidence scores. Each candidate box contains parameters (t... x ,t y ,t w ,t h ,t c ), where (t) x ,t y ) represents the center coordinates of the candidate box, (t) w ,t h () represents the center point of the candidate box. cThis is an array of confidence scores, containing the confidence score for each target (e.g., rebar). For location prediction, it is assumed that the Anchor Box is set at the top-left corner of each grid cell, with coordinates (c...). x ,c y ), width and height are (p w p h The final predicted coordinates are (b) x ,b y ,b w ,b h The corresponding formulas are as follows:
[0092] b x =(2σ(t) x )-0.5)+c x (5)
[0093] b y =(2σ(t) y )-0.5)+ c y (6)
[0094] b w =p w (2σ(t w )) 2 (7)
[0095] b h =p h (2σ(t h )) 2 (8)
[0096] Among them, (b) x ,b y (b) represents the center coordinates of the predicted candidate bounding box; w ,b h ) represents the height and width of the predicted bounding box; σ refers to the Sigmoid activation function. After outputting multiple predicted candidate boxes, candidate boxes with low confidence are discarded. The selection method can be flexibly determined according to the actual situation. The position of the reinforcing bar is obtained through non-maximum suppression. In this way, the anchor box of the target can be accurately determined through the Head.
[0097] In step 130, after the detection of the image to be recognized in step 120, an anchor box is added to each target in the image to be recognized. For example, see... Figure 6 For images of reinforcing bars with added anchor frames, an anchor frame is added to the cross-section of each reinforcing bar. Electronic devices can count the anchor frames on the image to be recognized, thereby enabling the counting of targets.
[0098] Understandably, if the deep learning model has not been trained before image counting-based detection, the method may further include the following prior to step 110:
[0099] Obtain a training dataset, which includes multiple images, each of which has a target to be detected and is labeled with a corresponding label.
[0100] The pre-built deep learning model is trained using the training dataset to obtain the trained deep learning model.
[0101] In this embodiment, the deep learning model is trained using conventional model training methods, which will not be described in detail here. By training the deep learning model, it acquires the ability to identify and count targets.
[0102] The inventors used 100 images of steel bars to conduct comparative tests on the existing YOLOv5s model and the ShuffleNet-Att-YOLOv5 model provided in this application. The test results of the YOLOv5s and ShuffleNet-Att-YOLOv5 models are shown in Table 1.
[0103] Table 1:
[0104] As shown in Table 1, the improved YOLOv5, while sacrificing a slight decrease in average precision (mAP), reduces model complexity for easier deployment on mobile devices, while maintaining a rebar recognition accuracy of 0.989. The number of model parameters (params) decreased from 7.02M to 3.85M, a reduction of 45.3%; FLOPs decreased from 15.8GFLOPs to 8GFLOPs, a reduction of 49.3%. In other words, the deep learning model ShuffleNet-Att-YOLOv5 in this application can reduce the number of model parameters, storage space, and computational load, while maintaining accuracy in both object detection and counting. This facilitates the deployment of deep learning models on small embedded devices or mobile devices, such as smartphones, reducing the performance requirements of hardware.
[0105] In this embodiment, the processing module can be an integrated circuit chip with signal processing capabilities. The processing module can be a general-purpose processor. For example, the processor can be a Central Processing Unit (CPU), a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application.
[0106] The storage module can be, but is not limited to, random access memory, read-only memory, programmable read-only memory, erasable programmable read-only memory, electrically erasable programmable read-only memory, etc. In this embodiment, the storage module can be used to store images to be recognized, deep learning models, etc. Of course, the storage module can also be used to store programs, which the processing module executes after receiving execution instructions.
[0107] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the electronic device described above can be referred to the corresponding steps in the aforementioned method, and will not be elaborated further here.
[0108] This application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program that, when run on a computer, causes the computer to execute the intelligent construction target counting method as described in the above embodiments.
[0109] Based on the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by hardware or by using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application can be embodied in the form of a software product. This software product can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, mobile hard drive, etc.) and includes several instructions to cause a computer device (such as a personal computer, electronic device, or network device, etc.) to execute the methods described in the various implementation scenarios of this application.
[0110] In summary, this application provides an intelligent target counting method, electronic device, and storage medium. In this scheme, the image to be identified is input into a trained deep learning model to obtain anchor boxes corresponding to each target in the image. Then, all anchor boxes in the image are counted to obtain the number of targets in the image. The deep learning model includes an input module (Input), a backbone network (Backbone), an attention module (Att), a feature fusion layer (Neck), and an output module (Head). Att includes channel attention submodules and spatial attention submodules. The backbone is a ShuffleNet v2 network, and the Head includes three YOLO Head detectors. Because the backbone network is a lightweight ShuffleNet v2 network, compared to the traditional YOLOv5s backbone network, it helps reduce model parameters and computational load. Furthermore, the introduction of a fusion channel and spatial attention mechanism allows for more effective learning and extraction of image features, improving the accuracy of model detection.
[0111] In the embodiments provided in this application, it should be understood that the disclosed apparatus, systems, and methods can also be implemented in other ways. The apparatus, systems, and methods embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code, which includes one or more executable instructions for implementing a specified logical function. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. Furthermore, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.
[0112] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.
Claims
1. A smart construction target counting method, characterized by, The method comprises: acquiring a to-be-recognized image, the to-be-recognized image being an image obtained by photographing a target to be detected; inputting the to-be-recognized image into a trained deep learning model to obtain an anchor box corresponding to each target in the to-be-recognized image output by the deep learning model, wherein the deep learning model comprises an input module Input, a backbone network Backbone, an attention module Att, a feature fusion layer Neck, and an output module Head, the Att comprises a channel attention submodule and a spatial attention submodule, the Backbone is a ShuffleNet v2 network, and the Head comprises three YOLO Head detectors; counting all anchor boxes in the to-be-recognized image to obtain the number of targets in the to-be-recognized image; wherein inputting the to-be-recognized image into the trained deep learning model to obtain an anchor box corresponding to each target in the to-be-recognized image output by the deep learning model comprises: outputting the inputted to-be-recognized image with three channels to the ShuffleNet v2 network through the Input in the deep learning model; extracting a first type of image feature from the to-be-recognized image through the ShuffleNet v2 network; convolving the first type of image feature through the channel attention submodule and the spatial attention submodule in the Att to obtain a second type of image feature; fusing the first type of image feature and the second type of image feature through the Neck to obtain a third type of image feature; predicting a target from the third type of image feature through the three YOLO Head detectors and marking the predicted target with an anchor box in the to-be-recognized image; convolving the first type of image feature through the channel attention submodule and the spatial attention submodule in the Att to obtain a second type of image feature comprises: performing multi-frequency feature fusion on the first type of image feature through the channel attention submodule to obtain fused first type of image feature; performing multi-scale deformable convolution on the fused first type of image feature through the spatial attention submodule to obtain the second type of image feature; performing multi-frequency feature fusion on the first type of image feature through the channel attention submodule to obtain fused first type of image feature comprises: The first type of image features are divided into n parts according to the channel, denoted as [X 0 , X 1 , …, X n-1 ], wherein n is a value that is divisible by the number of channels of the first type of image features. X i , the corresponding two-dimensional DCT frequency components are assigned, denoted as: where i is any integer from 0 to n-1; Fre i ∈R c is X i The processed multi-frequency vector, H is the height of the input X; W is the width of the input X; is the basis function of the two-dimensional discrete cosine transform; [u, v] refers to the two-dimensional index of the frequency component of X i ; [h, w] refers to the parameters in the basis function. by a compression function F sq (X), to obtain the fused first type of image features, wherein the compression function is represented as: F sq (X) = cat([Fre 0 ,Fre 1 ,…,Fre n-1 ]) 2. The method of claim 1, wherein, performing multi-scale deformable convolution on the fused first type of image feature through the spatial attention submodule to obtain the second type of image feature comprises: performing parallel convolution on the fused first type of image feature through the spatial attention submodule using three types of convolution kernels to obtain an intermediate image feature, wherein the sizes of the three types of convolution kernels are 1x1, 3x3, and 5x5, respectively; performing 3x3 convolution on the intermediate image feature through the spatial attention submodule, processing and outputting the convolved intermediate image feature through a sigmod activation function to obtain the second type of image feature.
3. The method of claim 1, wherein, The Neck comprises a path aggregation network (PANet), and the first type of image feature and the second type of image feature are fused by the Neck to obtain a third type of image feature, comprising: The first type of image feature and the second type of image feature are information intermingled by a path aggregation network (PANet) to obtain the third type of image feature.
4. The method of claim 1, wherein, Before the image to be recognized is acquired, the method further comprises: A training data set is acquired, the training data set comprising a plurality of images, each of the plurality of images having a target to be detected and being provided with a label corresponding to the target; The pre-built deep learning model is trained by using the training data set to obtain the trained deep learning model.
5. The method according to any one of claims 1-4, characterized in that, The target comprises any one of a reinforcing bar, a wooden column and a pipe.
6. An electronic device, comprising: The electronic device comprises a processor and a memory coupled to each other, and the memory stores a computer program, which, when executed by the processor, causes the electronic device to perform the method according to any one of claims 1-5.
7. A computer readable storage medium characterized by, The computer readable storage medium stores a computer program, which, when executed on a computer, causes the computer to perform the method according to any one of claims 1-5.