A method of counting objects and a computing device

Abstract

The present disclosure discloses a method for counting objects and a computing device. The method for counting objects comprises: inputting an image containing objects to be detected into a feature extraction component, and outputting feature extraction images of different sizes after processing by at least a local cross-stage network and a down-sampling layer; inputting the feature extraction images of different sizes into a feature fusion component, and outputting feature fusion images of corresponding sizes after processing; inputting the feature fusion images into a prediction component to obtain at least a prediction box containing the objects, and counting the number of the objects based on the prediction box.

Description

Technical Field

[0001] This disclosure relates to the field of computer vision technology, and more particularly to a method and computing device for counting objects. Background Technology

[0002] With the continuous development of computer vision technology, intelligent counting systems based on computer vision have emerged to automatically count objects such as industrial parts, electronic components, and goods. This helps to liberate workers from tedious and simple counting tasks, reducing their workload. Taking steel bars—a common material on construction sites—as an example, in actual construction scenarios, the quantity of steel bars entering the site is usually counted manually. However, due to the large quantity arriving each time, a significant amount of manpower is required to complete the counting. Manual counting places a heavy burden on workers, and the accuracy is not guaranteed, especially under adverse weather conditions where the error rate is high. Moreover, steel bars are expensive, and counting errors can lead to economic losses for the construction company.

[0003] Currently, computer vision-based counting systems are mainly divided into counting methods based on traditional image processing and counting methods based on deep learning. Continuing with the example of rebar, traditional image processing-based rebar counting methods typically require first performing image enhancement techniques on the incoming rebar image before counting the rebar within the image. However, this approach is usually redundant and complex, with errors in any intermediate step propagating to subsequent steps, leading to insufficient accuracy. Furthermore, it lacks robustness against interference, and its counting accuracy is insufficient for rebar cross-sections with irregular shapes, significant color variations, or large scale differences. Compared to traditional image processing-based rebar counting schemes, deep learning-based rebar counting schemes are simpler in steps, easier to implement, and offer significantly improved robustness against interference and counting accuracy. However, this approach typically suffers from high computational resource requirements, hindering model deployment in real-world scenarios; and its counting accuracy is insufficient for extremely dense rebar images.

[0004] Therefore, a new calculation scheme is needed for objects such as reinforcing bars. Summary of the Invention

[0005] This disclosure provides a method and computing device for counting objects in an attempt to solve or at least alleviate at least one of the problems mentioned above.

[0006] According to one aspect of this disclosure, a method for counting objects is provided, comprising: inputting a detection image containing objects into a feature extraction component, processing it at least through a local cross-stage network and a downsampling layer, and outputting feature extraction maps of different sizes; inputting the feature extraction maps of different sizes into a feature fusion component, processing them, and outputting feature fusion maps of corresponding sizes respectively; inputting the feature fusion maps into a prediction component to obtain at least a prediction box containing objects, and counting the number of objects based on the prediction boxes.

[0007] Optionally, in the method according to this disclosure, the feature extraction component includes coupled convolutional layers and the same number of local cross-stage networks and downsampling layers, with each downsampling layer correspondingly coupled before each local cross-stage network, wherein the number of local cross-stage networks and downsampling layers is 5.

[0008] Optionally, in the method according to this disclosure, the convolutional layer includes a convolutional processing block, which sequentially includes convolution, batch normalization, and activation function; the local cross-stage network includes a residual convolutional processing block, and the number of residual convolutional processing blocks is different in each local cross-stage network; the downsampling layer includes a convolutional processing block.

[0009] Optionally, the method according to this disclosure further includes: inputting the image to be detected containing the object into a convolutional layer, processing it, and outputting a preliminary feature extraction map; downsampling the preliminary feature extraction map to obtain a corresponding downsampled feature map, and processing it through a local cross-stage network to generate a first feature extraction map; downsampling the first feature extraction map to obtain a corresponding downsampled feature map, and processing it through a local cross-stage network to generate a second feature extraction map; downsampling the second feature extraction map to obtain a corresponding downsampled feature map, and processing it through a local cross-stage network to generate a third feature extraction map; downsampling the third feature extraction map to obtain a corresponding downsampled feature map, and processing it through a local cross-stage network to generate a fourth feature extraction map; downsampling the fourth feature extraction map to obtain a corresponding downsampled feature map, and processing it through a local cross-stage network to generate a fifth feature extraction map, wherein the first feature extraction map, the second feature extraction map, the third feature extraction map, the fourth feature extraction map, and the fifth feature extraction map have different sizes.

[0010] Optionally, in the method according to this disclosure, the residual convolution processing block includes: a first branch adapted to output the input feature map to the fusion module; a second branch including two convolution processing blocks and one attention module, wherein the convolution processing blocks are adapted to perform convolution processing on the input feature map, the attention module is adapted to enhance the features by generating channel weight vectors, and output the enhanced feature map to the fusion module; and a fusion module adapted to fuse the enhanced feature map with the input feature map to obtain a feature extraction map of the corresponding size.

[0011] Optionally, in the method according to this disclosure, the attention module is further adapted to process the input feature map to generate a channel weight vector; and multiply the input feature map with the channel weight vector to generate an enhanced feature map.

[0012] Optionally, in the method according to this disclosure, the channel weight vector is generated in the following manner:

[0013] M c (F) * =σ(MLP(δ(MLP(MaxPool(AvgPool(F)))))),

[0014] Where F represents the feature map, M c (F) * denoted as channel weight vector, AvgPool represents global average pooling, MaxPool represents global max pooling, MLP represents fully connected layer, δ represents ReLU activation function, and σ represents Sigmoid activation function.

[0015] Optionally, the method according to this disclosure further includes: selecting a third feature extraction map, a fourth feature extraction map, and a fifth feature extraction map as inputs to a feature fusion component; processing the fifth feature extraction map using a deep convolution module, and fusing the processed feature map with the fourth feature extraction map to obtain a first sub-feature map; processing the fourth feature extraction map using a deep convolution module, and fusing the processed feature map with the third feature extraction map to obtain a second sub-feature map; processing the fifth feature extraction map, the first sub-feature map, and the second sub-feature map using a deep convolution module to obtain a first fused feature map, a second fused feature map, and a third fused feature map respectively; processing the third fused feature map using a deep convolution module, and fusing the processed feature map with the second fused feature map to obtain a fourth fused feature map; processing the fourth fused feature map using a deep convolution module, and fusing the processed feature map with the first fused feature map to obtain a fifth fused feature map; and processing the third fused feature map, the fourth fused feature map, and the fifth fused feature map using a deep convolution module and a convolution module to output the first fused feature map, the second fused feature map, and the third fused feature map respectively.

[0016] Optionally, in the method according to this disclosure, the depth convolution module includes coupled depth hyperparameterized convolutional layers, batch normalization layers, and activation functions, wherein the convolution kernel of the depth hyperparameterized convolutional layer is composed of depth convolution kernels and conventional convolution kernels.

[0017] Optionally, in the method according to this disclosure, the object includes: industrial parts, electronic components, and goods.

[0018] According to another aspect of this disclosure, a computing device is provided, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for performing any of the methods described above.

[0019] According to another aspect of this disclosure, a computer-readable storage medium is provided for storing one or more programs, the one or more programs including instructions that, when executed by a computing device, cause the computing device to perform any of the methods described above.

[0020] In summary, based on the scheme disclosed herein, a deep learning-based automatic counting model is proposed to address the problem of insufficient counting accuracy in extremely dense rebar images. In the feature extraction component, an improved attention module (A) is used to enhance the model's effective feature extraction for objects with complex backgrounds and large scale variations, such as rebar. Furthermore, a feature fusion component is employed to perform multi-scale feature fusion, thereby strengthening the model's ability to recognize dense, small targets such as single rebars, significantly improving the model's counting accuracy.

[0021] The above description is merely an overview of the technical solution disclosed herein. In order to better understand the technical means of this disclosure and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this disclosure more apparent and understandable, specific embodiments of this disclosure are described below. Attached Figure Description

[0022] To achieve the foregoing and related objectives, certain illustrative aspects are described herein in conjunction with the following description and accompanying drawings. These aspects indicate various ways in which the principles disclosed herein may be practiced, and all aspects and their equivalents are intended to fall within the scope of the claimed subject matter. The foregoing and other objectives, features, and advantages of this disclosure will become more apparent from the following detailed description, taken in conjunction with the accompanying drawings. Throughout this disclosure, the same reference numerals generally refer to the same parts or elements.

[0023] Figure 1 A schematic diagram of a computing device 100 according to some embodiments of the present disclosure is shown;

[0024] Figure 2 A flowchart illustrating a method 200 for counting objects according to some embodiments of the present disclosure is shown;

[0025] Figure 3 A schematic diagram illustrating data enhancement according to some embodiments of the present disclosure is shown;

[0026] Figure 4 A schematic diagram of the structure of a residual convolution processing block according to some embodiments of the present disclosure is shown;

[0027] Figure 5 A schematic diagram of the structure of a feature fusion component according to some embodiments of the present disclosure is shown;

[0028] Figure 6 A schematic diagram of the convolution kernel of a depth hyperparameterized convolutional layer according to some embodiments of the present disclosure is shown. Detailed Implementation

[0029] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0030] Using computer vision technology to process images containing objects (including industrial parts, electronic components, goods, etc., according to embodiments of this disclosure) to achieve object counting is susceptible to the influence of the image shooting angle. For example, different shooting angles will make the size of the objects appear different, even for the same model, the variation can be significant. In addition, some parts and goods are usually counted in bundles, such as steel bars, where different steel bars are in close contact, making it difficult to identify the boundaries of individual steel bars, thus leading to counting errors.

[0031] Therefore, to address these problems, this disclosure proposes a method for automatically counting objects such as reinforcing bars. This method can be implemented using one or more computing devices.

[0032] Figure 1 A structural block diagram of a computing device 100 according to some embodiments of the present disclosure is shown.

[0033] like Figure 1 As shown, in the basic configuration 102, the computing device 100 typically includes a system memory 106 and one or more processors 104. A memory bus 108 can be used for communication between the processors 104 and the system memory 106.

[0034] Depending on the desired configuration, processor 104 can be any type of processor, including but not limited to: microprocessors (μP), microcontrollers (μC), digital information processors (DSPs), or any combination thereof. Processor 104 may include one or more levels of cache such as L1 cache 110 and L2 cache 112, processor core 114, and registers 116. Example processor core 114 may include an arithmetic logic unit (ALU), a floating-point unit (FPU), a digital signal processing (DSP) core, or any combination thereof. Example memory controller 118 may be used with processor 104, or in some implementations, memory controller 118 may be an internal part of processor 104.

[0035] Depending on the desired configuration, system memory 106 can be any type of memory, including but not limited to volatile memory (such as RAM), non-volatile memory (such as ROM, flash memory, etc.), or any combination thereof. Physical memory in a computing device typically refers to volatile RAM, and data on a disk needs to be loaded into physical memory before it can be read by processor 104. System memory 106 may include operating system 120, one or more applications 122, and program data 124. In some embodiments, application 122 may be arranged to execute instructions on the operating system using program data 124 by one or more processors 104. Operating system 120 may be, for example, Linux, Windows, etc., and includes program instructions for handling basic system services and performing hardware-dependent tasks. Application 122 includes program instructions for implementing various user-desired functions, and application 122 may be, for example, a browser, instant messaging software, software development tools (such as integrated development environments IDEs, compilers, etc.), but is not limited to these.

[0036] When computing device 100 starts up, processor 104 reads and executes program instructions from memory 106 of operating system 120. Application 122 runs on operating system 120, utilizing interfaces provided by operating system 120 and underlying hardware to implement various user-expected functions. When user starts application 122, application 122 is loaded into memory 106, and processor 104 reads and executes program instructions from memory 106 of application 122.

[0037] The computing device 100 also includes a storage device 132, which includes a removable storage device 136 (e.g., CD, DVD, USB flash drive, portable hard drive, etc.) and a non-removable storage device 138 (e.g., hard disk drive, HDD, etc.). Both the removable storage device 136 and the non-removable storage device 138 are connected to the storage interface bus 134.

[0038] The computing device 100 may also include a storage interface bus 134. The storage interface bus 134 enables communication from storage devices 132 (e.g., removable storage 136 and non-removable storage 138) to the basic configuration 102 via a bus / interface controller 130. At least a portion of the operating system 120, application 122, and program data 124 may be stored on the removable storage 136 and / or the non-removable storage 138, and loaded into system memory 106 via the storage interface bus 134 when the computing device 100 is powered on or when application 122 is to be executed, and executed by one or more processors 104.

[0039] The computing device 100 may also include an interface bus 140 that facilitates communication from various interface devices (e.g., output devices 142, peripheral interfaces 144, and communication devices 146) to the basic configuration 102 via a bus / interface controller 130. Example output devices 142 include a graphics processing unit 148 and an audio processing unit 150. They may be configured to facilitate communication with various external devices such as displays or speakers via one or more A / V ports 152. Example peripheral interfaces 144 may include a serial interface controller 154 and a parallel interface controller 156, which may be configured to facilitate communication with external devices such as input devices (e.g., keyboards, mice, pens, voice input devices, touch input devices) or other peripherals (e.g., printers, scanners, etc.) via one or more I / O ports 158. Example communication devices 146 may include a network controller 160, which may be arranged to facilitate communication with one or more other computing devices 162 via a network communication link through one or more communication ports 164.

[0040] A network communication link can be an example of a communication medium. A communication medium can typically be embodied in a modulated data signal, such as a carrier wave or other transmission mechanism, and can include any information delivery medium. A “modulated data signal” can be a signal whose data set, or its modifications, can be encoded as information within the signal. As a non-limiting example, a communication medium can include wired media such as wired networks or leased lines, and various wireless media including sound, radio frequency (RF), microwave, infrared (IR), or other wireless media. The term “computer-readable medium” as used herein can include both storage media and communication media.

[0041] The computing device 100 can be implemented as a personal computer, including desktop and laptop computer configurations. Of course, the computing device 100 can also be implemented as part of a small-sized portable (or mobile) electronic device, such as a cellular phone, digital camera, personal digital assistant (PDA), personal media player device, wireless network browsing device, personal head-mounted device, application-specific device, or a hybrid device that may include any of the above functions. It can even be implemented as a server, such as a file server, database server, application server, and web server. The embodiments of the present invention do not limit this.

[0042] In an embodiment according to this disclosure, computing device 100 is configured to execute a method 200 for counting objects according to this disclosure. An application 122 arranged on an operating system includes multiple program instructions for executing method 200, which may instruct processor 104 to execute the above-described method of this disclosure to count specific objects.

[0043] Figure 2 A flowchart illustrating a method 200 for counting objects according to some embodiments of the present disclosure is shown. According to embodiments of the present disclosure, method 200 generates an end-to-end automatic counting model through training. A collected image containing objects is input into this automatic counting model to count the number of objects contained in the image. Taking rebar as an example, inputting a cross-sectional image of the rebar into the automatic counting model directly outputs the number of rebars.

[0044] According to some embodiments, method 200 also includes a process of constructing and training an automatic counting model. The automatic counting model includes coupled feature extraction components, feature fusion components, and prediction components.

[0045] The following example, using steel bars as the object, illustrates the execution process of method 200.

[0046] When training the automatic counting model, the DCIC rebar counting dataset and the rebar data collected in this publication were used for model training. Furthermore, data augmentation techniques (such as random pruning, random affine transformation, random rotation, etc.) were employed to effectively expand the dataset. Figure 3 As shown, three data augmentation methods—random cropping (a), random affine transformation (b), and random rotation (c)—were applied to the training set. Then, the original image and the data augmentation results were fused using a mix-up approach (d), resulting in a total of 594 images for both the training and test sets. This disclosure improves the model's ability to extract essential features of reinforcing bars and its ability to locate reinforcing bars by fusing data from different modalities, thereby enhancing the model's accuracy.

[0047] Furthermore, the model is implemented using the PyTorch framework and employs Multi-Scale Training (MST) to improve its ability to recognize different scales and enhance the statistical accuracy of the number of steel bars in steel bar cross-section images with significant scale variations. Specifically, for each training batch, a scale is randomly selected from {352, ..., 608} as the image input size.

[0048] like Figure 2 As shown in 210, the image to be detected containing the object is input into the feature extraction component, and after processing by at least a local cross-stage network and a downsampling layer, feature extraction maps of different sizes are output.

[0049] According to some embodiments of this disclosure, the feature extraction component includes convolutional layers, local cross-stage networks, and downsampling layers, with the number of local cross-stage networks and downsampling layers being the same. Thus, each downsampling layer is correspondingly coupled before each local cross-stage network. In some embodiments, the number of local cross-stage networks and the downsampling layers is 5.

[0050] The convolutional layer comprises a convolutional processing block (C), which sequentially includes convolution, batch normalization (BN), and an activation function. In one embodiment, the activation function is the Mish activation function. The local cross-stage network includes at least a residual convolutional processing block (R), and the number of residual convolutional processing blocks (R) varies in each local cross-stage network. In some embodiments, the local cross-stage network may further include one or more convolutional processing blocks coupled to the residual convolutional processing blocks to enhance feature reuse. This disclosure does not impose further limitations in this regard. The downsampling layer also achieves downsampling of the input features through convolutional operations. According to one embodiment of this disclosure, all convolutional operations in the feature extraction component can be implemented using the convolutional processing block (C) as described above.

[0051] In each residual convolution processing block (R), the input features are divided into two parts: one part undergoes a convolution operation, and the other part is directly fused with the convolution output. This enhances feature reuse and improves model performance while significantly reducing the model's peak memory requirements and computational bottleneck.

[0052] Figure 4 A schematic diagram of the structure of a residual convolution processing block according to some embodiments of the present disclosure is shown.

[0053] The residual convolutional processing block (R) comprises two branches and a fusion module. The first branch directly outputs the input feature map to the fusion module. The second branch includes two convolutional processing blocks (C) and one attention module (A), which sequentially perform convolution processing on the input feature map and enhance the feature map by generating channel weight vectors, then output the enhanced feature map to the fusion module. Finally, the fusion module fuses the enhanced feature map with the input feature map to obtain a feature extraction map of the corresponding size. Specifically, the attention module (A) enhances the input features as follows: first, it processes the input feature map (i.e., the feature map output by the second convolutional processing block (C)) to generate channel weight vectors; then, it multiplies the input feature map with the channel weight vectors to generate the enhanced feature map.

[0054] The SE channel attention mechanism is widely used in feature extraction networks to enhance the network's ability to extract effective features for complex targets. The SE channel attention module usually determines the channel weight vector in the following way, as shown in Equation (1). First, global average pooling is used to compress the feature map, then two fully connected layers are used to associate different channel features with the ReLU activation function, and finally the Sigmoid function is used to obtain the channel weight vector M. c (F). The input feature map is then multiplied by the channel weight vector to finally output a feature map containing the channel weights, which is the enhanced feature map.

[0055] M c (F)=σ(MLP(δ(MLP(AvgPool(F))))) (1)

[0056] Where F represents the feature map, AvgPool represents global average pooling, MLP represents a fully connected layer, δ represents the ReLU activation function, and σ represents the Sigmoid activation function.

[0057] However, global average pooling cannot accurately reflect the response of the channel to the rebar features. Therefore, in the embodiments according to this disclosure, the above-mentioned SE channel attention module is improved and fused into the feature extraction network as attention module (A). Attention module (A) generates the channel weight vector using the following formula (2):

[0058] M c (F) * =σ(MLP(δ(MLP(MaxPool(AvgPool(F)))))) (2)

[0059] Where F represents the feature map, M c (F) *denoted as channel weight vector, AvgPool represents global average pooling, MaxPool represents global max pooling, MLP represents fully connected layer, δ represents ReLU activation function, and σ represents Sigmoid activation function.

[0060] In this way, the attention module (A) reacts to the channel response through both global max pooling and average pooling.

[0061] According to some embodiments of this disclosure, after the image to be detected is input into the feature extraction component, it is processed sequentially through "convolutional layer → downsampling layer 1 (i.e., the first downsampling layer, and so on) → local cross-stage network 1 → downsampling layer 2 → local cross-stage network 2 → downsampling layer 3 → local cross-stage network 3 → downsampling layer 4 → local cross-stage network 4 → downsampling layer 5 → local cross-stage network 5", and after processing by each local cross-stage network, the corresponding feature extraction map is output.

[0062] The specific processing procedure is as follows.

[0063] The image to be detected is input into a convolutional layer, and after convolution processing, a preliminary feature extraction map is output. Downsampling layer 1 is used to downsample the preliminary feature extraction map, resulting in a downsampled feature map. This downsampled feature map is then processed by a local cross-stage network 1 to generate the first feature extraction map. Downsampling layer 2 is used to downsample the first feature extraction map, resulting in a downsampled feature map. This downsampled feature map is then processed by a local cross-stage network 2 to generate the second feature extraction map. Downsampling layer 3 is used to downsample the second feature extraction map, resulting in a downsampled feature map. This downsampled feature map is then processed by a local cross-stage network 3 to generate the third feature extraction map. Downsampling layer 4 is used to downsample the third feature extraction map, resulting in a downsampled feature map. This downsampled feature map is then processed by a local cross-stage network 4 to generate the fourth feature extraction map. Downsampling layer 5 is used to downsample the fourth feature extraction map, resulting in a downsampled feature map. This downsampled feature map is then processed by a local cross-stage network 5 to generate the fifth feature extraction map. The first, second, third, fourth, and fifth feature extraction maps have different sizes.

[0064] In some embodiments, the number of residual convolution processing blocks (R) from local cross-stage network 1 to local cross-stage network 5 are 1, 2, 8, 8, and 4, respectively.

[0065] Typically, in addition to constructing the structure of the feature extraction component, it is also necessary to set the network parameters of the feature extraction component accordingly. Table 1 shows some network parameters of the feature extraction component according to some embodiments of this disclosure. It should be noted that Table 1 mainly shows the relevant parameters of the convolution kernel. For local cross-stage networks, only the convolution kernel parameters of the two convolution processing blocks (C) in the residual convolution processing block (R) are shown. In addition, in some embodiments, the stride of the downsampling layer is 2.

[0066] Table 1. Partial network parameters of the feature extraction component

[0067]

[0068]

[0069] In summary, the dimensions of the first feature extraction map, the second feature extraction map, the third feature extraction map, the fourth feature extraction map, and the fifth feature extraction map are, respectively, 304×304, 152×152, 76×76, 38×38, and 19×19.

[0070] In 220, feature extraction maps of different sizes are input into the feature fusion component, and after processing, feature fusion maps of the corresponding sizes are output.

[0071] According to the embodiments of this disclosure, a third feature extraction map (76×76), a fourth feature extraction map (38×38), and a fifth feature extraction map (19×19) are selected as inputs to the feature fusion component.

[0072] Because the feature extraction component involves five downsampling operations, small target features such as single rebars are lost, leading to a decrease in the accuracy of rebar counting. Therefore, this disclosure designs a network structure for the feature fusion component, such as... Figure 5 As shown, the outputs of the cross-stage local network 3-5 are fused in two directions, from top to bottom and from bottom to top. By fusing the steel bar features at different scales, more comprehensive steel bar feature information is obtained, thereby improving the accuracy of steel bar counting.

[0073] According to some embodiments, the feature fusion component includes multiple deep convolutional modules and convolutional modules. For example... Figure 5As shown, the fifth feature extraction map F5 is processed by the deep convolution module, and the processed feature map is fused with the fourth feature extraction map F4 to obtain the first sub-feature map f1. Similarly, the fourth feature extraction map F4 is processed by the deep convolution module, and the processed feature map is fused with the third feature extraction map F3 to obtain the second sub-feature map f2. Then, the deep convolution module is used to process the fifth feature extraction map F5, the first sub-feature map f1, and the second sub-feature map f2 respectively to obtain the first fused feature map T1, the second fused feature map T2, and the third fused feature map T3. According to some embodiments, multiple deep convolution modules can be used when processing the fifth feature extraction map F5, the first sub-feature map f1, and the second sub-feature map f2 respectively, such as... Figure 5 In the diagram, "×5" represents the processing of F5 / f1 / f2 by five coupled deep convolutional modules. Next, the third fused feature map T3 is processed by a deep convolutional module, and the processed feature map is fused with the second fused feature map T2 to obtain the fourth fused feature map T4. The fourth fused feature map T4 is then processed by a deep convolutional module, and the processed feature map is fused with the first fused feature map T1 to obtain the fifth fused feature map T5. Finally, the coupled deep convolutional and convolutional modules process the third fused feature map T3, the fourth fused feature map T4, and the fifth fused feature map T5 respectively, outputting the first fused feature map M1, the second fused feature map M2, and the third fused feature map M3.

[0074] In some embodiments, the convolution module may employ a conventional convolution kernel, which will not be elaborated here. The depth convolution module includes a coupled depth-wise over parameterized convolutional layer (Do-Conv), a batch normalization (BN) layer, and an activation function. Do-Conv is used instead of conventional convolution to further enhance the texture features of the rebar cross-section.

[0075] Figure 6 A schematic diagram of the Do-Conv convolution kernel is shown. Figure 6 As shown, the Do-Conv convolution kernel is composed of a depthwise convolution kernel D. T The convolution kernel consists of two convolutions: ∈W*H*Cin*Dmul and the traditional convolution kernel w∈Cout*Dmul*Cin. First, a depthwise convolution operation is performed on the two kernels to obtain a combined convolution kernel K. do ∈W*H*Cin*Cout, and then use this combined convolution kernel to perform traditional convolution with the input feature map. By superimposing depthwise convolution, the performance of this convolution kernel is enhanced, and the fusion of steel reinforcement features at multiple scales is improved.

[0076] At the same time, during the reasoning phase, K is used directly.do The kernel parameters fold two convolution operations into a single convolution operation to ensure the real-time performance of the feature fusion component.

[0077] In step 230, the feature fusion map is input into the prediction component to obtain at least a prediction box containing the object, and based on these prediction boxes, the number of objects is counted.

[0078] According to some embodiments of this disclosure, the prediction component ultimately outputs a 4D rebar coordinate vector, a 1D rebar confidence vector, and a 1D rebar category vector. Based on the rebar coordinate vector, a preliminary bounding box containing the object can be obtained. Then, a non-maximum suppression method is used to filter out the correct bounding boxes; the number of correct bounding boxes represents the number of rebars.

[0079] In some embodiments, the prediction component may also refer to the Head component in YOLOv4, which will not be described in detail here. Furthermore, in other embodiments, the YOLOv4-based Head component employs the Soft-NMS non-maximum suppression method to filter the prediction boxes. This disclosure does not impose excessive limitations in these respects.

[0080] According to the scheme of this disclosure, in the feature extraction component, an improved attention module (A) is used to enhance the model's effective feature extraction for objects with complex backgrounds and large scale variations, such as steel bars. Furthermore, this disclosure designs a feature fusion component to perform multi-scale feature fusion, thereby enhancing the model's ability to recognize dense small targets such as single steel bars, and utilizes the Soft-NMS nonmaximum suppression method to improve the model's counting accuracy.

[0081] The image of the steel bar cross-section is input into the automatic counting model of this disclosure. After processing by the aforementioned feature extraction component, feature fusion component, and prediction component, the final predicted bounding box is obtained. According to statistics, the prediction results of this disclosure have a higher accuracy rate. Table 2 shows a comparison of the recognition accuracy of several models.

[0082] Table 2 Comparison of recognition accuracy of several models

[0083] Model F1 score SSD 0.55 Faster Rcnn 0.861 Yolov4 0.974 The automatic counting model disclosed herein 0.996

[0084] In summary, to address the problem of insufficient counting accuracy in extremely dense rebar images, this disclosure proposes an automatic counting model based on deep learning. This model can accurately count the number of rebars entering the construction site. Furthermore, the model is completely end-to-end, directly inputting rebar cross-sectional images and directly outputting the number of rebars. Compared with existing methods, this disclosure has significant improvements in statistical accuracy and real-time performance.

[0085] The various techniques described herein can be implemented in combination with hardware or software, or a combination thereof. Thus, the methods and apparatus of this disclosure, or certain aspects or portions thereof, may take the form of program code (i.e., instructions) embedded in a tangible medium, such as a removable hard disk, USB flash drive, floppy disk, CD-ROM, or any other machine-readable storage medium, wherein when the program is loaded into and executed by a machine such as a computer, the machine becomes an apparatus for practicing this disclosure.

[0086] When the program code is executed on a programmable computer, the computing device generally includes a processor, a processor-readable storage medium (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device. The memory is configured to store program code; the processor is configured to execute the method of counting objects disclosed herein according to instructions in the program code stored in the memory.

[0087] By way of example, and not limitation, readable media include readable storage media and communication media. Readable storage media stores information such as computer-readable instructions, data structures, program modules, or other data. Communication media generally embodies computer-readable instructions, data structures, program modules, or other data in the form of modulated data signals such as carrier waves or other transmission mechanisms, and includes any information delivery medium. Any combination of the above is also included within the scope of readable media.

[0088] This disclosure also includes:

[0089] A9. The method described in A8, wherein the depth convolution module includes a coupled depth hyperparameterized convolutional layer, a batch normalization layer and an activation function, wherein the convolution kernel of the depth hyperparameterized convolutional layer is composed of a depth convolution kernel and a conventional convolution kernel.

[0090] A10. The method as described in any one of A1-9, wherein the object includes: industrial parts, electronic components, and goods.

[0091] In the specification provided herein, the algorithms and displays are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used with the examples of this disclosure. Based on the above description, the required structure for constructing such systems is apparent. Furthermore, this disclosure is not directed to any particular programming language. It should be understood that the contents of this disclosure described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing preferred embodiments of this disclosure.

[0092] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this disclosure may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0093] Similarly, it should be understood that, for the sake of brevity and to aid in understanding one or more of the various aspects of the disclosure, in the foregoing description of exemplary embodiments of the disclosure, various features of the disclosure are sometimes grouped together in a single embodiment, figure, or description thereof. However, this approach to disclosure should not be construed as reflecting an intention that the claimed disclosure requires more features than are expressly recited in each claim. Rather, as reflected in the following claims, the aspects of the disclosure consist of fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of the disclosure.

[0094] Those skilled in the art will understand that modules, units, or components of the devices disclosed in the examples herein can be arranged in the devices described in this embodiment, or alternatively, can be located in one or more devices different from the devices in this example. The modules in the foregoing examples can be combined into a single module or, in addition, can be divided into multiple sub-modules.

[0095] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0096] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features included in other embodiments but not others, combinations of features from different embodiments are intended to be within the scope of this disclosure and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0097] Furthermore, some of the embodiments described herein are methods or combinations of method elements that can be implemented by a processor of a computer system or by other means of performing the functions. Therefore, a processor having the necessary instructions for implementing the method or method elements forms means for implementing the method or method elements. Furthermore, the elements described herein in the apparatus embodiments are examples of means for implementing the functions performed by the elements for the purposes of this disclosure.

[0098] As used herein, unless otherwise specified, the use of ordinal numbers such as “first,” “second,” “third,” etc., to describe ordinary objects merely indicates different instances of similar objects and is not intended to imply that the objects being described must have a given order in time, space, ordering, or any other manner.

[0099] Although this disclosure has been described with reference to a limited number of embodiments, those skilled in the art will understand from the foregoing description that other embodiments are conceivable within the scope of this disclosure. Furthermore, it should be noted that the language used in this specification has been chosen primarily for readability and edibility purposes, and not for interpreting or limiting the subject matter of this disclosure. Therefore, many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the appended claims. Regarding the scope of this disclosure, the disclosure is illustrative and not restrictive, and the scope of this disclosure is defined by the appended claims.

Claims

1. A method for counting objects, comprising: An image containing an object to be detected is input into a feature extraction component. After processing by at least a local cross-stage network and a downsampling layer, feature extraction maps of different sizes are output. The feature extraction component includes coupled convolutional layers and the same number of local cross-stage networks and downsampling layers, with each downsampling layer correspondingly coupled before each local cross-stage network. The number of local cross-stage networks and downsampling layers is 5. Each convolutional layer includes a convolutional processing block, which sequentially includes convolution, batch normalization, and an activation function. Each local cross-stage network includes a residual convolutional processing block, and the number of residual convolutional processing blocks is different in each local cross-stage network. Each downsampling layer includes a convolutional processing block. The feature extraction maps of different sizes are input into the feature fusion component, and after processing, feature fusion maps of corresponding sizes are output respectively. The feature fusion map is input into the prediction component to obtain at least a prediction bounding box containing the object, and the number of the object is counted based on the prediction bounding box; The step of inputting the image to be detected containing the object into the feature extraction component, and after processing by at least a local cross-stage network and a downsampling layer, outputting feature extraction maps of different sizes, includes: The image to be detected, containing the object, is input into the convolutional layer, and after processing, a preliminary feature extraction map is output. The preliminary feature extraction map is downsampled to obtain the corresponding downsampled feature map, which is then processed by a local cross-stage network to generate the first feature extraction map. The first feature extraction map is downsampled to obtain a corresponding downsampled feature map, which is then processed by a local cross-stage network to generate a second feature extraction map. The second feature extraction map is downsampled to obtain the corresponding downsampled feature map, which is then processed by a local cross-stage network to generate the third feature extraction map. The third feature extraction map is downsampled to obtain a corresponding downsampled feature map, which is then processed by a local cross-stage network to generate a fourth feature extraction map. The fourth feature extraction map is downsampled to obtain a corresponding downsampled feature map, which is then processed by a local cross-stage network to generate the fifth feature extraction map. The first feature extraction map, the second feature extraction map, the third feature extraction map, the fourth feature extraction map, and the fifth feature extraction map have different sizes; The residual convolution processing block includes: The first branch is suitable for outputting the input feature map to the fusion module; The second branch includes two convolutional processing blocks and one attention module. The convolutional processing blocks are adapted to perform convolutional processing on the input feature map, and the attention module is adapted to enhance the features by generating channel weight vectors and output the enhanced feature map to the fusion module. The fusion module is adapted to fuse the enhanced feature map with the input feature map to obtain a feature extraction map of the corresponding size; The step of inputting feature extraction maps of different sizes into the feature fusion component, and outputting feature fusion maps of corresponding sizes after processing, includes: The third feature extraction map, the fourth feature extraction map, and the fifth feature extraction map are selected as inputs to the feature fusion component; The fifth feature map is processed by the depth convolution module, and the processed feature map is fused with the fourth feature map to obtain the first sub-feature map; The fourth feature map is processed by the depthwise convolution module, and the processed feature map is fused with the third feature map to obtain the second sub-feature map; The fifth feature extraction map, the first sub-feature map, and the second sub-feature map are processed by the depthwise convolution module to obtain the first fused feature map, the second fused feature map, and the third fused feature map respectively. The third fused feature map is processed by the depthwise convolution module, and then fused with the second fused feature map to obtain the fourth fused feature map. The fourth fusion feature map is processed by the depth convolution module, and then fused with the first fusion feature map to obtain the fifth fusion feature map. The third fused feature map, the fourth fused feature map, and the fifth fused feature map are processed by a depthwise convolution module and a convolution module respectively, and the first feature fused map, the second feature fused map, and the third feature fused map are output accordingly.

2. The method as described in claim 1, wherein, The attention module is also adapted to, The feature map in the input is processed to generate channel weight vectors; The feature map in the input is multiplied by the channel weight vector to generate the enhanced feature map.

3. The method as described in claim 2, wherein, The channel weight vector is generated as follows: M c (F) * =σ(MLP(δ(MLP(MaxPool(AvgPool(F)))))), Where F represents the feature map, M c (F) * denoted as channel weight vector, AvgPool represents global average pooling, MaxPool represents global max pooling, MLP represents fully connected layer, δ represents ReLU activation function, and σ represents Sigmoid activation function.

4. The method of claim 1, wherein, The depth convolution module includes a coupled depth hyperparameterized convolutional layer, a batch normalization layer, and an activation function, wherein the convolution kernel of the depth hyperparameterized convolutional layer is composed of a depth convolution kernel and a traditional convolution kernel.

5. The method according to any one of claims 1-4, wherein, The objects include: industrial parts, electronic components, and goods.

6. A computing device, comprising: One or more processors; Memory; One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing the method as described in any one of claims 1-5.

7. A computer-readable storage medium storing one or more programs, said one or more programs including instructions that, when executed by a computing device, cause the computing device to perform the method as described in any one of claims 1-5.

Images