Reinforcing steel bar semi-finished product point inspection embedded system based on point counting

Abstract

The invention discloses a reinforcing steel bar semi-finished product point inspection embedded system based on point counting, and the system comprises the following steps: employing an embedded development board Orange Pi device carrying an external visible light camera to shoot a diagonal slope image of a reinforcing steel bar semi-finished product of a construction site; the collected image is preprocessed and then sent to an NPU processor of an embedded development board Orange Pi to be processed; detecting and counting the steel bar semi-finished product image by using a detecting and counting function in the system, wherein a model adopted by the detecting and counting function is based on an improved target detection network P2Pnet; and counting the detection quantity and the inflection point positions, and printing a final result in a display window. According to the invention, a target detection method based on deep learning is utilized to intelligently count the semi-finished reinforcing steel bar products, the defect of manual counting of the semi-finished reinforcing steel bar products is overcome, the system remarkably improves the point inspection speed of the semi-finished reinforcing steel bar products through an automatic point counting technology, and compared with a traditional manual counting mode, the working time is greatly shortened.

Description

Technical Field

[0001] The invention relates to the field of construction site image processing, and in particular to an embedded system for checking semi-finished steel bars based on point counting. Background Art

[0002] In the construction industry, semi-finished steel bars are an indispensable and important material in the construction process. Their quality and quantity are directly related to the safety, stability and construction efficiency of the construction project. However, in the traditional construction process, the counting and inspection of semi-finished steel bars have long relied on manual operations. Manual counting is not only inefficient and difficult to adapt to the fast-paced construction environment, but also difficult to avoid human errors, resulting in confusion and inaccuracy in material management, which further affects the control of project costs and the guarantee of quality.

[0003] With the development of smart construction sites and technological advancement, manual counting can no longer meet the requirements of efficient, accurate and intelligent construction. To address this problem, an embedded system for semi-finished steel bar inspection based on point counting has been developed. Through embedded technology and point counting algorithms, the problems of low efficiency and high error rate caused by manual counting are fundamentally solved, thereby improving construction efficiency and reducing labor costs. At the same time, it ensures that the material management of construction projects is more accurate and safe, providing solid technical support for the quality and safety of construction projects. Summary of the invention

[0004] 1. Technical issues to be solved

[0005] In view of the deficiencies in the prior art, the present invention provides an embedded system for checking semi-finished steel bars based on point counting. Through the camera and image processing algorithm carried by the embedded system, rapid identification and counting of semi-finished steel bars can be achieved, thereby improving counting efficiency.

[0006] (II) Technical solution

[0007] To achieve the above purpose, the present invention is implemented by the following technical scheme: an embedded system for checking semi-finished steel bars based on point counting, comprising the following steps:

[0008] S1: Use Orange Pi, a domestic embedded development board equipped with an external visible light camera, to capture the diagonal images of the semi-finished steel bars at the construction site;

[0009] S2: After pre-processing, the collected images are sent to the NPU processor of Orange Pi, a domestic embedded development board, for processing;

[0010] S3: Import the trained weights into the development board and use the detection and counting function in the system to detect and count the semi-finished steel bar images. The network used is based on the improved target detection network P2Pnet.

[0011] S4: Count the number of tests and the position of the inflection points, and print out the final results in the display window.

[0012] The target detection network training used for the counting function in step S3 includes the following steps:

[0013] The first step is to collect training images: use a camera on the construction site to shoot the diagonal slopes of the semi-finished steel bars to obtain a data set, manually annotate the acquired images, and obtain the training labels corresponding to the images. The annotated objects are the inflection points of the semi-finished steel bars, and the annotation type is point annotation.

[0014] The second step is optimal network training: the obtained dataset of at least 500 semi-finished steel bars is divided into training set, validation set and test set; the training set is augmented before training, and all image shapes are filled with squares, and then scaled to 640×640; the object detection network is trained with the augmented and preprocessed training set, and the network is verified and tested using the divided validation set and test set to evaluate the network performance.

[0015] Among them, the specific steps of network structure module training in the second step are as follows:

[0016] (1) Construct a backbone network for extracting features of semi-finished steel bars: The backbone network uses the VGG19 network. The structure of the VGG19 network is a classic convolutional neural network, consisting of multiple convolutional layers, pooling layers, and fully connected layers. VGG19 focuses on achieving efficient feature extraction through smaller convolution kernels (3×3), while obtaining richer feature expressions through increasing the number of layers. The encoder part consists of 19 learnable convolutional layers and several pooling layers, which gradually downsample and extract features from feature maps. In the first two stages, each stage contains two convolutional layers and one pooling layer. In the third, fourth, and fifth stages, each stage contains four convolutional layers and one pooling layer, which extract deeper features and retain the feature information of the three stages.

[0017] (2) Construct a feature pyramid Transformer module that fuses feature information: including Rendering Transformer (RT), Grounding Transformer (GT) and Self-Transformer (ST), which enhances image representation capabilities through the interaction of multiple layers of features: RT achieves adaptive enhancement from low-level details to high-level semantics, and its input is high-level features and low-level features Combine x through the attention mechanism high_mask =BN(Conv3x3(x high)) and x low_gp =ReLU(BN(Conv1x1(AvgPool(x low ))) and generates a fused output; GT reversely guides low-level detail features from high-level abstract semantics, and the input is low-level x low and high-rise x high , calculate the attention f(x) through various methods such as dot product or Gaussian low , x high )=Softmax(θ(x low )·φ(x high ) T ), and get the enhanced output z = BN (Conv1x1 (y)) + x low ; ST uses multi-head self-attention to model the global interaction within the same feature layer, and the input is After projection q, k, v = Conv(x), 1 calculates attention The final output is combined with the residual connection to obtain out = Norm (Conv (output)) + x. Overall, FPT constructs rich spatial and semantic feature representations through cross-layer and intra-layer feature interactions.

[0018] (3) Construct a regression-classification branch attention (R-CBA) module: The two branches of R-CBA optimize the positioning and classification tasks respectively, so that the model can simultaneously achieve high-precision point annotation positioning and accurate target classification, especially in dense target scenes. This module contains a regression branch and a classification branch, and the two share a feature extraction network to ensure feature consistency and full utilization. In the regression branch, the input is the shared feature x, and the feature extraction and enhancement are performed through multi-layer convolution and embedded CBAM module (combining channel attention mechanism and spatial attention mechanism), and the final output is Where B is the batch size, N is the number of predicted points, and the output of each point contains its two-dimensional coordinates in the image, which is used to accurately locate the point annotation position of the target. In the classification branch, the shared feature x is also processed, and the key features are enhanced by CBAM and then output Where C is the number of categories, and the classification score marked on each point represents the probability distribution of the category to which it belongs.

[0019] As an optimization, in step S1, ensure that the interface between the Orange Pi development board and the external visible light camera matches, such as USB, MIPI, etc., and prepare the corresponding connecting cables. Fix the camera when shooting the diagonal inclined surface image of the semi-finished steel bar at the construction site to ensure that its shooting angle and position can accurately capture the image of the diagonal inclined surface of the semi-finished steel bar.

[0020] As an optimization, all image shapes are filled with squares during preprocessing in step S2, and their sizes are proportionally scaled to 640×640 to achieve the best detection effect.

[0021] As an optimization, in the second step of step S3, the data is divided into a training set, a validation set, and a test set in a ratio of 7:2:1. The method of data augmentation for the training set includes horizontal mirroring, brightness adjustment of the image, and salt and pepper noise of random intensity, where the brightness adjustment ratio is 0.5 to 1.5, and the noise adjustment is set to a 5% probability that each pixel is replaced with salt and pepper noise.

[0022] As an optimization step S4, the detection results output by the NPU processor of the embedded development board are counted, including summing the number of detected prediction points to obtain the number of semi-finished steel bars, drawing the inflection point position information on the result graph and displaying it on the touch screen of the Orange Pi development board.

[0023] (III) Beneficial effects

[0024] The present invention provides an embedded system for checking semi-finished steel bars based on point counting, which has the following features:

[0025] Beneficial effects:

[0026] The present invention uses a target detection method based on deep learning to intelligently count semi-finished steel bars, solving the drawbacks of manual counting of semi-finished steel bars. The system significantly improves the speed of checking semi-finished steel bars through automated point counting technology, greatly shortening the working time compared to the traditional manual counting method. The semi-finished steel bar inspection embedded system based on point counting has brought improvements and enhancements to the management of semi-finished steel bars on construction sites through automated and intelligent technical means. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a flow chart of the steel bar semi-finished product detection method of the present invention;

[0028] Figure 2 It is a framework diagram of network optimization and steel bar semi-finished product detection of the present invention;

[0029] Figure 3 This is a framework diagram of the feature pyramid transformer module of the present invention;

[0030] Figure 4 This is a framework diagram of the regression-classification branch attention detection module of the present invention;

[0031] Figure 5 This is a diagram showing the effect of the present invention on detecting semi-finished steel bars. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] See also Figure 1-5 , the present invention provides a technical solution:

[0034] like Figure 2 As shown, model training and testing are completed in the following steps:

[0035] The first step is to collect training images: use a camera on the construction site to shoot the diagonal slopes of the semi-finished steel bars to obtain a data set, manually annotate the acquired images, and obtain the training labels corresponding to the images. The annotated objects are the inflection points of the semi-finished steel bars, and the annotation type is point annotation.

[0036] The second step is optimal network training: divide the obtained dataset of at least 500 semi-finished steel bars into training set, validation set and test set; augment the training set before training, fill all image shapes with squares, and then scale them to 640×640; train the target detection network with the augmented and preprocessed training set, and use the divided validation set and test set to verify and test the network to evaluate the network performance. The specific steps of network training are as follows:

[0037] (1) Construct a backbone network for extracting features of semi-finished steel bars: The backbone network uses the VGG19 network. The structure of the VGG19 network is a classic convolutional neural network, consisting of multiple convolutional layers, pooling layers, and fully connected layers. VGG19 focuses on achieving efficient feature extraction through smaller convolution kernels (3×3), while obtaining richer feature expressions through increasing the number of layers. The encoder part consists of 19 learnable convolutional layers and several pooling layers, which gradually downsample and extract features from feature maps. In the first two stages, each stage contains two convolutional layers and one pooling layer. In the third, fourth, and fifth stages, each stage contains four convolutional layers and one pooling layer, which extract deeper features and retain the feature information of the three stages.

[0038] (2) Construct a feature pyramid Transformer module that fuses feature information: including Rendering Transformer (RT), Grounding Transformer (GT) and Self-Transformer (ST), which enhances image representation capabilities through the interaction of multiple layers of features: RT achieves adaptive enhancement from low-level details to high-level semantics, and its input is high-level features and low-level features Combine x through the attention mechanism high_mask =BN(Conv3x3(x high )) and x low_gp =ReLU(BN(Conv1xl(AvgPool(x low ))) and generates a fused output; GT reversely guides low-level detail features from high-level abstract semantics, and the input is low-level x low and high-rise x high , calculate the attention f(x) through various methods such as dot product or Gaussian low , x high )=Softmax(θ(x low )·φ(x high ) T ), and get the enhanced output z = BN (Conv1x1 (y)) + x low ; ST uses multi-head self-attention to model the global interaction within the same feature layer, and the input is After projection q, k, v = Conv(x), calculate the attention The final output is combined with the residual connection to obtain out = Norm (Conv (output)) + x. Overall, FPT constructs rich spatial and semantic feature representations through cross-layer and intra-layer feature interactions.

[0039] (3) Construct a regression-classification branch attention (R-CBA) module: The two branches of R-CBA optimize the positioning and classification tasks respectively, so that the model can simultaneously achieve high-precision point annotation positioning and accurate target classification, especially in dense target scenes. This module contains a regression branch and a classification branch, and the two share a feature extraction network to ensure feature consistency and full utilization. In the regression branch, the input is the shared feature x, and the feature extraction and enhancement are performed through multi-layer convolution and embedded CBAM module (combining channel attention mechanism and spatial attention mechanism), and the final output is Where B is the batch size, N is the number of predicted points, and the output of each point contains its two-dimensional coordinates in the image, which is used to accurately locate the point annotation position of the target. In the classification branch, the shared feature x is also processed, and the key features are enhanced by CBAM and then output Where C is the number of categories, and the classification score marked on each point represents the probability distribution of the category to which it belongs.

[0040] The third step is the detection function of semi-finished steel bars: the inclined surface pictures of semi-finished steel bars taken on site are input into the trained point annotation detection network, the inclined surface pictures are processed by the network, and the feature information is extracted. The predicted turning point positions and quantities of semi-finished steel bars are obtained by positioning and classification branches, and the results are output on the touch screen of the Orange Pi development board.

[0041] Based on the above steps, Figure 1 As shown, an embedded system for checking semi-finished steel bars based on point counting is shown, and the specific operation steps are as follows:

[0042] S1: Use Orange Pi, a domestic embedded development board equipped with an external visible light camera, to capture the diagonal images of the semi-finished steel bars at the construction site;

[0043] S2: After pre-processing, the collected images are sent to the NPU processor of Orange Pi, a domestic embedded development board, for processing;

[0044] S3: Import the trained weights into the development board and use the detection and counting function in the system to detect and count the semi-finished steel bar images. The network used is based on the improved target detection network P2Pnet.

[0045] S4: Count the number of tests and the position of the inflection points, and print out the final results in the display window.

[0046] In step S1, ensure that the interface between the Orange Pi development board and the external visible light camera matches, such as USB, MIPI, etc., and prepare the corresponding connecting cables. Fix the camera when shooting the diagonal inclined surface image of the semi-finished steel bar at the construction site to ensure that its shooting angle and position can accurately capture the image of the diagonal inclined surface of the semi-finished steel bar.

[0047] During the preprocessing in step S2, all image shapes are filled with squares and then proportionally scaled to 640×640 to achieve the best detection effect.

[0048] In the second step of step S3, the data is divided into a training set, a validation set, and a test set in a ratio of 7:2:1. The method of data augmentation for the training set includes horizontal mirroring, brightness adjustment of the image, and salt and pepper noise of random intensity, wherein the brightness adjustment ratio is 0.5 to 1.5, and the noise adjustment is set so that each pixel has a 5% probability of being replaced with salt and pepper noise.

[0049] In step S4, the detection results output by the NPU processor of the embedded development board are counted, including summing the number of detected prediction points to obtain the number of semi-finished steel bars, and drawing the inflection point position information on the result graph and displaying it on the touch screen of the OrangePi development board.

[0050] In the optimization process of the entire network of the present invention, the network gradually learns the multi-scale features and global context information of different semi-finished steel bars. When the model finds the best network model through continuous optimization, the entire network can be used as a tool for checking semi-finished steel bars. At this point, the model has the ability to check and count semi-finished steel bars on the construction site.

[0051] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An embedded system for checking semi-finished steel bars based on point counting, characterized in that: The following steps are involved: S1: Use Orange Pi, an embedded development board equipped with an external visible light camera, to capture images of the diagonal slope of semi-finished steel bars at the construction site; S2: After pre-processing, the collected image is sent to the NPU processor of the embedded development board Orange Pi for processing; S3: Use the detection and counting function in the system to detect and count the semi-finished steel bar images. The model used by the detection and counting function is based on the improved target detection network P2Pnet; S4: Count the number of tests and the position of the inflection points, and print out the final results in the display window; The training of the target detection network P2Pnet used for the detection counting function model in step S3 includes the following steps: The first step is to collect training images: use a camera on the construction site to shoot the diagonal slopes of the semi-finished steel bars to obtain a data set, manually annotate the acquired images, and obtain the training labels corresponding to the images. The annotated objects are the inflection points of the semi-finished steel bars, and the annotation type is point annotation. The second step is optimal network training: the obtained dataset of at least 500 semi-finished steel bars is divided into training set, validation set and test set; before training, the training set is augmented and all image shapes are filled with squares, and then the size is scaled to 640×640; the augmented and preprocessed training set is used to train the object detection network, and the divided validation set and test set are used to verify and test the network to evaluate the network performance; Among them, the specific steps of target detection network training in the second step are as follows: (1) Construct a backbone network for extracting features of semi-finished steel bars: The backbone network uses the VGG19 network. The encoder part consists of 19 learnable convolutional layers and several pooling layers, which gradually downsample and extract features from feature maps. In the first two stages, each stage contains two convolutional layers and one pooling layer. In the third, fourth, and fifth stages, each stage contains four convolutional layers and one pooling layer, which extract deeper features and retain the feature information of the three stages. (2) Construct a feature pyramid Transformer module that fuses feature information: It includes RT, GT, and ST. It enhances the image representation capability through the interaction of multiple layers of features: RT realizes adaptive enhancement from low-level details to high-level semantics, and its input is high-level features. and low-level features Combine x through the attention mechanism high_mask =BN(Conv3x3(x high )) and x low_gp =ReLU(BN(Conv1x1(AvgPool(x low ))) and generates a fused output; GT reversely guides low-level detail features from high-level abstract semantics, and the input is low-level x low and high-rise x high , calculate the attention f(x) by dot product or Gaussian method low , x high )=Softmax(θ(x low )·φ(x high ) T ), and get the enhanced output z = BN (Conv1x1 (y)) + x low ; ST uses multi-head self-attention to model the global interaction within the same feature layer, and the input is After projection q, k, v = Conv(x), calculate the attention The final output is combined with the residual connection to obtain out = Norm (Conv (output)) + x. Overall, FPT constructs rich spatial and semantic feature representations through cross-layer and intra-layer feature interactions; (3) Construct a regression-classification branch attention module: The module contains a regression branch and a classification branch. The two branches share a feature extraction network to ensure the consistency and full utilization of features. In the regression branch, the input is the shared feature x. Through multi-layer convolution and embedded CBAM modules, the channel attention mechanism and the spatial attention mechanism are combined to extract and enhance features, and the final output is Where B is the batch size, N is the number of predicted points, and the output of each point contains its two-dimensional coordinates in the image, which is used to accurately locate the point annotation position of the target; in the classification branch, the shared feature x is also processed, and the key features are enhanced by CBAM and then output Where C is the number of categories, and the classification score marked on each point represents the probability distribution of the category to which it belongs.

2. According to claim 1, a semi-finished steel bar inspection embedded system based on point counting is characterized in that: In step S1, ensure that the interface between the Orange Pi development board and the external visible light camera matches, and prepare the corresponding connecting wires. Fix the camera when shooting the diagonal inclined surface of the semi-finished steel bars on the construction site to ensure that its shooting angle and position can accurately capture the image of the diagonal inclined surface of the semi-finished steel bars.

3. According to claim 1, a semi-finished steel bar inspection embedded system based on point counting is characterized in that: During the preprocessing in step S2, all image shapes are filled into squares and then proportionally scaled to 640×640 to achieve the best detection effect.

4. According to claim 1, a semi-finished steel bar inspection embedded system based on point counting is characterized in that: In the second step, step S3 divides the data into a training set, a validation set, and a test set in a ratio of 7:2:

1. The method of data augmentation for the training set includes horizontal mirroring, brightness adjustment of the image, and salt and pepper noise of random intensity, where the brightness adjustment ratio is 0.5 to 1.5, and the noise adjustment is set to a 5% probability that each pixel is replaced with salt and pepper noise.

5. The embedded system for checking semi-finished steel bars based on point counting according to claim 1 is characterized in that: Step S4 counts the detection results output by the NPU processor of the embedded development board, including summing the number of detected prediction points to obtain the number of semi-finished steel bars, plotting the inflection point position information on the result graph and displaying it on the touch screen of the Orange Pi development board.

Images