An image recognition method and system for a smart construction site edge device

Abstract

This invention discloses an image recognition method for edge devices in smart construction sites, comprising the following steps: acquiring images to be recognized at the smart construction site; performing model pruning and lightweighting on a pre-trained image recognition neural network model; and using the pruned and lightweighted image recognition neural network model to perform image recognition on the images to be recognized. This invention provides a neural network model lightweighting algorithm based on adversarial learning polarization pruning, achieving efficient lightweighting of the neural network model while maintaining its recognition accuracy. The lightweighted model is then deployed on edge sensing devices in smart construction sites, meeting the low-power, high-real-time, and high-precision image recognition requirements of these devices.

Description

Technical Field

[0001] This invention relates to artificial intelligence technology, and more particularly to an image recognition method and system for edge devices in smart construction sites. Background Technology

[0002] Smart construction sites are a core scenario for the digital transformation of the construction industry and have entered the stage of large-scale application. They have achieved comprehensive monitoring and refined management of construction sites through information technology. Among them, image recognition technology is the core technology for on-site safety management and risk warning of smart construction sites, which can monitor workers' safety equipment wearing, behavior norms and equipment operation status in real time.

[0003] Because image recognition at smart construction sites requires processing a large amount of on-site image data, directly transmitting all image data to the cloud for processing would consume significant network bandwidth and cause processing delays, failing to meet the requirements of real-time monitoring and rapid response. Therefore, edge sensing technology is applied to smart construction sites. By performing preliminary processing and analysis of image data on edge devices, abnormal situations at the construction site can be quickly identified, improving response speed and processing efficiency.

[0004] However, edge devices in smart construction sites suffer from limited computing resources and storage capacity. Traditional complex image recognition neural network models cannot run efficiently on these devices, necessitating lightweight model processing. Current mainstream model lightweighting methods include model pruning, quantization, knowledge distillation, and low-rank decomposition. Among these, model pruning, by removing redundant parameters to obtain a sparse model, offers advantages such as wide applicability and good scalability. It can also be combined with other compression methods to achieve further lightweighting, making it the preferred method for edge device model optimization.

[0005] However, existing model pruning methods have obvious drawbacks. On the one hand, they lack the accuracy to identify redundant filters and cannot accurately distinguish which channels in the neural network need to be retained and which need to be pruned. On the other hand, the accuracy of the model is greatly reduced after pruning, making it difficult to balance the lightweight nature of the model with the recognition accuracy, and thus failing to meet the high requirements of smart construction site edge devices for image recognition.

[0006] In view of the shortcomings of the existing technologies, there is an urgent need for a new lightweight method that can minimize the loss of model recognition accuracy and adapt to the hardware characteristics and practical application needs of smart construction site edge devices. Summary of the Invention

[0007] The technical problem to be solved by the present invention is to provide an image recognition method and system for edge devices in smart construction sites, addressing the deficiencies in the existing technology.

[0008] The technical solution adopted by this invention to solve its technical problem is: an image recognition method for edge devices in smart construction sites, comprising the following steps: 1) Collect and acquire images of the smart construction site to be identified; 2) Perform model pruning and lightweighting on the pre-trained image recognition neural network model; 3) Use the image recognition neural network model after model pruning and lightweighting to perform image recognition on the image to be recognized.

[0009] According to the above scheme, step 2) specifically includes: 2.1) Load the parameters of the pre-trained image recognition neural network model, initialize the pre-trained image recognition neural network model, set the sparsity factor λ and polarization parameter μ, and obtain the network to be pruned; The network to be pruned adds an additional soft mask layer to the original model. This soft mask layer is used to measure the importance of each channel in the neural network. During training, polarization regularization is used to make the soft mask m tend towards polarization, and the soft mask value m... i Channel i that approaches 0 is determined to be a redundant channel; Wherein, the polarization parameter μ is a fixed value, the soft mask m is polarized under the action of polarization regularization, the corresponding neurons are gradually deactivated during the training process, and only the channels corresponding to the deactivated neurons are pruned to achieve accurate recognition of redundant channels; 2.2) Adopt an adversarial learning performance recovery strategy to update the parameters of the network to be pruned, thereby reducing the performance loss of pruning; 2.3) During training, regularization is applied to the weights of the pruned network; 2.4) Based on the updated soft mask value of the network to be pruned, the importance of each channel is evaluated. Channels with higher soft mask values ​​are retained, and redundant channels with lower soft mask values ​​are pruned to complete the lightweighting of the model.

[0010] According to the above scheme, the sparsity factor λ is dynamically adjusted based on the computing resources, storage capacity, and image recognition accuracy requirements of the edge devices in the smart construction site. If the device's computing resources and storage capacity are scarce, increase the sparsity factor λ; if the device's computing resources and storage capacity are sufficient, decrease the sparsity factor λ. If high accuracy is required for image recognition, the sparsity factor λ should be reduced accordingly.

[0011] According to the above scheme, step 2.2) is specifically as follows: First, fix the pre-trained network and update the discriminator. Input the training set data into the pre-trained network and the pruning network respectively to obtain the output results. Input the output results into the discriminator and update the discriminator parameters by minimizing the loss function through the stochastic gradient algorithm. Then, fix the discriminator and update the parameters of the network to be pruned. The training set data consists of image data collected on-site at smart construction sites, including image data related to workers wearing safety equipment, the operation of equipment on construction sites, and intrusion into dangerous areas.

[0012] According to the above scheme, step 2.3) is specifically as follows: Add a regularization term to the discriminator's loss function to constrain the magnitude of its parameter updates; the adversarial regularization term uses L2 regularization of the discriminator parameters. By updating the discriminator with pruned network features, the adversarial training time can be extended, preventing the discriminator from dominating the training process and achieving more effective prediction updates.

[0013] According to the above scheme, in step 2.3), the discriminator is updated by pruning the network features, as follows: The same batch of smart construction site image data is input into a pre-trained model and a pruning network respectively. Two types of core features are obtained through forward propagation. The output features of the pre-trained model are used as the baseline features, representing the high-precision features that the pruning network needs to approximate. The output features of the pruning network are used as the features to be optimized, which are the current feature state of the model after pruning. Calculate the difference between the two types of features: obtain the feature difference ΔF between the two core features by calculating the mean squared error of each dimension; The baseline features, features to be optimized, and feature differences are respectively input into the discriminator network; the discriminator performs forward inference on each type of feature and outputs the corresponding "source judgment probability": Output probability P1 for the baseline feature, output probability P2 for the feature to be optimized, and output probability P3 for ΔF. Construct a loss function with pruning features; construct a cross-entropy loss based on P1 and P2, incorporate the loss term corresponding to P3, and add the L2 regularization term of the discriminator parameters to the total loss; The discriminator parameters are updated using a loss function. Calculate the discriminant's ability to identify pruned features after the update. If the discriminant's ability to identify features does not improve significantly, it means that the update has effectively extended the training period, and the next iteration can begin.

[0014] Iterative updates and termination criteria; When the recognition accuracy of the pruned model is greater than 99% of that of the pre-trained model, and the computation time of the edge device meets the requirements, the discriminator is stopped from being updated.

[0015] Based on the above method, the present invention also provides an image recognition system for edge devices in smart construction sites, including an edge acquisition module, a lightweight model inference module, and an image recognition module; The edge acquisition module is used to acquire image data to be identified at the construction site of the smart construction site; The lightweight model inference module is used to obtain a lightweight optimized image recognition neural network model. The image recognition module is used to perform real-time recognition processing on the acquired image data.

[0016] The beneficial effects of this invention are: 1. This invention provides a lightweight neural network model algorithm based on adversarial learning polarization pruning, which solves the problems of inaccurate identification of redundant filters and large loss of model accuracy after pruning in existing model pruning methods, and achieves efficient lightweighting of neural network models while ensuring the model's recognition accuracy.

[0017] 2. This invention deploys the lightweight model on the edge sensing device of the smart construction site, which meets the low power consumption, high real-time and high precision image recognition requirements of the edge device of the smart construction site, realizes the preliminary processing and analysis of the image data at the edge of the construction site, and quickly identifies safety anomalies such as not wearing a safety helmet and intrusion into dangerous areas. Attached Figure Description

[0018] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings: Figure 1 This is a flowchart of a method according to an embodiment of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0020] like Figure 1 As shown, an image recognition method for edge devices in smart construction sites includes the following steps: 1) Acquire the image to be recognized; 2) Perform model pruning and lightweighting on the pre-trained image recognition neural network model; 2.1) Load the parameters of the pre-trained image recognition neural network model, initialize the pre-trained image recognition neural network model, set the sparsity factor λ and polarization parameter μ, and obtain the network to be pruned; The network to be pruned adds an additional soft mask layer to the original model. This soft mask layer is used to measure the importance of each channel in the neural network. During training, polarization regularization is used to make the soft mask m tend towards polarization, and the soft mask value m... i Channels approaching 0 are considered redundant channels; Among them, the polarization parameter μ is a fixed value, and the sparsity factor λ is dynamically adjusted according to the computing resources, storage capacity and image recognition accuracy requirements of the edge devices of the smart construction site; the soft mask m is polarized under the action of polarization regularization, and the corresponding neurons are gradually deactivated during the training process. Only the channels corresponding to the deactivated neurons are pruned to achieve accurate recognition of redundant channels.

[0021] 2.2) Adhesive learning performance recovery strategies are used to update the parameters of the network to be pruned, reducing the performance loss during pruning. First, fix the pre-trained network and update the discriminator. Input the training set data into the pre-trained network and the pruning network respectively to obtain the output results. Input the output results into the discriminator and update the discriminator parameters by minimizing the loss function through the stochastic gradient algorithm. Then fix the discriminator and update the parameters of the network to be pruned. The training set data consists of image data collected on-site at smart construction sites, including image data related to workers wearing safety equipment, the operation of equipment on construction sites, and intrusion into dangerous areas, making the model training more closely aligned with the actual application scenarios of smart construction sites. 2.3) During training, L2 regularization is applied to the weights of the pruned network to prevent model overfitting, and adversarial regularization constraints are applied to the discriminator. The discriminator is updated by updating the features of the pruned network to extend the adversarial training time, avoid the discriminator dominating the training process, and achieve more effective prediction updates.

[0022] In step 2.3), the discriminator is updated using pruned network features, as detailed below: The same batch of smart construction site image data is input into a pre-trained model and a pruning network respectively. Two types of core features are obtained through forward propagation. The output features of the pre-trained model are used as the baseline features, representing the high-precision features that the pruning network needs to approximate. The output features of the pruning network are used as the features to be optimized, which are the current feature state of the model after pruning. Calculate the difference between the two types of features: obtain the feature difference ΔF between the two core features by calculating the mean squared error of each dimension; The baseline features, features to be optimized, and feature differences are respectively input into the discriminator network; the discriminator performs forward inference on each type of feature and outputs the corresponding "source judgment probability": Output probability P1 for the baseline feature, output probability P2 for the feature to be optimized, and output probability P3 for ΔF. Construct a loss function with pruning features; construct a cross-entropy loss based on P1 and P2, incorporate the loss term corresponding to P3, and add the L2 regularization term of the discriminator parameters to the total loss; The discriminator parameters are updated using a loss function. Calculate the discriminant's ability to identify pruned features after the update. If the discriminant's ability to identify features does not improve significantly (e.g., P2 is still close to 0.5), it means that the update has effectively extended the training period, and the next iteration can begin.

[0023] Iterative updates and termination criteria; When the recognition accuracy of the pruned model is greater than 99% of that of the pre-trained model, and the computation time of the edge device meets the requirements, the discriminator is stopped from being updated.

[0024] 2.4) Based on the updated soft mask value of the network to be pruned, the importance of each channel is evaluated, the channels with higher soft mask values ​​are retained, and redundant channels with lower soft mask values ​​are pruned to complete the lightweighting of the neural network model. 3) Use the image recognition neural network model after model pruning and lightweighting to perform image recognition on the image to be recognized.

[0025] Based on the above method, the present invention also provides an image recognition system for edge devices in smart construction sites, including an edge acquisition module, a lightweight model inference module, and an image recognition module; The edge acquisition module is used to acquire image data to be identified at the construction site of the smart construction site; The lightweight model inference module is used to obtain a lightweight optimized image recognition neural network model. The image recognition module is used to perform real-time recognition processing on the acquired image data.

[0026] Effect verification.

[0027] To effectively evaluate the performance of the pruning algorithm in this invention, the VGG16 neural network, widely used in image recognition, was pruned. The polarization parameters of the experiment... Set to a fixed value. Sparse factor. As a dynamically adjustable pruning parameter, it is adjusted according to the specific experiment.

[0028] Pruning experiments using VGG16 were conducted on the CIFAR-10 dataset, and compared with algorithms that have achieved good results in model pruning, such as NS, GAL, Hrank, ARPruning, and ASCA. The results are shown in Table 1. In the performance evaluation experiments of the pruning algorithms, recognition accuracy, FLOPs compression rate, and parameter compression rate were used as metrics for pruning performance. The results of pruning VGG16 are shown in Table 1. The proposed method achieved a FLOPs compression rate of 70.4% and a parameter compression rate of 78.3% with an accuracy decrease of 0.29%, demonstrating its effectiveness.

[0029] Table 1 Results of VGG16 pruning on the CIFAR-10 dataset

[0030] It should be understood that those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. An image recognition method for edge devices in smart construction sites, characterized in that, Includes the following steps: 1) Collect and acquire images of the smart construction site to be identified; 2) Perform model pruning and lightweighting on the pre-trained image recognition neural network model; 3) Use the image recognition neural network model after model pruning and lightweighting to perform image recognition on the image to be recognized.

2. The image recognition method for edge devices in smart construction sites according to claim 1, characterized in that, Step 2) specifically includes: 2.1) Load the parameters of the pre-trained image recognition neural network model, initialize the pre-trained image recognition neural network model, set the sparsity factor λ and polarization parameter μ, and obtain the network to be pruned; The network to be pruned adds an additional soft mask layer to the original model. This soft mask layer is used to measure the importance of each channel in the neural network. During training, polarization regularization is used to make the soft mask m tend towards polarization, and the soft mask value m... i Channels approaching 0 are considered redundant channels; 2.2) Adopt an adversarial learning performance recovery strategy to update the parameters of the network to be pruned, thereby reducing the performance loss of pruning; 2.3) During training, regularization is applied to the weights of the pruned network; 2.4) Based on the updated soft mask value of the network to be pruned, the importance of each channel is evaluated. Channels with higher soft mask values ​​are retained, and redundant channels with lower soft mask values ​​are pruned to complete the lightweighting of the model.

3. The image recognition method for edge devices in smart construction sites according to claim 2, characterized in that, The sparsity factor λ is dynamically adjusted based on the computing resources, storage capacity, and image recognition accuracy requirements of edge devices in smart construction sites. If the device's computing resources and storage capacity are scarce, increase the sparsity factor λ; if the device's computing resources and storage capacity are sufficient, decrease the sparsity factor λ. If high accuracy is required for image recognition, the sparsity factor λ should be reduced accordingly.

4. The image recognition method for edge devices in smart construction sites according to claim 3, characterized in that, In step 2.2), the specific steps are as follows: First, fix the pre-trained network and update the discriminator. Input the training set data into the pre-trained network and the pruning network respectively to obtain the output results. Input the output results into the discriminator and update the discriminator parameters by minimizing the loss function through the stochastic gradient algorithm. Then, fix the discriminator and update the parameters of the network to be pruned. The training set data consists of image data collected on-site at smart construction sites, including image data related to workers wearing safety equipment, the operation of equipment on construction sites, and intrusion into dangerous areas.

5. The image recognition method for edge devices in smart construction sites according to claim 3, characterized in that, In step 2.3), the specific details are as follows: Add a regularization term to the discriminator's loss function to constrain the magnitude of its parameter updates; the adversarial regularization term uses L2 regularization of the discriminator parameters. By updating the discriminator with pruned network features, the adversarial training time can be extended, preventing the discriminator from dominating the training process and achieving more effective prediction updates.

6. The image recognition method for edge devices in smart construction sites according to claim 3, characterized in that, In step 2.3), the discriminator is updated using pruned network features, as follows: The same batch of smart construction site image data is input into a pre-trained model and a pruning network respectively. Two types of core features are obtained through forward propagation. The output features of the pre-trained model are used as the baseline features, representing the high-precision features that the pruning network needs to approximate. The output features of the pruning network are used as the features to be optimized, which are the current feature state of the model after pruning. Calculate the difference between the two types of features: obtain the feature difference ΔF between the two core features by calculating the mean squared error of each dimension; The baseline features, features to be optimized, and feature differences are respectively input into the discriminator network; the discriminator performs forward inference on each type of feature and outputs the corresponding source judgment probability: Output probability P1 for the baseline feature, output probability P2 for the feature to be optimized, and output probability P3 for ΔF. Construct a loss function with pruning features; construct a cross-entropy loss based on P1 and P2, incorporate the loss term corresponding to P3, and add the L2 regularization term of the discriminator parameters to the total loss; The discriminator parameters are updated using a loss function. Calculate the discriminant's ability to identify pruned features after the update. If the discriminant's ability to identify features does not improve significantly, it means that the update has effectively extended the training period, and the next iteration can begin. Iterative updates and termination criteria; When the recognition accuracy of the pruned model is greater than 99% of that of the pre-trained model, and the computation time of the edge device meets the requirements, the discriminator is stopped from being updated.

7. An image recognition system for edge devices in smart construction sites, characterized in that, Includes an edge acquisition module, a lightweight model inference module, and an image recognition module; The edge acquisition module is used to acquire image data to be identified at the construction site of the smart construction site; The lightweight model inference module is used to obtain a lightweight optimized image recognition neural network model. The image recognition module is used to perform real-time recognition processing on the acquired image data.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 6.

Images