Mobile terminal human body tumble detection method for photovoltaic power station construction based on YOLOv8

Abstract

The invention discloses a YOLOv8-based photovoltaic power station construction-oriented mobile terminal human body tumble detection method. The method comprises a data acquisition and preprocessing module, an algorithm optimization module, a model quantification module and a model mobile terminal deployment module. A human body tumble data set is acquired at a photovoltaic power station construction site and is processed through an image enhancement and Mosaic data set enhancement technology; on the basis of a YOLOv8 series lightweight model, a re-parameterized convolution method is adopted to replace traditional convolution, a lightweight cross-scale feature fusion module reconstructs a neck network, and a detection head is improved through re-parameterized convolution; quantizing the float32 model after the training is completed into an int8 format; a mobile terminal CPU and NPU deployment environment is adapted, and an average precision mean value, a recall rate, accuracy and the like are used as evaluation indexes. The method solves the problems of difficult deployment, high power consumption and insufficient precision and real-time performance of a traditional fall detection method for the mountain photovoltaic power station, improves the detection precision and reasoning speed, and is suitable for construction safety monitoring of the photovoltaic power station in a low-resource environment.

Description

Technical Field

[0001] This invention relates to the field of smart industrial engineering technology, specifically to a mobile-based human fall detection method based on YOLOv8 for photovoltaic power plant construction. Background Technology

[0002] Photovoltaic power plants, with their advantages of renewable resources and minimal environmental impact, have become an important part of the global energy supply system. my country has a vast territory, with abundant sunshine resources in its western mountainous and plateau regions, making them key areas for photovoltaic power plant construction. Guizhou, in particular, has seen a year-on-year increase in the scale of photovoltaic power plant construction due to its unique sunshine conditions and land resources. However, the construction scenario for mountainous photovoltaic power plants has significant unique characteristics: the construction areas are mostly karst landforms, with rugged terrain, steep slopes, and scattered work areas; at the same time, the subtropical monsoon climate causes large fluctuations in sunshine intensity, frequent fog during the rainy season, and damp ground, all of which significantly increase the risk of falls for construction workers.

[0003] Personnel safety is a core objective of project management during the construction of photovoltaic power plants. According to industry statistics, falls are the leading cause of injury and disability on construction sites. Falls in mountainous photovoltaic power plants not only cause personal injury to construction workers but also lead to work stoppages, project delays, and significant economic losses and social impacts. Therefore, achieving real-time and accurate detection of falls among construction workers is a crucial requirement for ensuring the safety of photovoltaic power plant construction and improving project efficiency.

[0004] Current fall detection research mainly includes three methods: first, using wearable sensors to collect motion data; second, monitoring vibration, infrared, or audio signals through environmental sensors; and third, using surveillance cameras to analyze video or images. However, these methods still face challenges in application at photovoltaic construction sites in mountainous areas like Guizhou. Guizhou's unique karst topography makes the deployment of traditional detection methods difficult; wearable devices may interfere with construction operations and power supply is unreliable; environmental sensors have coverage and accuracy issues in complex terrain; and it is difficult to deploy high-computing-power, high-power-consumption mobile devices in remote areas. This means that current fall detection still mainly relies on manual supervision by safety officers, which is insufficient to meet the real-time monitoring needs of large-scale construction sites.

[0005] Research on deep learning-based fall detection for mountainous photovoltaic power station scenarios is still in its early stages, and the lack of a dataset of fall detection data from construction sites in Guizhou's mountainous photovoltaic power stations significantly hinders the further development of related technologies. Therefore, developing a lightweight, low-power fall detection system tailored to the characteristics of Guizhou's mountainous regions can not only compensate for the shortcomings of existing technologies and provide a solid guarantee for personnel safety, but also significantly improve the construction efficiency of photovoltaic power stations, possessing significant engineering application value and far-reaching social significance. The application of lightweight deep learning models for mobile device detection is currently a research hotspot. Summary of the Invention

[0006] The present invention aims to provide a mobile fall detection method based on YOLOv8 for photovoltaic power plant construction, and to develop a lightweight, low-power, and high-precision mobile fall detection method adapted to mountain photovoltaic power plant scenarios.

[0007] To achieve the above objectives, this application provides the following technical solution: To achieve the above technical objectives, the present invention employs the following technical method: a mobile-based human fall detection method based on YOLOv8 for photovoltaic power plant construction, specifically including the following steps: S1: Data Acquisition and Processing. Data on worker falls was collected at a photovoltaic power station construction site in the mountains of Guizhou, and image and dataset augmentation methods were used. S2: Algorithm Targeting Improvements. First, based on the YOLOv8n model, the OREPA reparameterization method was used to replace ordinary convolutions, effectively improving the model's computational efficiency. Then, the lightweight cross-scale feature fusion module CCFM was used to reconstruct the neck network, strengthening the interaction of multi-scale information and achieving network lightweighting. Finally, RepConv reparameterized convolutions were used to improve the detection head, thereby simultaneously improving detection accuracy and inference speed. S3: Model Quantization. The model file generated after training used a data format similar to float32. To improve computational efficiency, the file was quantized and converted to int8 format. S4: Model Deployment. The weight parameters of each layer in the pth file obtained from workstation training were extracted and saved in bin format. The CPU will also call these parameters based on the layer during runtime and load them into RAM for computation. For NPU acceleration, the pth file needs to be converted to onnx format first, then quantized to int8, finally generating the rknn format model for NPU use. S5: Mean Average Precision (mAP), Recall, Precision, Parameter, GFLOPs, and Detection Frames Per Second (FPS) are used as evaluation metrics for model deployment performance.

[0008]

[0009] In the formula: TP is the number of positive classes predicted as positive; TN is the number of negative classes predicted as negative; FP is the number of negative classes predicted as positive; FN is the number of positive classes predicted as negative; r represents all possible values ​​of recall; SUM represents the total number of classes; FPS represents the number of images that can be processed in 1 second; Ptime represents the inference time for a single image.

[0010] Compared with the prior art, the present invention has the following beneficial effects: (1) This invention is based on the YOLOv8n model algorithm and innovatively uses the reparameterization method OREPA to replace ordinary convolution. This improvement can fully utilize the multi-branch structure to enhance feature learning ability during the model training stage, while transforming the multi-branch structure into a single branch through parameter fusion during the inference stage, effectively reducing the amount of computation and parameters. It significantly improves inference efficiency while ensuring that the model accuracy is basically not lost, making it more suitable for application scenarios with high requirements for computing speed. At the same time, the OREPA reparameterization method does not require major adjustments to the overall model architecture, has strong compatibility, and reduces the difficulty of model improvement and deployment.

[0011] (2) This invention addresses the problems of insufficient information transfer and high computational cost in multi-scale feature fusion of traditional neck networks. The proposed solution uses a lightweight cross-scale feature fusion module (CCFM) to reconstruct the neck network. The CCFM module, through the design of efficient feature interaction paths, can fully exploit the correlation information between feature maps of different scales, achieving deep fusion of multi-scale features and effectively enhancing the network's ability to detect targets of different sizes. Simultaneously, the module emphasizes lightweight design, simplifying redundant computational units and employing efficient convolution operations to significantly reduce the computational load and parameter count of the neck network while improving feature fusion performance. This further optimizes the overall computational efficiency and resource consumption of the model, providing strong support for deployment on resource-constrained devices.

[0012] (3) To further balance the model's detection accuracy and inference speed, this invention utilizes RepConv reparameterized convolution to improve the detection head. RepConv enhances the detection head's ability to capture and distinguish target features through a multi-branch structure during training, improving the model's detection accuracy for small and blurry targets. During inference, parameter reparameterization transforms the multi-branch structure into a concise single-branch structure, avoiding the additional computational overhead of multiple branches and significantly improving the detection head's inference speed. This improvement allows the detection head to maintain high feature extraction and target recognition accuracy while effectively reducing computational latency, ultimately achieving a dual improvement in model accuracy and inference speed, meeting the high requirements for overall model performance in practical applications. Attached Figure Description

[0013] Figure 1 This is a flowchart of the method for detecting falls by construction workers in a mobile photovoltaic power station according to the present invention; Figure 2 This is a model structure diagram of the improved YOLOv8 of the present invention; Figure 3 To improve the mAP50 accuracy variation curve of the model during training compared to other models; Figure 4 To improve the radar chart of various indicators of the model compared with other models; Figure 5 This is a physical diagram of the actual system of the present invention. Detailed Implementation

[0014] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings, but the scope of protection of the present invention is not limited to the following.

[0015] Traditional fall detection methods used in the construction of photovoltaic power stations in mountainous areas of Guizhou Province are complex to deploy and consume a lot of power, making it difficult to achieve real-time, low-power, and high-precision monitoring. To address this issue, this invention first constructs a dedicated dataset to collect fall data from construction sites, laying the foundation for model training. Second, based on YOLOv8n, the algorithm is optimized for mobile devices to improve hardware adaptability. Subsequently, the fp32 model is quantized to int8 format through linear transformation, significantly reducing model storage and memory usage without significantly affecting accuracy. Finally, the quantized model is deployed on both CPU and NPU environments to fully leverage the advantages of different hardware. These measures enable the method to achieve real-time and accurate fall detection even in low-resource and low-power environments, effectively overcoming the limitations of traditional methods in the Guizhou mountainous photovoltaic power station scenario. Based on the above, a mobile-based human fall detection method for photovoltaic power station construction is proposed. Figure 1 As shown, it includes the following steps: S1: Data collection and processing of fall data among photovoltaic power station construction workers. This involved collecting fall data from on-site workers at a photovoltaic power station construction site in the mountainous region of Guizhou. The data primarily collected data on three typical worker states: normal, in the process of falling, and already fallen. Finally, image augmentation and dataset augmentation methods were used. S1.1: Image preprocessing employs image denoising, overall normalization, and contrast enhancement to ensure that each image in the dataset has significant semantic information, thereby enhancing the learnable information of the images. S1.2: After the preprocessing of a single image is completed, the entire dataset is processed, that is, the Mosaic dataset augmentation method is adopted. First, the Mosaic canvas is initialized, then four images after image augmentation are randomly selected, and the image stitching point P=(xc, yc) is randomly selected. Finally, random stitching and scaling are performed to expand the original dataset of only 4029 images to 12087 images after dataset augmentation.

[0016] S2: The algorithm is improved by first replacing ordinary convolution with the YOLOv8n model algorithm using the reparameterization method OREPA to improve efficiency; then, the lightweight cross-scale feature fusion module CCFM is used to reconstruct the neck network, enhance multi-scale information interaction, and lighten the network; finally, the RepConv reparameterized convolution is used to improve the detection head, thereby achieving a dual improvement in accuracy and inference speed.

[0017] S2.1: Targeted improvements to convolution: Replacing traditional convolutions in the original model network with more advanced and cheaper partial convolutions, effectively reducing floating-point operations and hardware memory accesses. The formulas for calculating floating-point operations (FLOPs) and hardware memory accesses (Anum) are as follows:

[0018] In the formula: h represents the height of the input image; w represents the width of the input image; k represents the kernel size; cp represents the number of channels used in the partial convolution.

[0019] S2.2. Targeted improvements to the basic convolutional layer: The OREPA reparameterization method is used to replace ordinary convolution. Efficiency is improved by combining multi-branch feature learning in the training phase with parameter fusion in the inference phase. The equivalent convolution calculation formula in the inference phase is as follows:

[0020] Conv eq (x) represents the equivalent single-branch convolution output during the inference phase; x represents the input feature map; Conv kk (x) represents the k×k ordinary convolution of the main branch; Conv 11 (x) represents the 1×1 convolution of the branch; Id(x) represents the identity mapping branch; ⊕ represents the parameter fusion operation.

[0021] S2.3. Targeted improvements for the neck network: The neck network is reconstructed using a lightweight cross-scale feature fusion module (CCFM). Multi-scale information fusion is enhanced through depthwise separable convolutions and direct cross-scale interaction. The feature fusion calculation formula is as follows:

[0022] In the formula: F out This represents the fused feature map output by the CCFM module; F l Represents a large-scale feature map; F s Represents a small-scale feature map; DwConv () indicates a depthwise separable convolution operation; Up ( F s This indicates upsampling of the small-scale feature map; Down ( F l This indicates downsampling of the large-scale feature map; Concat () indicates the feature map splicing operation in the channel dimension.

[0023] S2.4. Targeted improvements to the detection head: The detection head structure is improved by using RepConv reparameterized convolution. During training, multi-branch convolution enhances feature learning ability, while during inference, it is fused into a single branch to improve speed. The calculation formula for the detection head output is as follows:

[0024] In the formula: Output This indicates the final output of the detection head; F 1, F 2, F 3 represents three feature maps of different scales output by the neck network; RepConv ( ) indicates a reparameterized convolution operation; ClassHead ( ) indicates a classification branch; RegHead ( ) indicates the regression branch; Cat ( ) indicates the splicing operation of multi-scale detection results in the channel dimension.

[0025] S3: For model quantization: The model uses int8 quantization, making the model more lightweight. The calculation formula is as follows:

[0026] In the formula: X This represents the raw fp32 data; Zero Point Indicates the mapping zero point; Scale Indicates the scaling factor; round Represents the floor function; b Indicates the number of quantization bits; X int This represents the quantized value.

[0027] S4: Model deployment involves extracting the weights of each layer from the pth file obtained during workstation training and converting it to bin format for storage. The CPU then loads these weights layer by layer into RAM for computation. Simultaneously, to accelerate NPU performance, the pth file is first converted to ONNX format and then quantized using int8 to generate a RKNN format model for NPU use. Deployment is divided into CPU and NPU phases, with the specific steps as follows: S4.1: The process for deploying quantization models on the CPU differs from conventional methods. First, the quantization model is transferred to the mobile device's DDR memory using an SD card or remote transmission. Then, a layer-by-layer extraction method is used to parse the convolution parameters, biases, and other key data for each layer from the model file. Finally, the extracted weight parameters and image data are loaded into RAM, where the CPU performs subsequent calculations. This method primarily meets the deployment requirements of most low-end mobile chips.

[0028] S4.2: When deploying a model on the NPU, the trained model needs to be quantized first, and then NPU computation is achieved through model conversion. The specific process is as follows: First, the trained model is converted to the general ONNX format, and the input / output dimensions and corresponding positions are determined by examining the network structure. Next, the ONNX model is converted to the proprietary RKNN format, using int8 quantization during this process. Finally, the converted model is stored in the mobile device's DDR memory via SD card or remote transmission, and NPU computation is initiated by the CPU calling the RKNN API. This process aims to meet the deployment requirements of intelligent chips with AI acceleration capabilities.

[0029] S5: Deploy and test the model, and use Mean Average Precision (mAP), Recall, Precision, Parameter, GFLOPs, and Detection Frames Per Second (FPS) as evaluation metrics for the model deployment performance.

[0030] This invention provides an implementation case to validate a mobile-based human fall detection method for photovoltaic power plant construction. The example dataset was collected at a photovoltaic power plant construction site in Puding County, Anshun City, Guizhou Province, China, during normal working hours for on-site personnel. The dataset encompasses images of falls taken from various shooting angles, under different environmental weather conditions, and with varying lighting. By collecting this data in a real photovoltaic power plant construction environment, various scenarios that may occur under the mountainous photovoltaic conditions of Guizhou can be better simulated, thus more comprehensively presenting the various forms and characteristics of typical human falls. Furthermore, the Mosaic dataset enhancement function is used to enrich the semantic information of the images. The dataset is in VOC format, with three categories: normal state, falling, and fallen. The original dataset contained 4029 images, which were enhanced to 12087 images using the Mosaic dataset enhancement function. Each image has a resolution of 640×640 pixels and was finally divided into training, validation, and test sets in an 8:1:1 ratio.

[0031] The system used for training in this invention is Windows 10, and the simulation software is Python 3.10. The model is built based on the PyTorch 2.01 framework. The CPU is an Intel Core i9-12900K, the GPU is an RTX-3090, and there is 32GB of RAM. The system used for deployment is Ubuntu 22.04, and the simulation software is Python 3.10. The model is built based on the PyTorch 2.01 framework and the Rknn-toolkit2 framework. The CPU is a Cortex-A76, A55, the GPU is an ARMMali-G610, and there is 8GB of RAM.

[0032] Figure 3 and Figure 4 This paper presents the average accuracy and performance metrics of several common classification models and improved models during model training. As shown in the figures, this paper compares them with current mainstream algorithms such as Nanodet, YOLOv5, YOLOv8n, YOLOv9t, and YOLOv11. To intuitively compare the performance differences between the improved models and other models, this paper uses the mean average accuracy as the metric to evaluate the training and testing processes of each model; the curves are detailed in [link to curve]. Figure 3 . Figure 3 The study shows the trend of the mean accuracy of different models during training with the number of iterations, indicating that the mean accuracy of each model gradually improves with training. Figure 4 This reveals the specific performance of each model in terms of precision, recall, number of parameters, computational cost, and frames per second, showing significant differences among the models in these metrics. Further analysis... Figure 4 Radar chart analysis shows that the CRO-YOLOv8 model has lower requirements in terms of parameter count and computational cost, while exhibiting the best performance in accuracy and recall, making it an ideal choice for edge computing scenarios that simultaneously meet the requirements of high accuracy and real-time performance. More performance parameters for each model are shown in Table 1, and the actual deployment performance of each model is shown in Table 2.

[0033] As shown in Table 1, among the 10 compared models, CRO-YOLO performed well in terms of mean precision, accuracy, and recall. Compared to the baseline model YOLOv8n, the improved CRO-YOLO improved mean precision, accuracy, and recall by 2.9%, 0.9%, and 3.7%, respectively, while reducing the number of model parameters and computational cost by 31.3% and 22.2%, respectively. This resulted in a significant reduction in computational cost and good real-time performance while maintaining high accuracy. Although Nanodet performed best in terms of parameter count and computational cost, enjoying the lowest computational cost, its significant decrease in mean precision made it difficult to achieve high accuracy requirements. While YOLOv9t had certain advantages in parameter count, computational cost, and model size, its mean precision, accuracy, and recall were all lower than the proposed CRO-YOLO model. Overall, compared to other lightweight models, CRO-YOLO demonstrates higher detection performance at a similar computational cost, achieving an effective balance between detection accuracy and efficiency. Its average accuracy of 88.6%, processing speed of 101.9 frames / s, parameter count of 2.06M, and computational requirement of 6.3GFLOPs make it an ideal choice for edge computing scenarios that balances high accuracy and real-time performance.

[0034] Table 1 Overall performance test results of each model

[0035] Table 2 shows that YOLOv9t and YOLOv11n may have slower inference speeds due to their relatively complex model structures or lack of sufficient optimization; while YOLOv5 and YOLOv8n perform excellently in inference speed, but their high power consumption is not conducive to long-term operation after deployment. Based on this, the improved model CRO-YOLO achieves a better balance between inference speed and power consumption, effectively reducing power consumption requirements while ensuring that the inference speed remains essentially unchanged. Therefore, CRO-YOLO balances performance and efficiency, making it a better choice for meeting practical application needs on resource-constrained mobile devices. See [link to specific deployment model] for details. Figure 5 As shown.

[0036] Table 2. Comparison of overall performance of each model after deployment

[0037] This invention proposes a mobile-based human fall detection method for photovoltaic power plant construction. Through case analysis and comparison with prediction models, the proposed model has the following advantages: It proposes to replace ordinary convolution with the OREPA reparameterization method. During the training phase, a multi-branch structure is used to enhance feature learning capabilities, and during the inference phase, parameter fusion is used to transform it into a single branch. This method significantly enhances the network's operating efficiency and robustness on mobile devices while ensuring fall detection accuracy. By flexibly adjusting the number of OREPA branches and the convolution kernel size, the network structure can be optimized according to the detection needs of different photovoltaic power plant scenarios, better adapting to data distribution and fall detection task types in complex environments. The lightweight cross-scale feature fusion module CCFM is used to reconstruct the neck network. Through depthwise separable convolution and direct cross-scale interaction, it helps to effectively preserve multi-scale fall features—directly mapping small-scale human detail features and large-scale semantic features to the fusion output layer, reducing the loss of original feature information. Meanwhile, the CCFM module also promotes the reuse of features at different scales, enhances the model's ability to express fall actions at different postures and distances, and allows low-level detailed features to be directly passed to high-level semantic layers, thereby generating higher-level fall feature representations and significantly improving the network's generalization ability in complex photovoltaic scenarios. By combining OREPA and CCFM to build a lightweight backbone-neck architecture and improving the detection head through RepConv reparameterization, the original "short and wide" structure of the model is optimized into a "thin and long" compact structure, which effectively reduces the demand for scarce hardware RAM resources and improves the model's compatibility with resource-intensive mobile RK3588 boards. The reduction in hardware memory accesses also further reduces the power consumption required for the model to operate on mobile devices, meeting the low-power requirements of real-time detection in photovoltaic power plants.

Claims

1. A mobile-based human fall detection method based on YOLOv8 for photovoltaic power plant construction, characterized in that, Includes the following steps: S1. Data Acquisition and Preprocessing: Collect data related to human falls, and process the data using image preprocessing and dataset augmentation techniques; S2: Targeted algorithm improvements. Based on the lightweight YOLOv8 series models, the improved detection model is constructed through the following optimizations: ① Replacing the traditional convolutional layers in the model with reparameterized convolution; ② Introducing a lightweight cross-scale feature fusion module to reconstruct the neck network; ③ Improving the detection head with a reparameterized convolutional structure. S3: Model quantization, which converts the trained floating-point model into a low-precision model using linear quantization techniques; S4: Mobile model deployment adapts the quantized model to the CPU and NPU deployment environment on mobile devices to achieve fall detection functionality; S5: Average precision, recall, accuracy, number of parameters, computational cost, and detection frame rate are used as evaluation metrics for model performance.

2. The detection method according to claim 1, characterized in that: In step S1, the image preprocessing techniques include image denoising, overall normalization, and contrast enhancement; the dataset augmentation technique adopts the Mosaic stitching enhancement method, which expands the dataset by randomly selecting multiple preprocessed images and randomly stitching and scaling them.

3. The method according to claim 2, characterized in that, In step S2, the reparameterized convolution method includes the OREPA reparameterization method, the lightweight cross-scale feature fusion module includes the CCFM module, and the reparameterized convolution structure includes the RepConv reparameterized convolution.

4. The method according to claim 3, characterized in that, In step S3, the floating-point model is in float32 format and the low-precision model is int8 format. The quantization process achieves linear transformation of the original data through scaling factor, zero-point mapping, rounding function and numerical clipping operation.

5. The method according to claim 4, characterized in that, The model deployment in step S4 includes: CPU deployment, extracting the model weight parameters obtained from training layer by layer and converting them into bin format for storage, and loading them into RAM layer by layer by the CPU to complete the calculation; NPU deployment, converting the trained model into onnx format, generating a dedicated format model adapted to NPU after int8 quantization, storing it through the mobile terminal's DDR memory, and having the CPU call the NPU interface to perform the calculation.

6. The method according to any one of claims 1-5, characterized in that, The method is applied to the construction of mountain photovoltaic power stations, and the mobile device is a low-resource, low-power edge computing device.

Images