A tower crane pin shaft component fault detection method, system, device and storage medium

Abstract

The application discloses a tower crane pin shaft component fault detection method, system, device and storage medium, the method comprises the following steps: locking the tower crane control system, and collecting the image of the tower crane pin shaft component to be detected in real time; according to the image of the tower crane pin shaft component to be detected, the installation condition category of the tower crane pin shaft component is identified through a pre-trained target detection model; according to the installation condition category, the fault condition of the tower crane pin shaft component is determined; according to the fault condition, fault warning is carried out, and the step of collecting the image of the tower crane pin shaft component to be detected in real time is returned until all the tower crane pin shaft components meet the preset condition, then the tower crane control system is unlocked. The embodiment of the application can intelligently, quickly and accurately automatically detect the installation condition category of the tower crane pin shaft component, improve the detection efficiency, and reduce the accident risk. The embodiment of the application can be widely applied to the field of construction site safety detection technology.

Description

Technical Field

[0001] This invention relates to the field of construction site safety inspection technology, and in particular to a method, system, equipment and storage medium for detecting faults in tower crane pin components. Background Technology

[0002] Tower cranes are common lifting equipment on construction sites. Due to the complexity of operation and environment, ensuring the safety of tower cranes during operation is paramount. Whether the tower crane's pin components are properly installed and correctly connected directly affects the safety of tower crane operation. Currently, in high-altitude operations on construction sites, manual inspection of the installation of tower crane pin components is often required, which is inefficient and increases the risk of accidents. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide an intelligent method, system, device and storage medium for detecting faults in tower crane pin components, aiming to achieve automatic detection of faults in tower crane pin components, improve detection efficiency and reduce accident risks.

[0004] To achieve the above objectives, one aspect of the present invention provides a method for detecting faults in tower crane pin components, the method comprising:

[0005] Lock the tower crane control system and acquire images of the tower crane pin components under test in real time;

[0006] Based on the image of the tower crane pin component to be detected, the installation category of the tower crane pin component is identified by a pre-trained target detection model;

[0007] The fault condition of the tower crane pin assembly is determined based on the installation condition category;

[0008] Based on the fault condition, a fault alarm is triggered, and the process returns to the step of real-time acquisition of images of the tower crane pin components to be inspected, until all the tower crane pin components meet the preset conditions, at which point the tower crane control system is unlocked.

[0009] Optionally, the training steps of the object detection model include:

[0010] Obtain an image dataset of labeled tower crane pin components;

[0011] The image dataset is denoised and cropped to obtain the first dataset;

[0012] The first dataset is augmented to obtain the second dataset; wherein the data augmentation includes at least one of random cropping, random flipping, and random scaling.

[0013] An initial detection model was constructed based on the PP-YOLOv2 pre-trained model;

[0014] The initial detection model is trained using the second dataset to obtain the target detection model.

[0015] Optionally, the step of constructing the initial detection model based on the PP-YOLOv2 pre-trained model includes:

[0016] Based on the network framework of the PP-YOLOv2 pre-trained model, the backbone network of the initial detection model is constructed using the ResNet50-vd residual network.

[0017] A first network for the initial detection model is constructed using a feature pyramid network; wherein, the first network is used to integrate the features output by the backbone network;

[0018] The detection head network of the initial detection model is constructed by sequentially arranging 3×3 convolutional layers and 1×1 convolutional layers;

[0019] Three anchor points are configured at each feature location in the feature map output by the initial detection model.

[0020] Configure the IoU-perceptual loss function in the output layer of the initial detection model.

[0021] Optionally, the step of training the initial detection model using the second dataset to obtain the target detection model includes:

[0022] The second dataset is divided into a training set and a validation set;

[0023] Configure the hyperparameters and training conditions of the model; wherein the hyperparameters include the learning rate, image input specifications, and batch size; and the training conditions include the number of iterations.

[0024] The initial detection model is trained using the training set according to the number of iterations, and the IoU loss value between the predicted box and the ground box is calculated using the IoU perceptual loss function.

[0025] The position and size of the predicted bounding boxes in the initial detection model are optimized based on the IoU loss value to obtain the first detection model;

[0026] The first detection model is fitted using a validation set to determine the hyperparameter values, thus obtaining the target detection model.

[0027] Optionally, the real-time acquisition of images of the tower crane pin components to be inspected includes:

[0028] Images of the tower crane pin assembly were captured using a camera.

[0029] The input stream from the camera is imported into the edge computing box to obtain an image of the tower crane pin component to be detected.

[0030] Optionally, the method further includes:

[0031] The images of the tower crane pin components to be inspected are encrypted and stored to obtain a first image set;

[0032] The images in the first image set are desensitized to obtain the third dataset;

[0033] The target detection model is updated based on the third dataset.

[0034] Optionally, the step of issuing a fault alarm based on the fault condition includes:

[0035] When the fault monitoring data exceeds the set threshold range, an alarm signal or alarm message is generated;

[0036] The alarm signal or alarm information shall be used to notify relevant personnel; wherein the alarm includes at least one of the following: sound alarm, SMS, email, and APP push.

[0037] Another aspect of this invention provides a tower crane pin component fault detection system, comprising:

[0038] The first module is used to lock the tower crane control system and acquire images of the tower crane pin components under test in real time.

[0039] The second module is used to identify the installation category of the tower crane pin component based on the image of the tower crane pin component to be detected using a pre-trained target detection model;

[0040] The third module is used to determine the fault status of the tower crane pin component based on the installation status category;

[0041] The fourth module is used to issue a fault alarm based on the fault condition and return to the step of real-time acquisition of images of the tower crane pin components to be tested until all the tower crane pin components meet the preset conditions, then the tower crane control system is unlocked.

[0042] The system may further include a fifth module: used to encrypt and store images of the tower crane pin components to be detected, to obtain a first image set; to desensitize the images in the first image set, to obtain a third dataset; and to update the target detection model based on the third dataset.

[0043] This invention also provides an electronic device, including a processor and a memory; the memory is used to store a program; the processor executes the program to implement the method described above.

[0044] This invention also provides a computer-readable storage medium storing a program that is executed by a processor to implement the method described above.

[0045] The embodiments of the present invention have the following beneficial effects: by locking the tower crane control system and acquiring images of the tower crane pin components to be detected in real time; based on the images of the tower crane pin components to be detected, identifying the installation status category of the tower crane pin components through a pre-trained target detection model; determining the fault status of the tower crane pin components based on the installation status category; issuing a fault alarm based on the fault status, and returning to the step of acquiring images of the tower crane pin components to be detected in real time until all tower crane pin components meet the preset conditions, the entire process of unlocking the tower crane control system can intelligently, quickly, and accurately automatically detect the installation status category of the tower crane pin components, improving detection efficiency, and performing detection work during the process of locking the tower crane control system can reduce the risk of accidents. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is a flowchart of the steps of the tower crane pin component fault detection method provided in the embodiment of the present invention;

[0048] Figure 2 This is a flowchart illustrating the tower crane pin component fault detection method provided in an embodiment of the present invention;

[0049] Figure 3 This is a schematic diagram of the data processing process of the tower crane pin component fault detection method provided in the embodiment of the present invention;

[0050] Figure 4 This is a network framework structure diagram of the PP-YOLOv2 pre-trained model provided in an embodiment of the present invention;

[0051] Figure 5 This is a flowchart of the model training process provided in an embodiment of the present invention;

[0052] Figure 6 This is a flowchart of the model deployment process provided in an embodiment of the present invention;

[0053] Figure 7 This is a schematic diagram of a tower crane pin component fault detection system module provided in an embodiment of the present invention;

[0054] Figure 8This is a flowchart of the tower crane pin component fault detection system provided in this embodiment of the invention;

[0055] Figure 9 This is a schematic diagram of the electronic device structure provided in an embodiment of the present invention. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0057] It should be noted that although functional modules are divided in the system diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the system or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0058] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein is for the purpose of describing embodiments of the invention only and is not intended to limit the invention.

[0059] In view of the problems of low efficiency and high accident risk of manual inspection in the existing technology, the present invention provides a method, system, equipment and storage medium for detecting faults in tower crane pin components, aiming to realize automatic detection of tower crane pin components, improve detection efficiency and reduce accident risk.

[0060] The present invention provides a method, system, device and storage medium for detecting faults in tower crane pin components, which will be specifically described through the following embodiments. First, a method for detecting faults in tower crane pin components according to an embodiment of the present invention is described.

[0061] Reference Figure 1 , Figure 2 and Figure 3 The tower crane pin component fault detection method of the present invention may include, but is not limited to, the following steps S100 to S400.

[0062] S100, lock the tower crane control system, and acquire images of the tower crane pin components to be tested in real time.

[0063] Specifically, locking the tower crane control system involves connecting to the tower crane central control system. Within the central control system, the operation is set to "tower crane lifting," initiating the lifting process. Simultaneously, only this sub-process (the detection process) is allowed to operate until the "tower crane lifting" process concludes, allowing the tower crane to stop operating and ensuring safe inspection and maintenance during the lifting process. In this embodiment, image data of the tower crane pin components can be acquired in real-time via a camera, or images of the tower crane pin components can be uploaded by the user. Step S100 includes steps S110 to S120.

[0064] S110. Take pictures of the tower crane pin components using a camera.

[0065] S120. Import the camera's input stream into the edge computing box to obtain an image of the tower crane pin component to be inspected.

[0066] Specifically, an edge computing box is an operating environment consisting of a network, sensors, processors, memory, batteries, and other hardware, used to process and store large amounts of data from IoT devices. An image of the tower crane pin component to be inspected can be obtained by directly connecting a camera to the edge computing box or by transmitting the camera's input stream to the edge computing box via a network.

[0067] S200. Based on the image of the tower crane pin component to be detected, identify the installation category of the tower crane pin component using a pre-trained target detection model.

[0068] The training steps for the object detection model include the following steps S210 to S250.

[0069] S210. Obtain the image dataset of the labeled tower crane pin components.

[0070] The image dataset consists of several images of tower crane pin components. Each image contains annotations indicating the pin component's location, size, shape, and installation status category. Installation status categories can include not connected, not properly installed, properly installed, and correctly connected. The annotation format is generally PASCAL VOC or COCO. PASCAL VOC is a commonly used object detection and image classification dataset containing images of multiple categories and their corresponding annotations. The PASCAL VOC annotation format is XML-based, with each image's annotation information stored in a corresponding XML file. COCO is a large-scale object detection, semantic segmentation, and image description dataset containing rich scenes, diverse objects, and complex annotations. The COCO dataset's annotation format is JSON-based.

[0071] The data annotation process is as follows: determine the number and location of pin components in each image; select each pin component by marking its location with a rectangle in the image; and categorize the selected rectangles by indicating the installation status of the pin components contained within them. Data annotation can be performed using LabelImg, a graphic image annotation tool that provides a visual interface for labeling targets on images and exporting annotation files.

[0072] In this embodiment of the invention, an annotated image dataset is obtained for model training.

[0073] S220. Denoise and crop the image dataset to obtain the first dataset.

[0074] S230. Perform data augmentation on the first dataset to obtain the second dataset; wherein the data augmentation includes at least one of random cropping, random flipping, and random scaling.

[0075] By performing random transformations or deformations on images, image datasets can be augmented to generate new datasets that increase sample size and diversity, thereby improving the effectiveness of model training.

[0076] In some embodiments, data augmentation may include at least one of random cropping, random flipping, and random scaling; random cropping and mirror flipping may increase the diversity of the dataset.

[0077] Random cropping is a technique that involves randomly cropping a portion of the original image according to a preset ratio to generate a new image, which can then be used as a training sample.

[0078] Random flipping involves randomly flipping the original image to generate a new image, which can then be used as a training sample. The flip direction can be randomly generated, and the flipping method can include horizontal or vertical flipping.

[0079] Random scaling involves randomly scaling the original image to generate a new image, which can then be used as a training sample. The scaling ratio can be randomly generated within a preset range, such as between 0.8 and 1.2.

[0080] In other embodiments, operations such as rotating, translating, adding noise, and color changing the image may also be included; rotation and scaling can improve the trained model's ability to detect tower crane pin components of different angles and sizes, while adding noise and color changes can improve the robustness of the trained model.

[0081] In some embodiments, OpenCV can be used to perform data augmentation on the first dataset. OpenCV (OpenSource Computer Vision Library) is an open-source computer vision and machine learning library that provides a series of functions and tools for processing images and videos. OpenCV provides multiple modules, including a core module, an image I / O and display module, an image processing module, a feature detection and description module, a computer vision module, a machine learning module, a deep learning module, etc., which can be used to perform random cropping, random flipping, and random scaling of images.

[0082] Based on the actual situation, select an appropriate data augmentation method to perform data augmentation processing on the first dataset to obtain the second dataset.

[0083] S240. Construct an initial detection model based on the PP-YOLOv2 pre-trained model.

[0084] Specifically, refer to Figure 4 , Figure 4 This is a network framework diagram of the PP-YOLOv2 pre-trained model provided in this embodiment of the invention. Here, conv represents the model's convolutional kernel, backbone represents the model's main network, FPN and Feature Pyramid serve as the feature pyramid, primarily responsible for feature fusion at each level, head is the detection head, and YOLO loss is the final object detection loss function. The PP-YOLOv2 model uses cross-entropy loss, L1 regularization loss, and object detection loss as three loss functions for backpropagation. This embodiment of the invention uses the above-mentioned PP-YOLOv2 model as a pre-trained model to construct an initial detection model.

[0085] Step S240 includes the following steps S241 to 245.

[0086] S241. A network framework based on the PP-YOLOv2 pre-trained model, using the ResNet50-vd residual network to construct the backbone network of the initial detection model.

[0087] Based on the network framework of the aforementioned PP-YOLOv2 pre-trained model, this embodiment of the invention replaces DarkNet53 with ResNet50-vd as the backbone network. To balance detection accuracy and efficiency, this embodiment replaces the convolution in the last stage. After replacement, the backbone network is ResNet50-vd-dcn, and the output features are C3, C4, and C5. C3, C4, and C5 are three feature layers of different dimensions input to the FPN. The feature vectors output after C3, C4, and C5 retain a large number of detailed features of the image, thus improving the accuracy of the detection model.

[0088] S242. A first network for the initial detection model is constructed using a feature pyramid network; wherein, the first network is used to integrate the features output by the backbone network.

[0089] The first network of the initial detection model is constructed using FPN (Feature Pyramid Network), also known as the Detection Neck Network, which is used to integrate the features output by the backbone network.

[0090] S243. The detection head network of the initial detection model is constructed by using 3×3 convolutional layers and 1×1 convolutional layers arranged in sequence.

[0091] The detection head network contains sequentially arranged 3×3 convolutional layers and 1×1 convolutional layers. The 3×3 convolutional processing is followed by the 1×1 convolutional processing to obtain the final detection result.

[0092] S244. Three anchor points are configured at each feature location in the feature map of the final output of the initial detection model.

[0093] Furthermore, the number of output channels for the detection results is 3(K+5), where K represents the number of installation categories of the pin component. Three different anchor points are configured at each feature position of the detected feature map. For each anchor point, the first K channels represent the predicted probability of class K, the next 4 channels represent the feature position, and the last channel represents the target score.

[0094] S245. Configure the IoU-perceptual loss function in the output layer of the initial detection model.

[0095] The perceptual loss function IoU is used to optimize the position and size of the predicted bounding box during training.

[0096] S250. The initial detection model is trained using the second dataset to obtain the target detection model.

[0097] Specifically, in this embodiment of the invention, a neural network based on the Paddle detection framework is loaded for model training to generate a four-class object detection model. This object detection model can identify and classify four installation categories to obtain the fault detection results of the tower crane pin components. (Refer to...) Figure 5 , Figure 5 This is a flowchart of the model training process provided in the embodiment of the present invention. Step S250 includes the following steps S251 to S254.

[0098] S251. Divide the second dataset into a training set and a validation set.

[0099] In some embodiments, the second dataset can be divided into a training set and a validation set in an 8:2 ratio.

[0100] S252. Configure the hyperparameters and training conditions of the model; among which, the hyperparameters include the learning rate, image input specifications, and batch size; the training conditions include the number of iterations.

[0101] Configure the model's hyperparameters. For example, before training begins, you can configure the initial learning rate to 0.0001, the image input size to 640×640 pixels, and the batch size to 12. Configure training conditions, such as configuring the number of training iterations to 1800 batches (epochs), using the adaptive learning rate gradient descent method (Adam) as the optimizer, and using dice_loss as the loss function.

[0102] S253. The initial detection model is trained using the training set according to the number of iterations, and the IoU perceptual loss function is used to calculate the IoU loss value between the predicted box and the ground truth box.

[0103] During training, a frozen layer is used. The backbone network layer is not trained in the first 100 epochs. After the 100th epoch, the backbone network layer is unfrozen. After unfreezing, the parameters of the feature extraction network of the backbone network layer will change during the training process.

[0104] Intersection over Union (IoU) is a metric for measuring the accuracy of object detection in a given dataset. IoU-aware loss is used in the network's output layer to calculate the IoU loss between predicted and ground truth boxes. Specifically, for each predicted box, its IoU value with the ground truth box is calculated, and the predicted box's position and size are penalized based on the magnitude of the IoU value. This allows the model to focus more on predicted boxes with lower IoU values, thereby optimizing the predicted box's position and size and improving the final detection accuracy.

[0105] S254. Optimize the position and size of the predicted boxes in the initial detection model based on the IoU loss value to obtain the first detection model.

[0106] In this embodiment of the invention, during optimization, the loss function used is binary cross-entropy (BCE) based on the IoU loss value, with a learning rate of p. The formula for calculating the binary cross-entropy loss in this embodiment of the invention is as follows:

[0107] loss=-t×log(σ(p))-(1-t)×log(1-σ(p))

[0108] Where t is the IoU between the predicted bounding box and the ground truth, and σ represents the sigmoid activation function. It's important to note that only the IoU-aware loss for positive samples is calculated.

[0109] S255. The first detection model is fitted using the validation set to determine the hyperparameter values ​​and obtain the target detection model.

[0110] In this embodiment of the invention, after obtaining the target detection model, the target detection model is deployed to the edge computing box, referring to... Figure 6 The deployment steps include:

[0111] 1. Convert the object detection model to a format suitable for edge computing boxes and export it. The trained object detection model is based on the PaddlePaddle framework, a deep learning framework typically implemented in Python. Converting the object detection model to a format suitable for edge computing boxes using Paddle Lite facilitates model loading and inference. Paddle Lite is a lightweight edge inference engine that enables the deployment of PaddlePaddle-trained deep learning models to edge devices such as mobile phones, IoT devices, embedded devices, and edge computing boxes.

[0112] 2. Load the exported object detection model into the edge computing box. To deploy the object detection model into a specific target edge computing box, the Paddle Lite model is converted to a format suitable for the target edge computing box using the framework and API provided by the target edge computing box for deploying neural network models.

[0113] S300. Determine the fault status of the tower crane pin components based on the installation category.

[0114] Specifically, the installation status categories can include not connected, not installed properly, installed properly, and properly connected. If the installation status is not connected or not installed properly, it is determined to be a component failure; if the installation status is installed properly or properly connected, it is determined to be a component without failure.

[0115] In other embodiments, the fault situation can be determined based on the number or location of component failures. For example, if the number of component failures exceeds a certain threshold, it is determined that a fault has occurred and maintenance is required.

[0116] S400: Issue a fault alarm based on the fault condition and return to the step of real-time acquisition of images of the tower crane pin components to be tested until all tower crane pin components meet the preset conditions, then unlock the tower crane control system.

[0117] Specifically, the preset conditions can be configured according to actual needs to determine whether an alarm or maintenance is required. When unlocking the tower crane control system, the specific operation is as follows: end this subprocess, release resources, and restore other tower crane operations. Step S400, which involves issuing an alarm based on the fault condition, may include the following steps S410 to S420.

[0118] S410. When the fault monitoring data exceeds the set threshold range, generate an alarm signal or alarm information.

[0119] S420. Notify relevant personnel of alarm signals or alarm information; wherein the alarm includes at least one of the following: sound alarm, SMS, email, and APP push.

[0120] Specifically, alarm notifications can improve safety and work efficiency at construction sites, reduce the risk of accidents caused by pin component failures, and ensure the safety of personnel at the construction site. Furthermore, real-time alarms can promptly remind workers to maintain and service the pin components, extending equipment lifespan, reducing maintenance costs, and improving construction efficiency and quality.

[0121] In some embodiments, when an abnormality is detected in the tower crane pin component or the abnormal data exceeds a set threshold range, an alarm mechanism is triggered to generate an alarm signal or alarm information. Specifically, this can be achieved by configuring a detection light and an alarm sound, so that the detection light is constantly on displaying a red light and the alarm sound is rang. The abnormal situation is then processed by cloud computing and pushed to the relevant person in charge via SMS, email or APP.

[0122] In some embodiments, historical inspection results can be summarized and analyzed, and various algorithms can be used to analyze and model the data of tower crane pin components to obtain information on operating status and fault prediction. Specifically, clustering algorithms can be used to classify historical inspection results into different states or categories, classification algorithms can be used to divide historical inspection results into normal and abnormal states, and regression algorithms can be used to predict the service life of tower crane pin components, which is beneficial for optimizing equipment maintenance plans and improving equipment lifespan and reliability.

[0123] Data analysis of tower crane pin assemblies can promptly identify faults and abnormal conditions, predict potential failures, and allow for timely maintenance and repair, thereby improving the efficiency and safety of tower crane operation.

[0124] The method in this embodiment of the invention may further include steps S500 to S700.

[0125] S500: Encrypt and store the images of the tower crane pin components to be inspected to obtain the first image set.

[0126] Specifically, the collected images of the tower crane pin components to be inspected are encrypted and stored using an encryption protocol to obtain the first image set.

[0127] S600. Desensitize the images in the first image set to obtain the third dataset.

[0128] The images in the first image set are desensitized to obtain the third dataset, which is used for model training again to help the model iteratively update.

[0129] S700. Update the object detection model based on the third dataset.

[0130] Using the third dataset as the second dataset and the object detection model as the initial detection model, the model training steps are repeated to update the model.

[0131] This invention also provides a tower crane pin component fault detection system, referring to... Figure 7 , Figure 7 This is a schematic diagram of a tower crane pin component fault detection system module provided in an embodiment of the present invention, including:

[0132] The first module is used to lock the tower crane control system and acquire images of the tower crane pin components under test in real time.

[0133] The second module is used to identify the installation category of the tower crane pin component based on the image of the tower crane pin component to be detected using a pre-trained target detection model;

[0134] The third module is used to determine the fault status of the tower crane pin components based on the installation condition category;

[0135] The fourth module is used to issue fault alarms based on the fault conditions and return to the step of real-time acquisition of images of the tower crane pin components to be tested until all tower crane pin components meet the preset conditions, at which point the tower crane control system is unlocked.

[0136] The system may also include a fifth module: used to encrypt and store images of the tower crane pin components to be detected, to obtain a first image set; to desensitize the images in the first image set, to obtain a third dataset; and to update the target detection model based on the third dataset.

[0137] Reference Figure 8 , Figure 8This is a flowchart of the tower crane pin component fault detection system provided in an embodiment of the present invention. In some embodiments, the system trains a target detection model according to the method of the present invention, and then deploys and runs the model. First, the system is powered on and starts up. After system checks and program initialization, the deployed target detection model is used to detect the installation status of the pin component. When connected to the network, remote monitoring, alarms, and management are performed. When not connected to the network, the remote and control interfaces are turned off.

[0138] This invention also provides an electronic device, with reference to... Figure 9 It includes a processor and memory; the memory is used to store programs; the processor executes programs to implement the methods described above.

[0139] This invention also provides a computer-readable storage medium storing a program, which is executed by a processor to implement the method described above.

[0140] The embodiments of the present invention have the following beneficial effects:

[0141] 1. It can acquire images of tower crane pin components to be inspected in real time, and perform fault detection on the images of tower crane pin components to be inspected through target detection models. It can automatically detect the installation status of tower crane pin components in an intelligent, fast and accurate manner, thereby improving detection efficiency and reducing the risk of accidents.

[0142] 2. It can perform detection during the process of controlling the tower crane to stop working, ensuring the accuracy of the detection data and the safety of operation and maintenance personnel;

[0143] 3. It can detect and identify potential problems with tower crane pin components as early as possible, and provide timely alarm information, which helps to reduce accident risks and improve work safety; intelligent and automatic fault detection helps to reduce the labor costs of tower crane pin component maintenance.

[0144] The effectiveness and practicality of a tower crane pin component fault detection method according to an embodiment of the present invention are described below, based on experimental results:

[0145] To verify the effectiveness and practicality of the tower crane pin component fault detection method of the present invention, the present invention embodiment was compared with YOLOv5x and PPYOLOv1 methods in terms of accuracy, average algorithm time consumption, and memory consumption. The results are shown in Table 1 below. Table 1 is a comparison table of the detection accuracy and speed of the present invention embodiment, YOLOv5x, and PPYOLOv1 for four installation categories of pin components.

[0146] Table 1

[0147]

[0148] Referring to Table 1, where mAP (Mean Average Precision) represents the average accuracy, in terms of the most important aspect of accuracy, the method of this embodiment achieves an accuracy of 98.95%, compared to 96.3% for the YOLOv5x model and 92.82% for PPYOLOv1. The method of this embodiment significantly outperforms YOLOv5x and PPYOLOv1. Through the lightweight network structure and algorithm optimization of PP-YOLOv2 on mobile devices in this embodiment, the accuracy of this embodiment is greatly improved.

[0149] Furthermore, as can be seen from Table 1, the detection speed of the method in this embodiment is much faster than that of YOLOv5x and PPYOLOv1. Therefore, the method in this embodiment can quickly detect the uranium-bearing components of tower cranes.

[0150] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this invention are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and sub-operations described as part of a larger operation are executed independently.

[0151] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0152] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0153] More specific examples (a non-exhaustive list) of computer-readable media include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0154] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0155] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

[0156] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of the present invention.

Claims

1. A method for detecting faults in tower crane pin components, characterized in that, include: Lock the tower crane control system and acquire images of the tower crane pin components under test in real time; Based on the image of the tower crane pin component to be detected, the installation category of the tower crane pin component is identified by a pre-trained target detection model; The fault condition of the tower crane pin assembly is determined based on the installation condition category; Based on the fault condition, a fault alarm is triggered, and the process returns to the step of real-time acquisition of images of the tower crane pin components to be inspected, until all the tower crane pin components meet the preset conditions, at which point the tower crane control system is unlocked. The training steps of the target detection model include: Obtain an image dataset of labeled tower crane pin components; The image dataset is denoised and cropped to obtain the first dataset; The first dataset is augmented to obtain the second dataset; wherein the data augmentation includes at least one of random cropping, random flipping, and random scaling. An initial detection model was constructed based on the PP-YOLOv2 pre-trained model; The initial detection model is trained using the second dataset to obtain the target detection model; The step of training the initial detection model using the second dataset to obtain the target detection model includes: The second dataset is divided into a training set and a validation set; Configure the hyperparameters and training conditions of the model; wherein the hyperparameters include the learning rate, image input specifications, and batch size; and the training conditions include the number of iterations. The initial detection model is trained using the training set according to the number of iterations, and the IoU perceptual loss function is used to calculate the IoU loss value between the predicted box and the ground truth box. The position and size of the predicted bounding boxes in the initial detection model are optimized based on the IoU loss value to obtain the first detection model; The first detection model is fitted using a validation set to determine the hyperparameter values, thereby obtaining the target detection model; The random cropping is used to randomly crop a portion of the original image according to a preset ratio to generate a new image as a training sample; The random flipping is used to randomly flip the original image to generate a new image as a training sample. The flipping direction is randomly generated, and the flipping method includes horizontal flipping or vertical flipping. The random scaling is used to randomly scale the original image to generate a new image as a training sample. The scaling ratio is randomly generated within a preset range. The installation status categories include not connected, not installed properly, installed properly, and properly connected. The identification of the installation status categories is constructed based on the following annotation method: determining the number and location of pin components in each image; selecting each pin component by marking its location in the image with a rectangle; and labeling the selected rectangle with the installation status category of the pin components contained within it. In the output feature map of the target detection model, each feature location is set with three anchor points, and the number of output channels corresponding to each anchor point is 3(K+5), where K represents the number of installation status categories of the tower crane pin component; for each anchor point, the first K channels are used to represent the predicted probability of the corresponding K installation status categories, the next 4 channels are used to represent the target's location information, and the last 1 channel is used to represent the target score; The construction of the initial detection model based on the PP-YOLOv2 pre-trained model includes: Based on the network framework of the PP-YOLOv2 pre-trained model, the backbone network of the initial detection model is constructed using the ResNet50-vd residual network. A first network for the initial detection model is constructed using a feature pyramid network; wherein, the first network is used to integrate the features output by the backbone network; The detection head network of the initial detection model is constructed by sequentially arranging 3×3 convolutional layers and 1×1 convolutional layers; Three anchor points are configured at each feature location in the feature map output by the initial detection model. Configure an IoU-perceptual loss function in the output layer of the initial detection model; The real-time acquisition of images of the tower crane pin components to be inspected includes: Images of the tower crane pin assembly were captured using a camera. The input stream from the camera is imported into the edge computing box to obtain an image of the tower crane pin component to be detected; The method further includes: The images of the tower crane pin components to be inspected are encrypted and stored to obtain a first image set; The images in the first image set are desensitized to obtain the third dataset; The target detection model is updated based on the third dataset.

2. The method for detecting faults in tower crane pin components according to claim 1, characterized in that, The step of issuing a fault alarm based on the fault condition includes: When the fault monitoring data exceeds the set threshold range, an alarm signal or alarm message is generated; The alarm signal or alarm information shall be used to notify relevant personnel; wherein the alarm includes at least one of the following: sound alarm, SMS, email, and APP push.

3. A tower crane pin component fault detection system, said system being applied to the method described in any one of claims 1-2, characterized in that, include: The first module is used to lock the tower crane control system and acquire images of the tower crane pin components under test in real time. The second module is used to identify the installation category of the tower crane pin component based on the image of the tower crane pin component to be detected using a pre-trained target detection model; The third module is used to determine the fault status of the tower crane pin component based on the installation status category; The fourth module is used to issue a fault alarm based on the fault condition and return to the step of real-time acquisition of images of the tower crane pin components to be tested until all the tower crane pin components meet the preset conditions, then the tower crane control system is unlocked. The training steps of the target detection model include: Obtain an image dataset of labeled tower crane pin components; The image dataset is denoised and cropped to obtain the first dataset; The first dataset is augmented to obtain the second dataset; wherein the data augmentation includes at least one of random cropping, random flipping, and random scaling. An initial detection model was constructed based on the PP-YOLOv2 pre-trained model; The initial detection model is trained using the second dataset to obtain the target detection model; The step of training the initial detection model using the second dataset to obtain the target detection model includes: The second dataset is divided into a training set and a validation set; Configure the hyperparameters and training conditions of the model; wherein the hyperparameters include the learning rate, image input specifications, and batch size; and the training conditions include the number of iterations. The initial detection model is trained using the training set according to the number of iterations, and the IoU perceptual loss function is used to calculate the IoU loss value between the predicted box and the ground truth box. The position and size of the predicted bounding boxes in the initial detection model are optimized based on the IoU loss value to obtain the first detection model; The first detection model is fitted using a validation set to determine the hyperparameter values, thereby obtaining the target detection model; The random cropping is used to randomly crop a portion of the original image according to a preset ratio to generate a new image as a training sample; The random flipping is used to randomly flip the original image to generate a new image as a training sample. The flipping direction is randomly generated, and the flipping method includes horizontal flipping or vertical flipping. The random scaling is used to randomly scale the original image to generate a new image as a training sample. The scaling ratio is randomly generated within a preset range.

4. An electronic device, characterized in that, Including the processor and memory; The memory is used to store programs; The processor executes the program to implement the method as described in any one of claims 1 to 2.

5. A computer-readable storage medium, characterized in that, The storage medium stores a program that is executed by a processor to implement the method as described in any one of claims 1 to 2.

Images