Freight cableway transportation safety monitoring method and device
Through the transfer learning of deep learning algorithms and YOLOv3 model, the problem of low accuracy of the monitoring system around the freight cableway is solved, accurate identification and alarm of foreign objects is realized, and construction safety is improved.
Patent Information
- Application Number
- CN202411303880.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-19
- Publication Date
- 2025-01-07
AI Technical Summary
The existing freight cableway peripheral monitoring system has low accuracy in complex environments, which is prone to false alarms, affecting construction safety.
Using deep learning algorithms, use LabImg tool to label video images, configure the YOLOv3 model, and introduce an adaptive multi-scale attention mechanism. Through transfer learning, train the model, generate the optimal weight file, deploy it to the monitoring system and jointly debug with the camera, and set up foreign objects. Intrusion alarm threshold to achieve accurate identification and alarm.
It improves the accuracy of foreign objects recognition around the freight cableway, reduces the false alarm rate, enhances the safety of the construction environment, and reduces the probability of accidents.
Smart Images
Figure CN119274130A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of power transmission engineering construction, and in particular to a freight ropeway transportation safety monitoring method and a freight ropeway transportation safety monitoring device based on the freight ropeway transportation safety monitoring method. Background Art
[0002] In the process of power transmission project construction in areas with high altitude and complex terrain such as hills and mountains, freight ropeways are usually used to transport construction materials due to their advantages such as high efficiency, low energy consumption and environmental protection.
[0003] Since the freight ropeway has a long transportation distance and is usually located in an area with complex terrain, when foreign objects such as people or animals invade the operation site of the freight ropeway, it will not only affect the normal operation of the freight ropeway, but also easily cause injuries to people or animals. Therefore, the construction process requires monitoring of the surrounding area of the freight ropeway and the upper and lower loading and unloading areas. At present, the surrounding area of the freight ropeway is generally monitored by installing a monitoring system around the freight ropeway, and the motion detection function of the camera in the monitoring system is used for early warning. However, due to the complex surrounding environment of the freight ropeway and the movement of sports cars on the ropeway during the operation of the freight ropeway, the motion detection of the camera is easily triggered by mistake, resulting in a false alarm. Therefore, the accuracy of the existing monitoring system for monitoring the surrounding area of the freight ropeway is low. Summary of the invention
[0004] In view of the deficiencies in the prior art, the present invention provides a freight ropeway transportation safety monitoring method and a freight ropeway transportation safety monitoring device based on the freight ropeway transportation safety monitoring method, which solves the problem of low accuracy of the monitoring system for monitoring the periphery of the freight ropeway in the prior art.
[0005] According to an embodiment of the present invention, a method for monitoring the safety of freight ropeway transportation includes the following steps:
[0006] S1. Obtain a video shot by a monitoring system for monitoring the surrounding area of the freight ropeway, extract frame images at fixed time intervals to obtain a number of images with foreign body intrusion, and pre-process the images;
[0007] S2, using the LabImg tool to annotate the foreign matter in the image obtained in step S1 to obtain an image set, and converting the annotated image set into a VOC data format to obtain a data set;
[0008] S3. Configure the YOLOv3 model, introduce the adaptive multi-scale attention mechanism, and use the dynamic activation function to replace the activation functions of all residual blocks in the backbone feature extraction network;
[0009] S4, using the data set obtained in step S2 and using the transfer learning method to train the YOLOv3 model, freezing the feature extraction network before 50 iterations of training, unfreezing the feature extraction network during the 51st to 300th iteration process, and synchronously training the output end until the loss function value curve converges, and exporting the optimal weight file of the evaluation index in the convergence stage;
[0010] S5. Deploy the optimal weight file obtained in step S4 to the monitoring system and debug it together with the camera in the monitoring system, set the foreign body intrusion alarm threshold, and the monitoring system will issue an alarm when the threshold is triggered.
[0011] On the other hand, according to an embodiment of the present invention, there is also provided a freight ropeway transportation safety monitoring device, including a monitoring system, the monitoring system including a video monitoring module and a data processing module that are communicatively connected, the video detection module including a plurality of cameras and the cameras are respectively arranged on the door frame of the freight ropeway, the data processing module including a memory and a processor, the memory storing a computer program that can be run on the processor, and the processor implementing the above-mentioned freight ropeway transportation safety monitoring method when executing the computer program.
[0012] Compared with the prior art, the present invention has the following beneficial effects: by adopting a deep learning algorithm to process video monitoring, firstly, the video of the monitoring area is collected and several images with foreign object intrusion are extracted, and the extracted images are preprocessed, and then the LabImg tool is used to annotate to obtain an image set, and the annotated image set is converted into a VOC data format to obtain a data set, and a YOLOv3 model is configured and trained by a transfer learning method to obtain an optimal weight file, and the optimal weight file is deployed to a monitoring system and jointly debugged with a camera in the monitoring system, and a foreign object intrusion alarm threshold is set to complete the detection of foreign object targets around the cableway and in the upper and lower loading and unloading areas, and when there is foreign object intrusion, the monitoring system issues an alarm, providing a decision-making basis for the cableway environmental safety early warning, which solves the technical problem of low accuracy of the existing monitoring system for monitoring the surrounding area of the freight cableway, and produces the technical effect of realizing accurate identification of foreign objects and improving the accuracy of the monitoring system alarm, which is beneficial to ensuring the safe operation of the freight cableway and reducing the probability of accidents. BRIEF DESCRIPTION OF THE DRAWINGS
[0013] Figure 1 This is a step diagram of a method for monitoring freight ropeway transportation safety according to Embodiment 1 of the present invention;
[0014] Figure 2 This is a schematic structural diagram of a freight ropeway transportation safety monitoring device according to a second embodiment of the present invention;
[0015] Figure 3It is a partial structural schematic diagram of a freight ropeway transportation safety monitoring device according to the second embodiment of the present invention.
[0016] In the above drawings: 1. Starting ground anchor; 2. Terminal ground anchor; 3. Steel wire rope; 4. Sports car; 5. Door frame; 6. Upper and lower loading and unloading areas; 701. Tension sensor; 702. Positioning sensor; 703. Distance measuring sensor; 801. Camera. DETAILED DESCRIPTION
[0017] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.
[0018] Embodiment 1
[0019] like Figure 1 As shown, an embodiment of the present invention proposes a method for monitoring the safety of freight ropeway transportation, which is used to improve the intelligence level of a monitoring system for monitoring the surrounding area of a freight ropeway and to improve the accuracy of the monitoring system in identifying foreign objects.
[0020] Please refer to Figure 1 The freight ropeway transportation safety monitoring method provided in this embodiment includes the following steps:
[0021] S1. Obtain a video shot by a monitoring system that monitors the surrounding area of the freight ropeway, extract frame images at fixed time intervals to obtain a number of images with foreign body intrusion, and pre-process the images.
[0022] In step S1, a video of intruding foreign objects captured by a monitoring system that monitors the surrounding area of the freight ropeway is obtained, the video is divided into frames and processed to extract images with intrusion of foreign objects at fixed time intervals (such as one frame per second), and the obtained images are preprocessed to improve the clarity of foreign objects in the images to ensure that the obtained images are valid.
[0023] S2. Use the LabImg tool to annotate the foreign objects in the image obtained in step S1 to obtain an image set, and convert the annotated image set into a VOC (Visual Object Classes) data format to obtain a data set.
[0024] In step S2, the LabImg tool is used to accurately annotate foreign objects (such as people, animals, and vehicles) in the image, and an annotation file is generated. The annotated file is converted into the VOC data format to obtain a data set, and a training set, a validation set, and a test set are generated for subsequent use.
[0025] S3. Configure the YOLOv3 model, introduce the adaptive multi-scale attention mechanism, and use the dynamic activation function to replace the activation functions of all residual blocks in the backbone feature extraction network.
[0026] In step S3, the environment is configured first, a deep learning framework (such as TensorFlow or PyTorch) is installed, and YOLOv3 and related dependent libraries are configured. An adaptive multi-scale attention mechanism is introduced in the process of configuring the YOLOv3 model, which can improve the ability to extract features of intruding foreign objects with complex backgrounds in images. Then, the activation functions of all residual blocks in the backbone feature extraction network are replaced with dynamic activation functions, which effectively avoids the zero gradient problem of back propagation of model training parameters under negative input conditions.
[0027] S4. Use the data set obtained in step S2 and adopt the transfer learning method to train the YOLOv3 model. Freeze the feature extraction network before 50 iterations of training. Unfreeze the feature extraction network during the 51st to 300th iteration process, and train synchronously with the output end until the loss function value curve converges, and export the optimal weight file of the evaluation index in the convergence stage.
[0028] In step S4, the transfer learning method is used to train the model end-to-end. The feature extraction network is frozen before 50 iterations of training. The feature extraction network is unfrozen during the 51st to 300th iteration process. The joint output end is trained synchronously until the loss function value curve converges. The weight file with the best evaluation index in the convergence stage is exported, so that the model can realize the recognition of specific foreign objects invading the monitoring screen.
[0029] S5. Deploy the optimal weight file obtained in step S4 to the monitoring system and debug it together with the camera in the monitoring system, set the foreign body intrusion alarm threshold, and the monitoring system will issue an alarm when the threshold is triggered.
[0030] In step S5, the optimal weight file obtained in step S4 is deployed to the monitoring system and jointly debugged with the camera in the monitoring system. The picture captured by the camera is transmitted to the monitoring system and inferred and identified through the model, and a foreign object intrusion alarm threshold is set in the monitoring system. When the model identifies a foreign object, the detection result is returned to the monitoring system and an alarm is issued through the monitoring system, thereby realizing accurate identification of foreign objects invading the freight ropeway operation site, which is beneficial to improving the accuracy of the monitoring system alarm.
[0031] In this embodiment, in step S1, preprocessing includes grayscale transformation, contrast adjustment and filtering of the image to improve the clarity of foreign matter in the image and facilitate subsequent image labeling.
[0032] In this embodiment, in step S3, the ASFF module is used to fuse images from different scales and different scenes, images of multiple scales are input and an adaptive pooling operation is performed on each image to scale its size, and then the weight of each image is calculated, the most representative image is selected, and the channels in the selected image are weighted according to the weight to obtain a feature vector.
[0033] In this embodiment, in step S4, the transfer learning formula is as follows:
[0034] D(s)={X s ,P(X s )},
[0035] D(t)={X t ,P(X t )},
[0036] Among them, s represents the source point, t represents the target point, D(s) is the source domain, D(t) is the target domain, Xs is the feature space of the source domain, Xt is the feature space of the target domain, P(Xs) is the marginal probability distribution corresponding to Xs, and P(Xt) is the marginal probability distribution corresponding to Xt.
[0037] In this embodiment, in step S4, during the training process of the transfer learning YOLOv3 model, a loss function is constructed from 9 dimensions in 3 aspects and 3 scales, and the network parameters are updated through reverse iteration. The formula is as follows:
[0038] L P0 =L P0_obj +L P0_location +L P0_cls ,
[0039] L P1 =L P1_obj +L P1_location +L P1_cls ,
[0040] L P2 =L P2_obj +L P2_location +L P2_cls ,
[0041] L total =L P0 +L P1 + LP2 ,
[0042] Wherein, the subscripts P0, P1, and P2 represent the feature maps of 52×52, 26×26, and 13×13, respectively; LP0, LP1, and LP2 represent the total losses for judging the presence or absence of a target, locating the target position, and classifying the target on the three feature maps of 52×52, 26×26, and 13×13, respectively; LP0_obj, LP1_obj, and LP2_obj are the losses used to judge whether an object is contained in feature maps of different sizes; LP0_location, LP1_location, and LP2_location represent the losses used to identify and determine the target position on feature maps of different sizes, respectively; LP0_cls, LP1_cls, and LP2_cls represent the losses used to predict the category to which the target belongs in feature maps of different sizes, respectively; Ltotal represents the total loss consumed in the process of identifying and classifying targets on three different feature maps.
[0043] Embodiment 2
[0044] like Figure 2 and Figure 3 As shown, the second embodiment of the present invention provides a freight ropeway transportation safety monitoring device, which is used to implement the freight ropeway transportation safety monitoring method of the first embodiment.
[0045] Please refer to Figure 2 and Figure 3 The freight ropeway transportation safety monitoring device provided in this embodiment includes a monitoring system, the monitoring system includes a video monitoring module and a data processing module that are communicatively connected, the video detection module includes a plurality of cameras 801 and the cameras 801 are respectively arranged on the door frame 5 of the freight ropeway, the data processing module includes a memory and a processor, the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, the freight ropeway transportation safety monitoring method in embodiment one is implemented.
[0046] Specifically, the operation process of the freight ropeway transportation safety monitoring device provided in this embodiment is as follows: the camera 801 installed on the gantry 5 of the freight ropeway captures video images of the surrounding area of the freight ropeway and the upper and lower loading and unloading areas 6 and sends the video to the data processing module; after receiving the image transmitted by the camera 801, the data processing module recognizes the image according to the optimal weight file obtained by training the freight ropeway transportation safety monitoring method in Example 1, and determines whether there is a specific foreign object invading the image; if there is a foreign object invading, an alarm is issued through the monitoring system to prompt the staff to handle it; if there is no foreign object invading, monitoring is continued, thereby realizing accurate identification of foreign objects invading the freight ropeway operation site, improving the accuracy of the monitoring system alarm while ensuring the safe operation of the freight ropeway and reducing the probability of accidents.
[0047] In this embodiment, two cameras 801 are respectively provided on the gantry 5, and the shooting directions of the two cameras 801 on the same gantry 5 are different. By respectively providing a plurality of cameras 801 on each gantry 5 of the freight ropeway, and the two cameras 801 on the same gantry 5 shoot in different directions, it is ensured that the images shot by the cameras 801 fully cover the periphery of the freight ropeway and the upper and lower loading and unloading areas 6, thereby reducing blind spots.
[0048] In this embodiment, a data monitoring module is also included, which includes a tension sensor 701, a positioning sensor 702 and a distance sensor 703. The tension sensor 701 is installed on the steel wire rope 3 of the starting anchor 1 and the steel wire rope 3 of the terminal anchor 2 of the freight cableway, and the positioning sensor 702 and the distance sensor 703 are respectively installed on the sports car 4 of the freight cableway. The tension sensor 701 in the data monitoring module is used to monitor the tension of the steel wire rope 3 of the starting anchor 1 and the tension of the steel wire rope 3 of the terminal anchor 2, and then determine whether the weight of the goods in the process of transporting the goods by the freight ropeway exceeds the bearing limit of the steel wire rope 3 and whether the steel wire rope 3 is broken or damaged. The positioning sensor 702 installed on the sports car 4 is used to monitor the operation process of the sports car 4, determine whether the route of the sports car 4 is correct and whether the sports car 4 reaches the loading and unloading position. The distance measuring sensor 703 is used to measure the distance of the sports car 4 to the ground during the movement of the sports car 4, so as to grasp the operation status of the sports car 4, that is, the data monitoring module is used to monitor the steel wire rope 3 and the sports car 4 of the freight ropeway, so as to determine the operation status of the freight ropeway, and promptly issue an alarm and stop the operation of the freight ropeway when a fault occurs in the freight ropeway.
[0049] In detail, the tension sensor 701 adopts a sensor based on MEMS sensor technology, the positioning sensor 702 selects a GPS sensor, and the distance measurement sensor 703 adopts a laser sensor with the emission port of the laser sensor facing the ground.
[0050] In this embodiment, the freight ropeway transportation safety monitoring device also includes a wireless communication module, and the wireless communication module includes a first wireless communicator, a second wireless communicator and a third wireless communicator. The first wireless communicator is electrically connected to the camera 801 and is communicatively connected to the data processing module, the second wireless communicator is electrically connected to the tension sensor 701 and is communicatively connected to the data processing module, and the third wireless communicator is electrically connected to the positioning sensor 702 and the ranging sensor 703 respectively and is communicatively connected to the data processing module. The freight ropeway transportation safety monitoring device also includes a first wireless communicator for transmitting the video image taken by the camera 801 to the data processing module, a second wireless communicator for transmitting the tension data of the wire rope 3 collected by the tension sensor 701 to the data processing module, and a third wireless communicator for transmitting the position data of the sports car 4 collected by the positioning sensor 702 and the height data of the sports car 4 collected by the ranging sensor 703 to the data processing module, so as to send the data collected by the data monitoring module to the data processing module for processing, so that the staff can monitor the operating status of the freight ropeway in the background of the monitoring system.
[0051] In this embodiment, the freight ropeway transportation safety monitoring device further includes a power supply module, which includes an electrically connected photovoltaic panel and a storage battery, the photovoltaic panel and the storage battery are respectively installed on the gantry 5, and the storage battery is electrically connected to the camera 801. The camera 801 arranged on the gantry 5 is maintained in operation by the power provided by the storage battery, and the photovoltaic panel converts solar energy into electrical energy and stores it in the storage battery for use by the camera 801, so that the freight ropeway transportation safety monitoring device can be powered, so that the freight ropeway transportation safety monitoring device can be conveniently installed in hilly, mountainous and other environments where electricity is inconvenient.
[0052] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.
Claims
1. A method for monitoring the safety of freight ropeway transportation, characterized in that: The following steps are included: S1. Obtain a video shot by a monitoring system for monitoring the surrounding area of the freight ropeway, extract frame images at fixed time intervals to obtain a number of images with foreign body intrusion, and pre-process the images; S2, using the LabImg tool to annotate the foreign matter in the image obtained in step S1 to obtain an image set, and converting the annotated image set into a VOC data format to obtain a data set; S3. Configure the YOLOv3 model, introduce the adaptive multi-scale attention mechanism, and use the dynamic activation function to replace the activation functions of all residual blocks in the backbone feature extraction network; S4, using the data set obtained in step S2 and adopting the transfer learning method to train the YOLOv3 model, freezing the feature extraction network before 50 iterations of training, unfreezing the feature extraction network during the 51st to 300th iteration process, and synchronously training the output end until the loss function value curve converges, and exporting the optimal weight file of the evaluation index in the convergence stage; S5. Deploy the optimal weight file obtained in step S4 to the monitoring system and debug it together with the camera in the monitoring system, set the foreign body intrusion alarm threshold, and the monitoring system will issue an alarm when the threshold is triggered.
2. A method for monitoring the safety of freight ropeway transportation as claimed in claim 1, characterized in that: In step S1, preprocessing includes grayscale conversion, contrast adjustment and filtering of the image.
3. A method for monitoring the safety of freight ropeway transportation as claimed in claim 1, characterized in that: In step S3, the ASFF module is used to fuse images from different scales and different scenes. Images of multiple scales are input and adaptive pooling operation is performed on each image to scale its size. The weight of each image is then calculated, and the most representative image is selected. The channels in the selected image are weighted according to the weight to obtain a feature vector.
4. A method for monitoring the safety of freight ropeway transportation as claimed in claim 1, characterized in that: In step S4, the transfer learning formula is as follows: D(s)={X s ,P(X s )}, D(t)={X t ,P(X t )}, Among them, s represents the source point, t represents the target point, D(s) is the source domain, D(t) is the target domain, Xs is the feature space of the source domain, Xt is the feature space of the target domain, P(Xs) is the marginal probability distribution corresponding to Xs, and P(Xt) is the marginal probability distribution corresponding to Xt.
5. A method for monitoring the safety of freight ropeway transportation as claimed in claim 4, characterized in that: In step S4, during the training process of the transfer learning YOLOv3 model, a loss function is constructed from 9 dimensions in 3 aspects and 3 scales, and the network parameters are updated through reverse iteration. The formula is as follows: L P0 =L P0_obj +L P0_location +L P0_cls , L P1 =L P1_obj +L P1_location +L P1_cls , L P2 =L P2_obj +L P2_location +L P2_cls , L total =L P0 +L P1 + LP2 , Wherein, the subscripts P0, P1, and P2 represent the feature maps of 52×52, 26×26, and 13×13, respectively; LP0, LP1, and LP2 represent the total losses for judging the presence or absence of a target, locating the target position, and classifying the target on the three feature maps of 52×52, 26×26, and 13×13, respectively; LP0_obj, LP1_obj, and LP2_obj are the losses used to judge whether an object is contained in feature maps of different sizes; LP0_location, LP1_location, and LP2_location represent the losses used to identify and determine the target position on feature maps of different sizes, respectively; LP0_cls, LP1_cls, and LP2_cls represent the losses used to predict the category to which the target belongs in feature maps of different sizes, respectively; Ltotal represents the total loss consumed in the process of identifying and classifying targets on three different feature maps.
6. A freight ropeway transportation safety monitoring device, characterized in that: It includes a monitoring system, which includes a video monitoring module and a data processing module that are communicatively connected, the video detection module includes multiple cameras and the cameras are respectively arranged on the door frame of the freight ropeway, the data processing module includes a memory and a processor, the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, it implements the freight ropeway transportation safety monitoring method described in any one of claims 1 to 5.
7. A freight ropeway transportation safety monitoring device as claimed in claim 6, characterized in that: Two cameras are respectively arranged on the gantry, and the shooting directions of the two cameras on the same gantry are different.
8. A freight ropeway transportation safety monitoring device as claimed in claim 6, characterized in that: It also includes a data monitoring module, which includes a tension sensor, a positioning sensor and a distance measuring sensor. The tension sensor is installed on the steel wire rope of the starting anchor and the steel wire rope of the terminal anchor of the freight cableway, and the positioning sensor and the distance measuring sensor are respectively installed on the sports car of the freight cableway.
9. A freight ropeway transportation safety monitoring device as claimed in claim 8, characterized in that: It also includes a wireless communication module, which includes a first wireless communicator, a second wireless communicator and a third wireless communicator. The first wireless communicator is electrically connected to the camera and is communicatively connected to the data processing module. The second wireless communicator is electrically connected to the tension sensor and is communicatively connected to the data processing module. The third wireless communicator is electrically connected to the positioning sensor and the ranging sensor respectively and is communicatively connected to the data processing module.
10. A freight ropeway transportation safety monitoring device as claimed in claim 6, characterized in that: It also includes a power supply module, which includes an electrically connected photovoltaic panel and a battery. The photovoltaic panel and the battery are respectively installed on the door frame, and the battery is electrically connected to the camera.