A truck loading and unloading crane pipe state recognition method and system

Abstract

The application provides a truck loading and unloading crane pipe state recognition method and system, the method comprises the following steps: constructing a crane pipe image data set; the data set comprises a first data set of crane pipe categories and a second data set of detection parts of crane pipes of different categories; constructing a first detection network model for crane pipe category recognition; constructing a second detection network model for crane pipe part recognition; using the trained first detection network model to predict the crane pipe category of a crane pipe image to be recognized; using the trained second detection network model to predict the crane pipe part of the image whose crane pipe category is predicted; and analyzing the crane pipe category prediction result and the crane pipe part prediction result to obtain the use state of the current crane pipe. Based on the method, the application further provides a truck loading and unloading crane pipe state recognition system. The application uses a detection model to predict the crane pipe category and part, can maximize the empowerment of deep learning technology to the chemical industry, and promotes the modernization of emergency management capability by informatization.

Description

Technical Field

[0001] This invention belongs to the technical field of the petrochemical industry, and specifically relates to a method and system for identifying the status of loading arms on loading and unloading vehicles. Background Technology

[0002] An loading arm, also known as a telescopic loading arm, is a type of retractable pipe used in the petroleum and chemical industries for loading and unloading liquids such as oil, water, and gas. Currently, loading arms are specialized equipment in the petroleum and chemical industries for fluid loading and unloading processes. They are also called fluid loading arms and use rotary joints to connect to rigid pipes and elbows to facilitate the transfer of liquid media between trains, tank cars, and storage and transportation pipelines on trestles.

[0003] Currently, higher requirements are being placed on the safe production of hazardous chemicals, and information technology is being used to strengthen the detection and early warning of accident-prone areas and processes, resolutely curbing the occurrence of various accidents. In recent years, with the development of deep learning technology, computer vision technology based on deep learning methods has begun to empower various industries, but its application in the petrochemical industry, especially in video surveillance, is relatively limited. Given the widespread adoption of video surveillance technology, how to provide a reliable and effective intelligent video surveillance prevention method for regulating loading and unloading behavior in the chemical industry is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0004] To address the aforementioned technical issues, this invention proposes a method and system for identifying the status of loading arms on loading and unloading vehicles. By employing a detection model to predict the type and location of the loading arms, this invention can maximize the application of deep learning technology in the chemical industry and promote the modernization of emergency management capabilities through information technology.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A method for identifying the status of loading arms on a loading vehicle includes the following steps:

[0007] Historical data of loading arm images were acquired to construct a loading arm image dataset; the dataset includes a first dataset of loading arm categories and a second dataset of detection sites for different categories of loading arms.

[0008] A first detection network model for identifying loading arm categories is constructed; the first detection network model is trained using a first dataset; a second detection network model for identifying loading arm parts is constructed, and the second detection network model is trained using a second dataset;

[0009] The trained first detection network model is used to predict the type of the loading arm in the image to be identified; the trained second detection network model is used to predict the location of the loading arm in the image with the predicted type; the results of the loading arm type prediction and the loading arm location prediction are analyzed to obtain the current usage status of the loading arm.

[0010] Furthermore, the process of acquiring historical data of loading arm images and constructing a loading arm image dataset includes:

[0011] Acquire video streams from monitoring equipment within the loading and unloading area at different time intervals;

[0012] The collected video stream was processed using a video frame extraction method to obtain the loading arm image;

[0013] The first dataset is obtained by labeling the types of loading arms in the loading arm images; after the first labeling of the loading arm types is completed, the loading arm parts are labeled a second time to form the second dataset.

[0014] The first dataset is divided into a first training set, a first validation set, and a first test set according to a preset ratio; the second dataset is divided into a second training set, a second validation set, and a second test set according to a preset ratio.

[0015] Furthermore, the labeling of the loading arm categories in the loading arm image includes labeling them as top loading arm, thick bottom loading arm, and thin bottom loading arm based on the installation position and thickness of the loading arm in the loading arm image.

[0016] After the category of the loading arm is marked once, the parts of the loading arm are marked a second time, including marking the inspection parts of the top loading arm, the thick bottom loading arm, and the thin bottom loading arm as the loading arm head, the loading arm tail, and the loading arm corner, respectively.

[0017] Furthermore, the process of training the first detection network model using the first dataset includes: training the first detection network model using the first training set;

[0018] The process of training the second detection network model using the second dataset includes: training the second detection network model using the second training set.

[0019] Furthermore, the first detection network model employs an attention mechanism added to the YOLOv5 algorithm. The principle of the attention mechanism is as follows: the first dataset is processed sequentially through the Backbone layer and Head layer of the YOLOv5 algorithm to extract features, resulting in a feature map of a preset size. By calculating the relationship between the target pixel in the image to be identified and all other points in the feature map, the features of the target pixel are weighted using the relationship to obtain effective target pixel features.

[0020] Furthermore, the second detection network model adopts the Hrnet keypoint detection algorithm; the detection process of the Hrnet keypoint detection algorithm is as follows: extract the feature maps corresponding to different detection parts; the corresponding position of the maximum feature value in the feature map is used as the key point of the loading arm head, loading arm tail and loading arm corner.

[0021] Furthermore, the process of using the trained first detection network model to predict the loading arm category of the loading arm image to be identified includes: inputting the loading arm image to be identified into the trained first detection network model, and using the trained first detection network model to extract features and predict the category of the loading arm image to be identified, thereby obtaining the loading arm category of the loading arm image to be identified.

[0022] Furthermore, the process of using the trained second detection network model to predict the loading arm category in the image includes: inputting the region of the loading arm identified by the first detection network model in the original loading arm image to the second detection network model for location prediction, and obtaining the key point coordinates and category information of different parts of the loading arm.

[0023] Furthermore, the process of analyzing the prediction results of the loading arm category and the prediction results of the loading arm location to obtain the current usage status of the loading arm includes:

[0024] Obtain the coordinates and category information of key points in different parts of the loading arm;

[0025] If the detection model predicts that the loading arm category is top loading arm, then there are 2 key point coordinates. Connecting the two key point coordinates will result in a line segment. If the angle between this line segment and the horizontal axis of the image is less than 10°, then the top loading arm is in use; otherwise, the top loading arm is not in use.

[0026] If the detection model predicts that the loading arm is a thick bottom loading arm, then there are 3 key point coordinates. Connecting the 3 key points will result in 2 line segments. If the line segments are of different lengths and the included angle between the two line segments is less than 100°, the detected thick bottom loading arm is located at the rear of the loading and unloading vehicle. In this case, the thick bottom loading arm is in use; otherwise, the thick bottom loading arm is not in use.

[0027] If the detection model predicts that the loading arm is a thin bottom loading arm, then there are three key point coordinates. Connecting the three key points yields two line segments. If the line segments are of different lengths, the angle between the two line segments is greater than 100°, and the detected thin bottom loading arm is located in the middle of the loading and unloading vehicle, then the thin bottom loading arm is in use; otherwise, the thin bottom loading arm is not in use.

[0028] The present invention also proposes a loading arm status recognition system, the system comprising a dataset construction module, a training module, and a recognition module;

[0029] The dataset construction module is used to acquire historical data of loading arm images and construct a loading arm image dataset; the dataset includes a first dataset of loading arm categories and a second dataset of detection sites for different categories of loading arm.

[0030] The training module is used to construct a first detection network model for identifying loading arm categories; train the first detection network model using a first dataset; construct a second detection network model for identifying loading arm parts; and train the second detection network model using a second dataset.

[0031] The recognition module is used to predict the type of the loading arm in the image to be recognized using a trained first detection network model; to predict the location of the loading arm in the image with the predicted type using a trained second detection network model; and to analyze the prediction results of the loading arm type and location to obtain the current usage status of the loading arm.

[0032] The effects described in the invention are merely those of the embodiments, and not all the effects of the invention. One of the above technical solutions has the following advantages or beneficial effects:

[0033] This invention proposes a method and system for identifying the status of loading arms on loading and unloading vehicles. The method includes acquiring historical image data of loading arms and constructing an image dataset. The dataset includes a first dataset of loading arm categories and a second dataset of detection locations for different categories of loading arms. A first detection network model for loading arm category identification is constructed. The first detection network model is trained using the first dataset. A second detection network model for loading arm location identification is constructed and trained using the second dataset. The trained first detection network model is used to predict the loading arm category of the images to be identified. The trained second detection network model is used to predict the loading arm location for images with predicted loading arm categories. The prediction results of the loading arm category and location are analyzed to obtain the current usage status of the loading arms. Based on this method for identifying the status of loading arms on loading and unloading vehicles, this invention also proposes a system for identifying the status of loading arms on loading and unloading vehicles. This invention uses a detection model to predict the loading arm category and location, which can maximize the application of deep learning technology in the chemical industry and promote the modernization of emergency management capabilities through informatization.

[0034] In this invention, when performing loading arm category identification, an improved YOLOv5 first detection network model is proposed. The CCF attention fusion module is added to the existing YOLOv5 target detection algorithm to detect and identify different loading arm images, thereby improving the model's ability to extract target features more effectively.

[0035] This invention combines a first detection network model and a second detection network model, with location detection as the primary method and category detection as the secondary method. It can identify the position and category information of different loading arms and different loading arm parts. By combining the identification results of the two algorithm models, the usage status of the loading arms can be determined, which greatly improves the accuracy and reliability of the loading arm status determination.

[0036] This invention analyzes the results after identification. First, it determines the type and location area of ​​the detected loading arm. Then, it filters out high-quality key point detection results. Finally, it combines the position information of different parts of the loading arm and judges the current usage status of the loading arm based on the angle and position of the loading arm. Attached Figure Description

[0037] like Figure 1 This is a flowchart of a loading and unloading arm status identification method according to Embodiment 1 of the present invention;

[0038] like Figure 2 This is a schematic diagram of the improved YOLOv5 target detection algorithm according to Embodiment 1 of the present invention;

[0039] like Figure 3 This is a schematic diagram of a loading and unloading arm status recognition system according to Embodiment 2 of the present invention. Detailed Implementation

[0040] To clearly illustrate the technical features of this solution, the invention will be described in detail below through specific embodiments and in conjunction with the accompanying drawings. The following disclosure provides many different embodiments or examples for implementing different structures of the invention. To simplify the disclosure of the invention, components and arrangements of specific examples are described below. Furthermore, reference numerals and / or letters may be repeated in different examples. This repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed. It should be noted that the components illustrated in the drawings are not necessarily drawn to scale. Descriptions of well-known components, processing techniques, and processes are omitted in this invention to avoid unnecessarily limiting the invention.

[0041] Example 1

[0042] Embodiment 1 of this invention proposes a method for recognizing the status of loading arms in loading and unloading vehicles, which solves the defects of loading arm status recognition in the prior art and enables computer vision technology to begin to empower the chemical industry.

[0043] A chemical plant's monitoring system captures real-time video footage. At regular intervals, several images are extracted from the captured video stream and fed into the target detection model (i.e., the loading arm category detection network model) and key point detection model (i.e., the key point detection network model for key points on the loading arm) used in this invention. The target detection model detects the loading arm's target information at a given moment, including its location and category. The key point detection model detects key point information on the loading arm at a given moment. Key point information includes the location and category of the loading arm's head, tail, and corners. The detected target and key point information are analyzed to determine the current loading arm usage status and to determine whether loading / unloading operations are currently in progress.

[0044] like Figure 1 This is a flowchart of a loading and unloading arm status identification method according to Embodiment 1 of the present invention;

[0045] In step S100, the process begins;

[0046] In step S110, historical data of loading arm images are obtained to construct a loading arm image dataset; the dataset includes a first dataset of loading arm categories and a second dataset of detection sites for different categories of loading arm.

[0047] The specific process includes: acquiring video data from several cameras located in the loading and unloading area of ​​the chemical plant and collecting video streams from different time periods; and processing the collected video streams using a video frame extraction method to obtain several images.

[0048] After compiling all the images, label them as top loading arm, thick bottom loading arm, and thin bottom loading arm according to the installation position and thickness of the loading arm in the images.

[0049] After the first labeling of the loading arm categories is completed, the second labeling of the loading arm parts is performed, including: labeling the detection parts of the top loading arm, the thick bottom loading arm, and the thin bottom loading arm as loading arm head, loading arm tail, and loading arm corner, respectively; the first dataset is obtained by labeling the loading arm categories in the loading arm images; the second dataset is obtained by labeling the loading arm parts after the first labeling of the loading arm categories is completed.

[0050] The first dataset is divided into a first training set, a first validation set, and a first test set according to a preset ratio; the second dataset is divided into a second training set, a second validation set, and a second test set according to a preset ratio.

[0051] In step S120, a first detection network model for identifying loading arm categories is constructed; the first detection network model is trained using a first dataset;

[0052] The first detection network model adopts an attention mechanism added to the YOLOv5 algorithm. The principle of the attention mechanism is as follows: the first dataset is processed sequentially through the Backbone layer and Head layer of the YOLOv5 algorithm to extract features and obtain a feature map of a preset size. By calculating the relationship between the target pixel in the image to be identified and all other points in the feature map, the features of the target pixel are weighted using the relationship to obtain effective target pixel features.

[0053] like Figure 2 This is a schematic diagram of the improved YOLOv5 object detection algorithm in Embodiment 1 of the present invention. The YOLOv5 object detection algorithm is improved by adding a CrissCross Attention (CCF) module. Borrowing from the principle of the self-attention mechanism, it uses a Q (Query) vector and a K (Key) vector to determine the weights, and then fuses them with the V (Value) vector. The attention module calculates the interrelationships between the target feature pixel and all other points in the feature map, and uses these interrelationships to weight the features of the target pixel, thereby obtaining more effective target pixel features.

[0054] The process of CCF attention module execution includes: the first dataset is sequentially processed by the backbone layer and head layer of the yolov5 algorithm to extract features and obtain a feature map H of size (C*H*W);

[0055] After passing through three 1*1 convolutional layers, features Q, K, and V are generated respectively. The size of Q (Query vector) and K (Key vector) is (C'*H*W), while the size of V (Value vector) remains unchanged at (C*H*W). The specific implementation includes: taking all channel values ​​of a certain pixel in the feature map of Q: q = Q(i,j), size = (1,c'); taking all channel values ​​of all pixels in the same row and column as q in the feature map of K, taking the intersection position twice, but only selecting once, k size = (c',H+W-1); q*k to get q_atten,size = (1,H+W-1), and performing a Softmax operation on these H+W-1 values, that is, the weight sum is 1; repeating the above process for all pixels in Q, that is, obtaining the normalized weight of each pixel, at this time Atten,size = (Batch,H,W,H+W-1).

[0056] The Q and K features Atten, size = (Batch, H, W, H + W - 1) obtained above are processed by Affinity and softmax operations respectively, then aggregated with the V feature, and finally connected to the input feature map H via residual concatenation. The specific process includes: taking all weights of a pixel in Atten, A = Atten(i, j), size = (1, H + W - 1); taking a certain channel C of V... n Feature map V n ,size=(h,w), select V n The value in the same row and column as A is denoted as V. n size = (1, h + w - 1); V n Multiplying by AT yields the weighted V. n Value, size = (1,1); repeat the above operation for all channels in V; repeat the channel-repeated operation for all pixels in Atten; residual network: H' = CCAAtten(H) + H.

[0057] A second detection network model for identifying the parts of the loading arm is constructed, and the second detection network model is trained using a second dataset. The detection process of the Hrnet keypoint detection algorithm is as follows: extract the feature maps corresponding to different detection parts; the corresponding position of the maximum feature value in the feature map is used as the key point of the loading arm head, loading arm tail and loading arm corner.

[0058] In step S130, the trained first detection network model is used to predict the loading arm category of the loading arm image to be identified; the trained second detection network model is used to predict the loading arm location of the image with the predicted loading arm category.

[0059] The process of predicting the category of the loading arm image to be identified using the trained first detection network model includes: inputting the loading arm image to be identified into the trained first detection network model, and performing feature extraction and category prediction on the loading arm image to be identified through the trained first detection network model to obtain the loading arm category of the loading arm image to be identified.

[0060] The process of predicting the loading arm location in an image with a predicted loading arm category using a trained second detection network model includes: inputting the region of the loading arm identified by the first detection network model in the original loading arm image to the second detection network model for location prediction, thereby obtaining the key point coordinates and category information of different parts of the loading arm.

[0061] In step S140, the prediction results of the loading arm category and the prediction results of the loading arm location are analyzed to obtain the current usage status of the loading arm.

[0062] Obtain the coordinates and category information of key points in different parts of the loading arm;

[0063] If the detection model predicts that the loading arm category is top loading arm, then there are 2 key point coordinates. Connecting the two key point coordinates will result in a line segment. If the angle between this line segment and the horizontal axis of the image is less than 10°, then the top loading arm is in use; otherwise, the top loading arm is not in use.

[0064] If the detection model predicts that the loading arm is a thick bottom loading arm, then there are 3 key point coordinates. Connecting the 3 key points will result in 2 line segments. If the line segments are of different lengths and the included angle between the two line segments is less than 100°, the detected thick bottom loading arm is located at the rear of the loading and unloading vehicle. In this case, the thick bottom loading arm is in use; otherwise, the thick bottom loading arm is not in use.

[0065] If the detection model predicts that the loading arm is a thin bottom loading arm, then there are three key point coordinates. Connecting the three key points yields two line segments. If the line segments are of different lengths, the angle between the two line segments is greater than 100°, and the detected thin bottom loading arm is located in the middle of the loading and unloading vehicle, then the thin bottom loading arm is in use; otherwise, the thin bottom loading arm is not in use.

[0066] In step S150, the current loading and unloading behavior is judged according to whether it is the operation time and the usage status of the loading arm, and an alarm is generated and pushed to the client for non-standard behavior.

[0067] If the current time is within the loading and unloading operation period and the status of different loading arms is "in use", then the loading and unloading behavior is compliant; otherwise, the loading and unloading behavior is non-compliant and an alarm will be pushed to the user terminal. If the current time is outside the loading and unloading operation period and the status of different loading arms is "not in use", then the loading and unloading behavior is compliant; otherwise, the loading and unloading behavior is non-compliant and an alarm will be pushed to the user terminal.

[0068] The loading arm status identification method proposed in Embodiment 1 of this invention uses a detection model to predict the type and location of the loading arm, which can maximize the application of deep learning technology in the chemical industry and promote the modernization of emergency management capabilities through informatization.

[0069] The present invention, in embodiment 1, proposes a method for identifying the status of loading arms in loading and unloading vehicles. When identifying the type of loading arms, an improved YOLOv5 first detection network model is proposed. The CCF attention fusion module is added to the existing YOLOv5 target detection algorithm to detect and identify different loading arm images, thereby improving the model to extract target features more effectively.

[0070] The loading arm status identification method proposed in Embodiment 1 of this invention combines a first detection network model and a second detection network model, with location detection as the main method and category detection as the auxiliary method. It can identify the position and category information of different loading arms and different loading arm parts. The identification results of the two algorithm models are combined to determine the usage status of the loading arm, which greatly improves the accuracy and reliability of the loading arm status identification.

[0071] The present invention provides a method for identifying the status of loading arms in a loading and unloading vehicle. The method analyzes the identification results, firstly determining the type and location area of ​​the detected loading arms, then filtering out high-quality key point detection results, and finally combining the position information of different parts of the loading arms to determine the current usage status of the loading arms based on the angle and position of the loading arms.

[0072] Example 2

[0073] Based on the loading and unloading arm status recognition method proposed in Embodiment 1 of the present invention, Embodiment 2 of the present invention also proposes a loading and unloading arm status recognition system, which includes a dataset construction module, a training module and a recognition module.

[0074] The dataset construction module is used to acquire historical data of loading arm images and construct a loading arm image dataset; the dataset includes a first dataset of loading arm categories and a second dataset of detection sites for different categories of loading arms.

[0075] The training module is used to construct a first detection network model for identifying loading arm categories; train the first detection network model using a first dataset; construct a second detection network model for identifying loading arm parts, and train the second detection network model using a second dataset;

[0076] The recognition module is used to predict the type of the loading arm in the image to be identified using the first detection network model after training; to predict the location of the loading arm in the image with the predicted type using the second detection network model after training; and to analyze the prediction results of the loading arm type and the location of the loading arm to obtain the current usage status of the loading arm.

[0077] The process of building the module includes: acquiring video data from several cameras located in the loading and unloading area of ​​the chemical plant and collecting video streams at different time periods; and processing the collected video streams using a video frame extraction method to obtain several images.

[0078] After compiling all the images, label them as top loading arm, thick bottom loading arm, and thin bottom loading arm according to the installation position and thickness of the loading arm in the images.

[0079] After the first labeling of the loading arm categories is completed, the second labeling of the loading arm parts is performed, including: labeling the detection parts of the top loading arm, the thick bottom loading arm, and the thin bottom loading arm as loading arm head, loading arm tail, and loading arm corner, respectively; the first dataset is obtained by labeling the loading arm categories in the loading arm images; the second dataset is obtained by labeling the loading arm parts after the first labeling of the loading arm categories is completed.

[0080] The first dataset is divided into a first training set, a first validation set, and a first test set according to a preset ratio; the second dataset is divided into a second training set, a second validation set, and a second test set according to a preset ratio.

[0081] The training module implementation process includes: the first detection network model adopts an attention mechanism added to the YOLOv5 algorithm; the principle of the attention mechanism is: the first dataset is processed sequentially through the Backbone layer and Head layer of the YOLOv5 algorithm to extract features and obtain a feature map of a preset size; by calculating the relationship between the target pixel in the image to be identified and all other points in the feature map, the features of the target pixel are weighted using the relationship to obtain effective target pixel features.

[0082] like Figure 2 This is a schematic diagram of the improved YOLOv5 object detection algorithm in Embodiment 1 of the present invention. The YOLOv5 object detection algorithm is improved by adding a CrissCross Attention (CCF) module. Borrowing from the principle of the self-attention mechanism, it uses a Q (Query) vector and a K (Key) vector to determine the weights, and then fuses them with the V (Value) vector. The attention module calculates the interrelationships between the target feature pixel and all other points in the feature map, and uses these interrelationships to weight the features of the target pixel, thereby obtaining more effective target pixel features.

[0083] The process of CCF attention module execution includes: the first dataset is sequentially processed by the backbone layer and head layer of the yolov5 algorithm to extract features and obtain a feature map H of size (C*H*W);

[0084] After passing through three 1*1 convolutional layers, features Q, K, and V are generated respectively. The size of Q (Query vector) and K (Key vector) is (C'*H*W), while the size of V (Value vector) remains unchanged at (C*H*W). The specific implementation includes: taking all channel values ​​of a certain pixel in the feature map of Q: q = Q(i,j), size = (1,c'); taking all channel values ​​of all pixels in the same row and column as q in the feature map of K, taking the intersection position twice, but only selecting once, k size = (c',H+W-1); q*k to get q_atten,size = (1,H+W-1), and performing a Softmax operation on these H+W-1 values, that is, the weight sum is 1; repeating the above process for all pixels in Q, that is, obtaining the normalized weight of each pixel, at this time Atten,size = (Batch,H,W,H+W-1).

[0085] The Q and K features Atten, size = (Batch, H, W, H + W - 1) obtained above are processed by Affinity and softmax operations respectively, then aggregated with the V feature, and finally connected to the input feature map H via residual concatenation. The specific process includes: taking all weights of a pixel in Atten, A = Atten(i, j), size = (1, H + W - 1); taking a certain channel C of V... n Feature map V n ,size=(h,w), select V n The value in the same row and column as A is denoted as V. n size = (1, h + w - 1); V n Multiplying by AT yields the weighted V. n Value, size = (1,1); repeat the above operation for all channels in V; repeat the channel-repeated operation for all pixels in Atten; residual network: H' = CCAAtten(H) + H.

[0086] A second detection network model for identifying the parts of the loading arm is constructed, and the second detection network model is trained using a second dataset. The detection process of the Hrnet keypoint detection algorithm is as follows: extract the feature maps corresponding to different detection parts; the corresponding position of the maximum feature value in the feature map is used as the key point of the loading arm head, loading arm tail and loading arm corner.

[0087] The process of implementing the recognition module includes: the process of predicting the category of the loading arm image to be recognized using the trained first detection network model includes: inputting the loading arm image to be recognized into the trained first detection network model, and performing feature extraction and category prediction on the loading arm image to be recognized through the trained first detection network model to obtain the loading arm category of the loading arm image to be recognized.

[0088] The process of predicting the loading arm location in an image with a predicted loading arm category using a trained second detection network model includes: inputting the region of the loading arm identified by the first detection network model in the original loading arm image to the second detection network model for location prediction, thereby obtaining the key point coordinates and category information of different parts of the loading arm.

[0089] The system also includes an analysis and reporting module; the process of the analysis and reporting module includes: analyzing the prediction results of the loading arm category and the prediction results of the loading arm location to obtain the current usage status of the loading arm.

[0090] Obtain the coordinates and category information of key points in different parts of the loading arm;

[0091] If the detection model predicts that the loading arm category is top loading arm, then there are 2 key point coordinates. Connecting the two key point coordinates will result in a line segment. If the angle between this line segment and the horizontal axis of the image is less than 10°, then the top loading arm is in use; otherwise, the top loading arm is not in use.

[0092] If the detection model predicts that the loading arm is a thick bottom loading arm, then there are 3 key point coordinates. Connecting the 3 key points will result in 2 line segments. If the line segments are of different lengths and the included angle between the two line segments is less than 100°, the detected thick bottom loading arm is located at the rear of the loading and unloading vehicle. In this case, the thick bottom loading arm is in use; otherwise, the thick bottom loading arm is not in use.

[0093] If the detection model predicts that the loading arm is a thin bottom loading arm, then there are three key point coordinates. Connecting the three key points yields two line segments. If the line segments are of different lengths, the angle between the two line segments is greater than 100°, and the detected thin bottom loading arm is located in the middle of the loading and unloading vehicle, then the thin bottom loading arm is in use; otherwise, the thin bottom loading arm is not in use.

[0094] Based on whether the current time is within the operation period and the status of the loading arms, the system determines whether the current loading and unloading behavior is in compliance with regulations, and generates an alarm and pushes it to the client if there is any non-compliant behavior. If the current time is within the loading and unloading operation period and the status of different loading arms is "in use", the loading and unloading behavior is compliant; otherwise, the loading and unloading behavior is non-compliant, and an alarm is pushed to the user's client. If the current time is outside the loading and unloading operation period and the status of different loading arms is "not in use", the loading and unloading behavior is compliant; otherwise, the loading and unloading behavior is non-compliant, and an alarm is pushed to the user's client.

[0095] The loading arm status recognition system proposed in Embodiment 2 of this invention uses a detection model to predict the type and location of the loading arm, which can maximize the application of deep learning technology in the chemical industry and promote the modernization of emergency management capabilities through informatization.

[0096] The loading arm status recognition system proposed in Embodiment 2 of this invention proposes an improved YOLOv5 first detection network model when recognizing loading arm categories. The CCF attention fusion module is added to the existing YOLOv5 target detection algorithm to detect and recognize different loading arm images, thereby improving the model to extract target features more effectively.

[0097] The loading arm status recognition system proposed in Embodiment 2 of this invention combines a first detection network model and a second detection network model. It focuses on location detection and supplements it with category detection. It can identify the position and category information of different loading arms and different loading arm parts. The system combines the recognition results of the two algorithm models to judge the usage status of the loading arms, which greatly improves the accuracy and reliability of the loading arm status judgment.

[0098] The loading arm status recognition system proposed in Embodiment 2 of this invention analyzes the recognition results. First, it determines the type and location area of ​​the detected loading arm. Then, it filters out high-quality key point detection results. Finally, it combines the position information of different parts of the loading arm and judges the current usage status of the loading arm based on the angle and position of the loading arm.

[0099] The description of the relevant part of the loading arm status recognition system provided in Embodiment 2 of this application can be found in the detailed description of the corresponding part of the server energy consumption test visualization method provided in Embodiment 1 of this application, and will not be repeated here.

[0100] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that the elements inherent in a process, method, article, or apparatus that includes a list of elements are included. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. Additionally, portions of the technical solutions provided in the embodiments of this application that are consistent with the implementation principles of corresponding technical solutions in the prior art have not been described in detail to avoid excessive elaboration.

[0101] While specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art can make other modifications or variations based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for identifying a status of a loading and unloading vehicle lance, characterized in that, The method comprises the following steps: acquiring crane image historical data and constructing a crane image dataset; the dataset comprises a first dataset of crane categories and a second dataset of detection positions of cranes of different categories; constructing a first detection network model for crane category identification; and training the first detection network model using the first dataset; constructing a second detection network model for crane position identification; and training the second detection network model using the second dataset; using the trained first detection network model to predict the category of a crane in a to-be-identified crane image; using the trained second detection network model to predict the position of the crane in the image whose category has been predicted; and analyzing the category prediction result and the position prediction result to obtain the use state of the current crane; the process of using the trained second detection network model to predict the position of the crane in the image whose category has been predicted comprises: inputting the region of the crane identified by the first detection network model in the original to-be-identified crane image into the second detection network model to predict the position, and obtaining the key point coordinates and category information of different positions of the crane; the process of analyzing the category prediction result and the position prediction result to obtain the use state of the current crane comprises: obtaining the key point coordinates and category information of different positions of the crane; if the category of the crane predicted by the detection model is a top crane, the number of key point coordinates is two, a line segment is obtained by connecting the two key point coordinates, and if the angle between the line segment and the horizontal axis of the image is less than 10°, the top crane is in use, otherwise, the top crane is not in use; if the category of the crane predicted by the detection model is a thick bottom crane, the number of key point coordinates is three, two line segments are obtained by connecting the three key points, if the lengths of the line segments are different, the angle between the two line segments is less than 100°, and the detected thick bottom crane is located at the tail of the loading and unloading vehicle, the thick bottom crane is in use at this time, otherwise, the thick bottom crane is not in use; if the category of the crane predicted by the detection model is a thin bottom crane, the number of key point coordinates is three, two line segments are obtained by connecting the three key points, if the lengths of the line segments are different, the angle between the two line segments is greater than 100°, and the detected thin bottom crane is located at the middle of the loading and unloading vehicle, the thin bottom crane is in use at this time, otherwise, the thin bottom crane is not in use.

2. The method of claim 1, wherein, the process of acquiring crane image historical data and constructing a crane image dataset comprises: acquiring video streams collected by monitoring devices in the loading and unloading vehicle area at different time periods; processing the collected video streams by using a video frame extraction method to obtain crane images; performing primary labeling on the categories of the cranes in the crane images to obtain a first dataset; and performing secondary labeling on the positions of the cranes after the primary labeling on the categories of the cranes is completed to form a second dataset; dividing the first dataset into a first training set, a first validation set and a first test set according to a preset ratio; and dividing the second dataset into a second training set, a second validation set and a second test set according to a preset ratio.

3. The method of claim 2, wherein, The one-time labeling of the crane pipe class in the crane pipe image includes labeling as a top crane pipe, a thick bottom crane pipe and a thin bottom crane pipe according to the installation position and thickness of the crane pipe in the crane pipe image; The secondary labeling of the position of the crane pipe after the one-time labeling of the class of the crane pipe includes labeling the detection position of the top crane pipe, the thick bottom crane pipe and the thin bottom crane pipe as a crane pipe head, a crane pipe tail and a crane pipe corner, respectively.

4. The loading and unloading crane pipe state identification method according to claim 3, characterized in that, The process of training the first detection network model by using the first data set includes training the first detection network model by using a first training set; The process of training the second detection network model by using the second data set includes training the second detection network model by using a second training set.

5. The method of claim 1, wherein, The first detection network model adds an attention mechanism in the yolov5 algorithm; the principle of the attention mechanism is that the first data set obtains a feature map of a preset size after extracting features through the Backbone layer and the Head layer in the yolov5 algorithm; by calculating the mutual relationship between the target pixel points in the image to be identified and all other points in the feature map, the features of the target pixel points are weighted to obtain effective target pixel point features.

6. The method of claim 1, wherein, The second detection network model uses an Hrnet key point detection algorithm; the process of the Hrnet key point detection algorithm detection is to extract the position features corresponding to different detection positions; the corresponding position of the maximum feature value in the position feature map is taken as the key point of the crane pipe head, the crane pipe tail and the crane pipe corner.

7. The method of claim 5, wherein, The process of predicting the class of the crane pipe in the image to be identified by using the trained first detection network model includes inputting the image to be identified into the trained first detection network model, extracting features and predicting the class of the image to be identified by using the trained first detection network model, and obtaining the class of the crane pipe in the image to be identified.

8. A loading and unloading crane arm state recognition system for performing a loading and unloading crane arm state recognition method according to any one of claims 1 to 7, characterized by The system includes a data set construction module, a training module and an identification module; The data set construction module is used to obtain crane pipe image historical data and construct a crane pipe image data set; the data set includes a first data set of crane pipe classes and a second data set of detection positions of different classes of crane pipes; The training module is used to construct a first detection network model for crane pipe class identification; the first detection network model is trained by using the first data set; a second detection network model for crane pipe position identification is constructed, and the second detection network model is trained by using the second data set; The identification module is used to predict the class of the crane pipe in the image to be identified by using the trained first detection network model; The second detection network model is used to predict the position of the crane pipe in the image whose class is predicted; the class prediction result and the position prediction result of the crane pipe are analyzed to obtain the use state of the current crane pipe.

Images