Gas distribution station remaining article detection method based on explosion-proof inspection robot

Abstract

The invention discloses a fuel gas distribution station remaining article detection method based on an anti-explosion inspection robot, and belongs to the field of remaining article detection. The method comprises the steps that an inspection route in a fuel gas distribution station is set, and the anti-explosion inspection robot is started; acquiring video data collected by the explosion-proof inspection robot; processing the video data in real time by using an improved YOLOv8 network to obtain a prediction result of each frame of image; sending the video data, the prediction result of each frame of image and the positioning information of the explosion-proof inspection robot to a user; and judging whether the robot returns to the starting point, if so, ending inspection, and otherwise, continuing inspection. The problem that an existing left article detection method is insufficient in precision is solved.

Description

Technical Field

[0001] The present invention belongs to the field of detection of leftover articles, and in particular relates to a method for detecting leftover articles in a gas distribution station based on an explosion-proof inspection robot. Background Art

[0002] As a core functional module of intelligent video surveillance systems, the detection of abandoned objects plays an important role in the fields of public safety, production safety and security. For example, terrorists leave packages or boxes containing dangerous goods in scenes with large traffic such as shopping malls, stations, schools and subways. If they are not discovered and handled in time, it will lead to irreversible consequences and incalculable losses. As obstacles that affect road traffic, the traffic hazards and security threats caused by road abandoned objects on the road should not be underestimated. The abandoned objects at the industrial production site may seriously affect the production efficiency and operation safety. Therefore, the research on abandoned object detection is of great significance.

[0003] The gas production industry (such as gas distribution stations) has strict requirements on workers' work behavior and on-site item management. Tools left behind by workers (such as explosion-proof wrenches, screwdrivers, etc.) may be accidentally touched or tripped by other people, causing injuries or equipment damage. Small transport vehicles left in the evacuation passage will seriously affect the evacuation of personnel in the event of an accident.

[0004] Most existing technical solutions are based on surveillance video (fixed camera) for detection of leftover items. The main algorithms are background detection and target detection algorithms based on deep learning. However, fixed cameras have the following problems: 1. The camera is often hung at a high place to shoot the target area. The proportion of the remains in the video is small, and the existing algorithm has low detection accuracy for small targets.

[0005] 2. The area monitored by the camera depends on the number of cameras deployed. Summary of the invention

[0006] In view of the above-mentioned deficiencies in the prior art, the present invention provides a method for detecting leftover items at a gas distribution station based on an explosion-proof inspection robot, which solves the problem of insufficient accuracy of the existing leftover item detection methods.

[0007] In order to achieve the above-mentioned invention object, the technical solution adopted by the present invention is: a method for detecting leftover items in a gas distribution station based on an explosion-proof inspection robot, comprising the following steps: S1. Set up the inspection route within the gas distribution station and start the explosion-proof inspection robot; S2, obtaining video data collected by the explosion-proof inspection robot; S3, use the improved YOLOv8 network to process video data in real time and obtain the prediction results of each frame image; S4, sending the video data, the prediction results of each frame image and the explosion-proof inspection robot positioning information to the user; S5. Determine whether the robot has returned to the starting point. If so, end the inspection; otherwise, return to step S2.

[0008] Furthermore, the step S1 is specifically as follows: S101. Use GPS equipment to collect data in the area where the explosion-proof inspection robot can travel in the gas distribution station to obtain GPS data; S102, in the map drawing software, drawing the basic outline of the map according to the GPS data; S103, constructing a visual user map interface according to the basic outline of the map; S104. Mark the inspection route trajectory R on the visual user map interface according to actual needs, obtain the inspection route in the gas distribution station, and start the explosion-proof inspection robot.

[0009] Furthermore, the improved YOLOv8 network in step S3 includes a backbone network layer, a Neck network layer and a Head network layer; the backbone network layer is a mobilenetv4 network; the ConvNext structure and the SPPF structure of the backbone network layer are both connected to the Neck network layer; the Neck network layer includes an upsample upsampling structure and a first Concat structure respectively connected to the SPPF structure, a second Concat structure respectively connected to the ConvNext structure and the upsample upsampling structure, a first IB structure connected to the second Concat structure, a CBS structure respectively connected to the first Concat structure and the first IB structure, and a second IB structure connected to the first Concat structure; the first IB structure and the second IB structure are both connected to the Head network layer; the Head network layer includes a first detection head and a second detection head respectively connected to the first IB structure and the second IB structure.

[0010] Furthermore, the loss function of the improved YOLOv8 network in step S3 is:

[0011] in, To improve the loss function of the YOLOv8 network; It is the area intersection ratio of the predicted box A and the real box B; is the focusing coefficient with a value of [0,1]; is the power regularization term; is the Euclidean distance function; is the center point of the prediction box A; is the center point of the real frame B; is the diagonal length of the minimum bounding box of the prediction box A and B; is the balance parameter; The consistency of the aspect ratio between the predicted box A and the real box B; is the prediction box; is the real frame; is the inverse tangent function; is the ratio of pi; is the width of the prediction box; is the height of the prediction box; is the width of the real frame; is the height of the actual frame.

[0012] Furthermore, the step S3 is specifically as follows: S301, collecting historical inspection data, and using the historical inspection data to train the improved YOLOv8 network to obtain a trained target detection network; S302: Use the trained target detection network to process the video data in real time to obtain the prediction result of each frame of the image:

[0013] in, To predict the results; The trained target detection network; is the image frame; The horizontal coordinate of the center point of the detected object; The ordinate of the center point of the detected object; To detect the width of the object; To detect the height of the object; To detect the category information of the item; A binary function to determine whether an object is detected; Is an empty set.

[0014] The beneficial effects of the present invention are: using the explosion-proof inspection robot, with appropriate route planning, more areas can be monitored; the proportion of leftover items in the video from the perspective of the explosion-proof inspection robot will not be very small and can be easily detected; and high-intensity manual inspections can be replaced. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 The figure is a flow chart of the method of the present invention.

[0016] Figure 2 This is a structural diagram of the improved YOLOv8 network of the present invention.

[0017] Figure 3 Schematic diagram of the categories of residues in an embodiment of the present invention.

[0018] Figure 42 is a system architecture diagram in an embodiment of the present invention.

[0019] Figure 5 Schematic diagram of a user interface in an embodiment of the present invention. DETAILED DESCRIPTION

[0020] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0021] like Figure 1 As shown, in one embodiment of the present invention, a method for detecting leftover items at a gas distribution station based on an explosion-proof inspection robot comprises the following steps: S1. Set up the inspection route within the gas distribution station and start the explosion-proof inspection robot; S2, obtaining video data collected by the explosion-proof inspection robot; S3, use the improved YOLOv8 network to process video data in real time and obtain the prediction results of each frame image; S4, sending the video data, the prediction results of each frame image and the explosion-proof inspection robot positioning information to the user; S5. Determine whether the robot has returned to the starting point. If so, end the inspection; otherwise, return to step S2.

[0022] In this embodiment, the S1 user marks and selects the inspection route and starts the inspection robot, which is specifically implemented as follows: The user marks or selects the route for the robot to inspect the leftover items in the user interface. Specifically, in order to enable the inspection robot invented by this patent to correctly detect the leftover items at a specific location, a GPS (Global Positioning System) positioning method is adopted here. The specific steps are as follows: Data collection: Use GPS equipment to collect data in the area where the map needs to be drawn (the area where the inspection robot can drive). Record the longitude and latitude coordinates and related information received by the GPS equipment.

[0023] Data processing: Import the collected GPS data into mapping software, such as QGIS or ArcGIS. These software can read GPS data files and convert them into elements such as points, lines and areas on the map.

[0024] Map drawing: In map drawing software, the basic outline of the map is drawn according to GPS data. For example, using QGIS, you can add vector layers to draw lines such as roads and rivers, and use point layers to represent locations, landmarks, etc. At the same time, you can add map annotations, legends and other information to enhance the readability of the map.

[0025] Visual interface: Use front-end frameworks such as React, Vue.js and Leafleft map library to build a visual user interface map. Route marking and selection: Users can mark the inspection route trajectory R (p0, p1, p2, ..., pi, p0) on the visual map according to actual needs, or mark it in advance and store it in the database for later use. Pi represents the positioning information (longitude, latitude).

[0026] Real-time position update: During the movement, the robot should obtain its own longitude and latitude position information in real time through the GPS receiver and feed it back to the user.

[0027] Start the robot to inspect along the specified route and run the system.

[0028] The step S1 is specifically as follows: S101. Use GPS equipment to collect data in the area where the explosion-proof inspection robot can travel in the gas distribution station to obtain GPS data; S102, in the map drawing software, drawing the basic outline of the map according to the GPS data; S103, constructing a visual user map interface according to the basic outline of the map; S104. Mark the inspection route trajectory R on the visual user map interface according to actual needs, obtain the inspection route in the gas distribution station, and start the explosion-proof inspection robot.

[0029] like Figure 2As shown, the improved YOLOv8 network in step S3 includes a backbone network layer, a Neck network layer and a Head network layer; the backbone network layer is a mobilenetv4 network; the ConvNext structure and the SPPF structure of the backbone network layer are both connected to the Neck network layer; the Neck network layer includes an upsample upsampling structure and a first Concat structure respectively connected to the SPPF structure, a second Concat structure respectively connected to the ConvNext structure and the upsample upsampling structure, a first IB structure connected to the second Concat structure, a CBS structure respectively connected to the first Concat structure and the first IB structure, and a second IB structure connected to the first Concat structure; the first IB structure and the second IB structure are both connected to the Head network layer; the Head network layer includes a first detection head and a second detection head respectively connected to the first IB structure and the second IB structure.

[0030] In this embodiment, the improved YOLOv8 network (M4-YOLO) is used to detect the target in the video frame to obtain the category and bounding box coordinates of the target, which is specifically implemented as follows: Collect previous inspection video data and retain the data frames containing the leftover items. The data set is as follows: Figure 3 As shown, the annotation tools are used to annotate the remaining items in the data. The annotation information includes the location information (cx, cy, w, h) of the items in the picture and the category information L, where the category L of the items includes tool boxes and tools such as screwdrivers and wrenches carried by people. The location information (cx, cy) represents the coordinates of the center point of the item, and (w, h) represents the width and height of the item.

[0031] Considering that the explosion-proof robot is a mobile device and its computing resources and storage capacity are limited, ① the backbone network of YOLOv8 is replaced with the mobilenetv4 network proposed by Google, which is an efficient architecture suitable for mobile devices and achieves optimal performance on multiple hardware platforms. In addition, the explosion-proof robot can zoom in during inspections, or when the distance to the abandoned items is close, the proportion of the abandoned items in the video will not be very small, so the small target detection layer can be discarded. The specific method is ② using the IB module in mobilenetv4, reconstructing Neck according to the idea of ​​PAFPN, and only incorporating two layers of features of different scales, and named it IBNeck2. The detection head is changed from three to two, which will reduce the model calculation amount, the number of parameters, and improve the detection speed. The improved model is M4-YOLO, such as Figure 2 shown.

[0032] The loss function of the improved YOLOv8 network in step S3 is:

[0033] Among them, in order to improve the bounding box regression accuracy, To improve the loss function of the YOLOv8 network; It is the area intersection ratio of the predicted box A and the real box B; is the focusing coefficient with a value of [0,1]; is the power regularization term; is the Euclidean distance function; is the center point of the prediction box A; is the center point of the real frame B; is the diagonal length of the minimum bounding box of the prediction box A and B; To balance the parameters, the larger the IoU between the predicted box A and the real box B, the larger the value; The consistency of the aspect ratio between the predicted box A and the real box B; is the prediction box; is the real frame; is the inverse tangent function; is the ratio of pi; is the width of the prediction box; is the height of the prediction box; is the width of the real frame; is the height of the actual frame.

[0034] in: The introduction of is to make the model pay more attention to high-quality samples (i.e. samples with higher IoU values) during training, and suppress the contribution of low-quality samples (i.e. samples with lower IoU values) to the loss function. This strategy helps the model better learn the characteristics of the target on a limited data set and further improve the detection performance. The introduction of improves the Bbox regression accuracy by adaptively reweighting the loss and gradient of high and low IoU targets. This design allows the model to pay more attention to samples that have an important impact on performance improvement during training, thereby further improving the performance of object detection.

[0035] The step S3 is specifically as follows: S301, collecting historical inspection data, and using the historical inspection data to train the improved YOLOv8 network to obtain a trained target detection network; S302: Use the trained target detection network to process the video data in real time to obtain the prediction result of each frame of the image:

[0036] in, To predict the results; The trained target detection network; is the image frame; The horizontal coordinate of the center point of the detected object; The ordinate of the center point of the detected object; To detect the width of the object; To detect the height of the object; To detect the category information of the item; A binary function to determine whether an object is detected; Is an empty set.

[0037] In this embodiment, the PyTorch deep learning framework is used to build the M4-YOLO model, and the model is trained with the collected data. The initial learning rate is set to 0.02, the weight decay coefficient is set to 0.0005, the image input size is 640×640, the batch size is 8, and the training ends after 300 rounds and is exported. It should be noted that the learning framework can also be TensorFlow, paddle, etc.

[0038] Load the trained M4-YOLO model algorithm and process each frame of data in the video , the calculation formula is as follows:

[0039] Among them, the output is the detected abandoned items target information sequence O =[( xx , cy , w,h,L ),…], where the legacy categories are L , the location information is ( xx , cy , w,h ). If no left-behind object is detected, then O =[ ].

[0040] Send each processed frame of image and positioning information to the user. The specific implementation is as follows: Each frame of data will be processed I , possible relic sequences O and robot positioning information p (longitude, latitude) are packaged into data packets and sent to the user end in real time. The collected information is also stored.

[0041] When you find something left behind (when O is not an empty set), using the category L and location information ( xx , cy , w,h), mark the leftovers on the original image, and use voice prompts and other functions to remind users, and use real-time positioning information to mark the road section where the leftovers are found so that users can clean them up later. At the same time, a road section containing leftovers can be represented as a sequence Z=[( I t , p t , O t ), …, I n , p n , O n )], indicating continuous t arrive n If a relic is found in the frame image, return the frame image I , the legacy sequence O and robot positioning information p .

[0042] Based on the system constructed by the present invention, Figure 4 This is an architecture diagram of the left-behind object detection system for the inspection robot provided by the present invention.

[0043] Specifically, the left-behind object detection system for patrol robots provided by the present invention includes a data layer, a visual computing and information transmission layer, and a user monitoring interface.

[0044] The data layer is used to collect video image data, collect robot positioning information through GPS, and use a database to store historical inspection videos, robot positioning information, and information such as leftover items, including: The data acquisition and preprocessing module uses the high-definition camera of the inspection robot to collect video image data and converts the collected data into the format required by the visual computing and information transmission layer.

[0045] The data storage module stores historical inspection videos, leftover items, robot inspection trajectories and inspection maps, etc.

[0046] Among them, the visual computing and information transmission layer uses the improved YOLOv8 algorithm (M4-YOLO) to process the image data collected by the data layer, detect and mark possible abandoned items, and send the above information and robot positioning information to the user monitoring interface.

[0047] Among them, the user monitoring interface receives the data sent by the robot and makes this information into a visual interface, including real-time monitoring images, real-time inspection tracks of the robot and information prompts of left-behind items.

[0048] Specifically, Figure 5 A simplified diagram of the user monitoring interface in the system provided by the present invention is shown.

[0049] The user monitoring interface specifically includes: The real-time monitoring video and abandoned items alarm prompt module receives the video data processed by the visual computing and information transmission layer and displays it to the user for real-time review, and prompts the user whether there are any abandoned items. If so, the location information of the abandoned items is marked in the video image.

[0050] The inspection trajectory visualization module, that is, the inspection map and the visualization of the real-time trajectory changes of the inspection robot on the map, makes it convenient for users to view the inspection progress.

[0051] Route planning module, which is used for marking selection for each inspection route.

Claims

1. A method for detecting leftover items at a gas distribution station based on an explosion-proof inspection robot, characterized in that: The following steps are involved: S1. Set up the inspection route within the gas distribution station and start the explosion-proof inspection robot; S2, obtaining video data collected by the explosion-proof inspection robot; S3, use the improved YOLOv8 network to process video data in real time and obtain the prediction results of each frame image; S4, sending the video data, the prediction results of each frame image and the explosion-proof inspection robot positioning information to the user; S5. Determine whether the robot has returned to the starting point. If so, end the inspection; otherwise, return to step S2.

2. The method for detecting leftover items at a gas distribution station based on an explosion-proof inspection robot according to claim 1 is characterized in that: The step S1 is specifically as follows: S101. Use GPS equipment to collect data in the area where the explosion-proof inspection robot can travel in the gas distribution station to obtain GPS data; S102, in the map drawing software, drawing the basic outline of the map according to the GPS data; S103, constructing a visual user map interface according to the basic outline of the map; S104. Mark the inspection route trajectory R on the visual user map interface according to actual needs, obtain the inspection route in the gas distribution station, and start the explosion-proof inspection robot.

3. The method for detecting leftover items at a gas distribution station based on an explosion-proof inspection robot according to claim 1 is characterized in that: The improved YOLOv8 network in step S3 includes a backbone network layer, a Neck network layer and a Head network layer; the backbone network layer is a mobilenetv4 network; the ConvNext structure and the SPPF structure of the backbone network layer are both connected to the Neck network layer; the Neck network layer includes an upsample upsampling structure and a first Concat structure respectively connected to the SPPF structure, a second Concat structure respectively connected to the ConvNext structure and the upsample upsampling structure, a first IB structure connected to the second Concat structure, a CBS structure respectively connected to the first Concat structure and the first IB structure, and a second IB structure connected to the first Concat structure; the first IB structure and the second IB structure are both connected to the Head network layer; the Head network layer includes a first detection head and a second detection head respectively connected to the first IB structure and the second IB structure.

4. The method for detecting leftover items at a gas distribution station based on an explosion-proof inspection robot according to claim 1 is characterized in that: The loss function of the improved YOLOv8 network in step S3 is: in, To improve the loss function of the YOLOv8 network; It is the area intersection ratio of the predicted box A and the real box B; is the focusing coefficient with a value of [0,1]; is the power regularization term; is the Euclidean distance function; is the center point of the prediction box A; is the center point of the real frame B; is the diagonal length of the minimum bounding box of the prediction box A and B; is the balance parameter; The consistency of the aspect ratio between the predicted box A and the real box B; is the prediction box; is the real frame; is the inverse tangent function; is the ratio of pi; is the width of the prediction box; is the height of the prediction box; is the width of the real frame; is the height of the actual frame.

5. The method for detecting leftover items at a gas distribution station based on an explosion-proof inspection robot according to claim 1 is characterized in that: The step S3 is specifically as follows: S301, collecting historical inspection data, and using the historical inspection data to train the improved YOLOv8 network to obtain a trained target detection network; S302: Use the trained target detection network to process the video data in real time to obtain the prediction result of each frame of the image: in, To predict the results; The trained target detection network; is the image frame; The horizontal coordinate of the center point of the detected object; The ordinate of the center point of the detected object; To detect the width of the object; To detect the height of the object; To detect the category information of the item; A binary function to determine whether an object is detected; Is an empty set.

Images