Fuel tank safety monitoring method, system, electronic device and storage medium

Abstract

The present invention provides a fuel tank safety monitoring method, system, electronic device and storage medium. The method includes performing suspicious target detection on the preprocessed monitoring video; extracting key frames from the monitoring video and tracking the pedestrian position according to the change in the pedestrian position between the current frame and the previous frame; distinguishing daily scenes, refueling scenes, and abnormal behavior scenes of personnel in the fuel tank area. For the abnormal behavior scenes of personnel in the fuel tank area, based on the suspicious target category, the spatial position of the suspicious target, the action category performed by the pedestrian, and the spatial position where the pedestrian performs the action, a person holding a tubular object is defined as a dangerous object; based on the ID of the dangerous object and the spatial coordinates of the fuel tank, the spatial relationship between the dangerous object and the fuel tank is judged. If the proximity degree of the spatial relationship exceeds a preset threshold, an alarm decision is made. The present invention can make an alarm decision based on the spatial relationship between the dangerous object and the fuel tank, thereby avoiding misjudgment and missed judgment during alarm.

Description

Technical Field

[0001] The present invention relates to the technical field of fuel tank monitoring, and particularly to a fuel tank safety monitoring method, system, electronic device and storage medium. Background Art

[0002] The transformation of traditional industries empowered by artificial intelligence is a current major trend. In the 1980s of the last century, due to the limitations of the computing power of computing devices and its own complex parameters, deep learning models could never surpass human capabilities in certain aspects. However, in the 21st century with the booming development of technology, thanks to the emergence of high-performance computing devices, various advanced deep learning models have emerged one after another. In the field of computer vision, the object recognition ability of visual models that surpasses humans has brought new solutions to existing problems in the industry. Thus, it has become a reality to apply excellent computer vision models to fuel tank anti-theft.

[0003] Among the existing technologies, the existing fuel tank anti-theft measures can be divided into three categories, namely, methods based on physical modification, physical sensors, and deep learning. Among them, for the method based on physical modification, its core lies in increasing the cost for criminals to damage the fuel tank through physical means. This method involves weighing the strength and convenience of the fuel tank. Increasing the structural strength of the fuel tank may make operations such as refueling and maintenance more complex or time-consuming. When considering convenience, the safety of the fuel tank cannot be guaranteed. The method based on physical sensors focuses on various physical signals that appear in the crime scene, and is committed to converting physical signals into electrical signals by using specific physical effects or characteristics. After the electrical signals are digitally processed and analyzed, they are alarmed or prompted in the form of numbers, graphics, and sounds, so as to achieve the effect of anti-theft. Due to the lack of modeling of the characteristics of abnormal behaviors of personnel in the fuel tank area itself and the neglect of the internal connection between criminals and objects in this scenario in this type of technology, physical signals generated by non-criminal activities are converted into electrical signals, resulting in misjudgments and missed judgments in the alarm decisions made by this type of technology. In order to overcome the drawbacks of the above traditional methods, influenced by the big data era, a method based on machine vision to understand images has emerged. Currently, such methods stay at extracting static image features and ignore the changes in abnormal behaviors of personnel in the fuel tank area over time, resulting in the inability to objectively understand the real scene and further leading to misjudgments. Summary of the Invention

[0004] Based on this, the purpose of the present invention is to provide a fuel tank safety monitoring method, system, electronic device and storage medium to solve the deficiencies in the above existing technologies.

[0005] In a first aspect, the present invention provides a fuel tank safety monitoring method, and the method includes:

[0006] Receive the surveillance video of the fuel tank area and preprocess the surveillance video to obtain the daily scene;

[0007] Based on the target detection model, perform suspicious target detection on the preprocessed surveillance video to record the category of the suspicious target and the spatial position of the suspicious target;

[0008] Based on the spatio-temporal behavior detection model, perform behavior detection on the preprocessed surveillance video to record the action category performed by the pedestrian and the spatial position where the pedestrian performs the action;

[0009] Extract the key frames in the surveillance video and perform preprocessing. Track the position of the pedestrian according to the change in the position of the pedestrian in the current frame and the previous frame in the preprocessed key frames;

[0010] Distinguish the daily scene, the refueling scene, and the abnormal behavior scene of personnel in the fuel tank area. Based on the abnormal behavior scene of personnel in the fuel tank area, the category of the suspicious target, the spatial position of the suspicious target, the action category performed by the pedestrian, and the spatial position where the pedestrian performs the action, define the suspicious target holding a tubular object as a dangerous object;

[0011] Extract the ID of the dangerous object. Based on the ID of the dangerous object and the spatial coordinates of the fuel tank, judge the spatial relationship between the dangerous object and the fuel tank. If the proximity of the spatial relationship exceeds the preset threshold, make an alarm decision.

[0012] Compared with the prior art, the beneficial effects of the present invention are as follows: By performing suspicious target detection and behavior detection, it is possible to record the category of the suspicious target, the spatial position of the suspicious target, the action category performed by the pedestrian, and the spatial position where the pedestrian performs the action, and it is possible to define a dangerous object. Based on the spatial relationship between the dangerous object and the fuel tank, an alarm decision can be made, thereby avoiding misjudgment and missed judgment during alarm.

[0013] Further, the steps of receiving the surveillance video of the fuel tank area and preprocessing the surveillance video include:

[0014] Collect the surveillance video of the fuel tank area based on a camera and compress the surveillance video into a video data stream by encoding in the h264 standard;

[0015] Transmit the video data stream to a PC and transcode it into the mp4 format and cache it in the buffer of the PC.

[0016] Further, the steps of performing suspicious target detection on the preprocessed surveillance video based on the target detection model to record the category of the suspicious target and the spatial position of the suspicious target include:

[0017] Loop to extract images from the surveillance video, adjust the size of the images, and convert the images into a two-dimensional numpy array format;

[0018] Load the pre-trained object detection model, and input the image in the two-dimensional numpy array format into the pre-trained object detection model;

[0019] Based on the pre-trained object detection model, perform spatial feature extraction, enhanced spatial feature extraction, feature point anchor box classification and regression on the image in the two-dimensional numpy array format in sequence, so as to obtain the category of suspicious objects and the spatial positions of the suspicious objects in the image.

[0020] Further, the step of performing behavior detection on the preprocessed surveillance video based on the spatio-temporal behavior detection model to record the action categories performed by pedestrians and the spatial positions where the pedestrians perform includes:

[0021] Loop to extract images from the surveillance video, adjust the size of the images, and convert the overlapping images into a three-dimensional numpy array format;

[0022] Load the pre-trained spatio-temporal behavior detection model, and input the image in the three-dimensional numpy array format into the pre-trained spatio-temporal behavior detection model;

[0023] Based on the pre-trained spatio-temporal behavior detection model, perform spatial feature extraction, temporal feature extraction, spatio-temporal feature fusion, and feature point anchor box classification and regression on the image in the three-dimensional numpy array format in sequence to obtain the categories and spatial positions of pedestrian actions in the image sequence.

[0024] Further, the step of extracting key frames from the surveillance video, performing preprocessing, and tracking the positions of pedestrians according to the change in the positions of pedestrians in the current frame and the previous frame in the preprocessed key frames includes:

[0025] Loop to extract images from the surveillance video, adjust the size of the images, and convert the images into a two-dimensional numpy array format;

[0026] Judge whether the image converted into the two-dimensional numpy array format is the first picture;

[0027] If so, assign IDs to all pedestrians in the first picture through the object detection algorithm, and record the spatial features of the pedestrians.

[0028] Further, after the step of judging whether the image converted into the two-dimensional numpy array format is the first picture, the method further includes:

[0029] If the image converted to a two-dimensional numpy array format is not the first picture, the Hungarian algorithm is used to match and the Kalman filter algorithm is used to predict the position of the pedestrian.

[0030] Further, the step of determining the spatial relationship between the dangerous object and the fuel tank based on the ID of the dangerous object and the spatial coordinates of the fuel tank, and making an alarm decision if the proximity of the spatial relationship exceeds a preset threshold includes:

[0031] Based on the spatial coordinates of the fuel tank, the fuel tank area is set as a sensitive area;

[0032] Monitor the spatial coordinate points of the dangerous object in the sensitive area;

[0033] Based on the surveillance video, calculate the residence time of the dangerous object in the sensitive area, and determine whether the residence time exceeds the residence time threshold;

[0034] If so, make an alarm decision.

[0035] In a second aspect, the present invention also provides a fuel tank safety monitoring system, the system includes:

[0036] A receiving module, configured to receive a surveillance video of the fuel tank area, and preprocess the surveillance video to obtain a daily scene;

[0037] A first detection module, configured to perform suspicious target detection on the preprocessed surveillance video based on a target detection model to record the category of the suspicious target and the spatial position of the suspicious target;

[0038] A second detection module, configured to perform behavior detection on the preprocessed surveillance video based on a spatio-temporal behavior detection model to record the action category performed by the pedestrian and the spatial position where the pedestrian performs the action;

[0039] An extraction module, configured to extract key frames from the surveillance video, and perform preprocessing, and track the position of the pedestrian according to the change in the position of the pedestrian in the current frame and the previous frame in the preprocessed key frames;

[0040] A definition module, configured to distinguish the daily scene, the refueling scene, and the abnormal behavior scene of personnel in the fuel tank area, and define a suspicious target holding a tubular object as a dangerous object based on the abnormal behavior scene of personnel in the fuel tank area, the category of the suspicious target, the spatial position of the suspicious target, the action category performed by the pedestrian, and the spatial position where the pedestrian performs the action;

[0041] A judgment module, configured to extract the ID of the dangerous object, determine the spatial relationship between the dangerous object and the fuel tank based on the ID of the dangerous object and the spatial coordinates of the fuel tank, and make an alarm decision if the proximity of the spatial relationship exceeds a preset threshold.

[0042] In a third aspect, the present invention further provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where when the processor executes the computer program, the above-mentioned fuel tank safety monitoring method is implemented.

[0043] In a fourth aspect, the present invention further provides a storage medium, on which a computer program is stored, and when the program is executed by a processor, the above-mentioned fuel tank safety monitoring method is implemented. Description of the Drawings

[0044] Figure 1 It is a flowchart of the fuel tank safety monitoring method in the first embodiment of the present invention;

[0045] Figure 2 It is a structural block diagram of the fuel tank safety monitoring system in the second embodiment of the present invention;

[0046] Figure 3 It is a structural block diagram of the electronic device in the third embodiment of the present invention.

[0047] Main Element Symbol Description:

[0048] 10. Receiving module; 20. First detection module; 30. Second detection module; 40. Extraction module; 50. Definition module; 60. Judgment module;

[0049] 70. Bus; 71. Processor; 72. Memory; 73. Communication interface.

[0050] The following specific embodiments will further illustrate the present invention in conjunction with the above-mentioned drawings. Specific Embodiments

[0051] To facilitate the understanding of the present invention, the present invention will be described more comprehensively below with reference to the relevant drawings. Several embodiments of the present invention are given in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive.

[0052] It should be noted that when an element is referred to as being "fixed to" another element, it can be directly on the other element or there can also be an intermediate element. When an element is considered to be "connected to" another element, it can be directly connected to the other element or there may be an intermediate element at the same time. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are for illustrative purposes only.

[0053] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which this invention belongs. The terms used herein in the description of the present invention are for the purpose of describing specific embodiments only and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0054] Embodiment 1

[0055] Please refer to Figure 1 , which shows the fuel tank safety monitoring method in the first embodiment of the present invention. The method includes steps S1 to S6:

[0056] S1. Receive the monitoring video of the fuel tank area and preprocess the monitoring video to obtain the daily scene.

[0057] It should be explained that a suspicious target holding a tubular object is a suspicious target, that is, a suspicious pedestrian, a pedestrian holding a tubular object. The daily scene includes a refueling scene, a refueling platform, and a fuel gun.

[0058] Specifically, step S1 includes steps S11 to S12:

[0059] S11. Collect the monitoring video of the fuel tank area based on the camera and compress the monitoring video into a video data stream encoded in the h264 standard.

[0060] S12. Transmit the video data stream to the PC and transcode it into the mp4 format and cache it in the buffer of the PC.

[0061] It can be understood that after the monitoring video of the fuel tank area is collected by the camera, it is compressed into a video data stream encoded in the h264 standard. The PC establishes a connection with the monitor at the truck rearview mirror through the TCP protocol, and the monitoring video data stream is transmitted through the established channel at both ends. The PC receives the video data stream from the channel, transcodes it into the mp4 format of the video and caches it in the memory for subsequent steps to extract key frames.

[0062] S2. Detect suspicious targets in the preprocessed monitoring video based on the target detection model to record the categories of suspicious targets and the spatial positions of suspicious targets.

[0063] Specifically, step S2 includes steps S21 to S23:

[0064] S21, repeatedly extract images from the monitoring video, adjust the size of the images, and convert the images into the two-dimensional numpy array format;

[0065] It should be explained that the images are repeatedly taken out from the monitoring video in the memory. In this embodiment, when adjusting the size of the images, the adjustment is made according to the required size of the object detection pre-trained model.

[0066] It is worth noting that the monitoring video of the fuel tank area is read from the PC buffer, key frames are extracted, and the key frames are pre-processed, and then passed into the object detection model for suspicious object detection, and the categories and spatial positions of the suspicious objects are recorded. Specifically, first, the parameters of the pre-trained model are loaded into the memory, and then the images in the numpy array format are passed into the model.

[0067] S22, load the object detection pre-trained model, and pass the images in the two-dimensional numpy array format into the object detection pre-trained model;

[0068] S23, based on the object detection pre-trained model, sequentially perform spatial feature extraction, enhanced spatial feature extraction, and feature point anchor box classification and regression on the images in the two-dimensional numpy array format to obtain the categories of the suspicious objects in the images and the spatial positions of the suspicious objects;

[0069] It should be explained that for the abnormal behavior scenario of personnel in the fuel tank area, tubular objects and pedestrians are marked as suspicious objects, and the fuel tank is marked as a key object. In order to distinguish the refueling scenario, the fuel nozzle and the gas station platform are marked as safe objects.

[0070] It is worth noting that in this embodiment, the object detection model is the YOLOv7 model. During the training process of the YOLOv7 model, first, the hyperparameters and configurations related to training are set and saved. Next, the directories for storing the model weights and results are defined. The model initialization includes loading from the pre-trained weights or creating a new model, and classifying different model parameters for optimization. The settings of the optimizer are adjusted according to the hyperparameters, including the learning rate and weight decay. During the training process, the model iterates over the training dataset at each epoch, uses a gradient scaler to handle the gradients, and applies EMA (Exponential Moving Average) to improve the generalization ability of the model. At the end of each epoch, the learning rate scheduler is updated, the model performance is evaluated, and the best model is saved. The entire training process focuses on efficiency and accuracy to ensure that the model performs well in handling complex object detection tasks. The specific parameter settings are shown in Table 1:

[0071] Table 1

[0072]

[0073] After 300 epochs of training, a target detection model suitable for the scenario of abnormal behavior of personnel in the fuel tank area is obtained, and the mAP metrics for 13 types of objects are as shown in Table 2:

[0074] Table 2

[0075]

[0076] S3. Based on the spatio-temporal behavior detection model, perform behavior detection on the preprocessed monitoring video to record the action categories performed by pedestrians and the spatial positions where the pedestrians perform.

[0077] Specifically, the step S3 includes steps S31 to S33:

[0078] S31. Extract images from the monitoring video in a loop, adjust the size of the images, and convert the overlapping images into a three-dimensional numpy array format.

[0079] It should be explained that the images are taken from the monitoring video in the memory in a loop, the current image is retained, the size of the taken images is adjusted according to the size requirements of the images during the pre-training of the spatio-temporal behavior detection pre-trained model, and 16 images are overlapped and converted into a 3D numpy array format.

[0080] It is worth noting that the monitoring video of the fuel tank area is read from the PC buffer, the key frame sequence is extracted, the key frames are preprocessed, and then passed into the spatio-temporal behavior detection model for action detection, and the categories and spatial positions of the actions performed by pedestrians are recorded.

[0081] S32. Load the spatio-temporal behavior detection pre-trained model, and pass the images in the three-dimensional numpy array format into the spatio-temporal behavior detection pre-trained model.

[0082] S33. Based on the spatio-temporal behavior detection pre-trained model, perform spatial feature extraction, temporal feature extraction, spatio-temporal feature fusion, and feature point anchor box classification and regression on the images in the three-dimensional numpy array format in sequence to obtain the categories and spatial positions of the actions of pedestrians in the image sequence.

[0083] It should be noted that for the spatio-temporal behavior detection pre-training model, during the training process, AdamW is selected as the optimizer, the initially set learning rate is 0.0001, and the weight decay is set to 0.0005. The batch size we set is 8, and gradient accumulation is performed 16 times. During the training of the abnormal behavior monitoring video dataset of personnel in the fuel tank area, the YOWOv2 model undergoes 9 epochs, and the learning rate is adjusted to half of the original value in the 3rd, 4th, 5th, and 6th epochs. Usually, the size of the input frame is reshaped to 224×224, and the mAP performance shown in Table 3 is obtained for the three types of atomic actions.

[0084] Table 3

[0085]

[0086] S4. Extract the key frames from the monitoring video and perform preprocessing. Track the positions of pedestrians according to the changes in the positions of pedestrians in the current frame and the previous frame in the preprocessed key frames.

[0087] Specifically, step S4 includes steps S41 to S44:

[0088] S41. Continuously extract images from the monitoring video, adjust the sizes of the images, and convert the images into the format of two-dimensional numpy arrays.

[0089] S42. Determine whether the image converted into the format of two-dimensional numpy arrays is the first picture.

[0090] S43. If so, assign IDs to all pedestrians in the first picture through the object detection algorithm and record the spatial features of the pedestrians.

[0091] S44. If the image converted into the format of two-dimensional numpy arrays is not the first picture, use the Hungarian algorithm to match and the Kalman filter algorithm to predict the positions of the pedestrians.

[0092] It should be explained that if the current picture is the first picture, directly assign an ID to each pedestrian in the picture through the object detection algorithm and predict the position of the pedestrian in the next frame based on the Kalman filter algorithm. After assigning IDs to the pedestrians in the current picture, record the spatial features of the pedestrians so that the original IDs can be assigned to the pedestrians in the picture again through the recorded spatial features after occlusion. If it is not the first picture, use the Hungarian algorithm to match the positions of the pedestrians predicted by the Kalman filter algorithm and the positions of the pedestrians predicted by the object detection to make the assignment of pedestrian IDs more accurate.

[0093] It should be noted that the monitoring video of the fuel tank area is read from the PC buffer, key frames are extracted, and the key frames are preprocessed. According to the change in the position of pedestrians in the current frame and the previous frame, they are tracked. Specifically, the original image, the category of the object in the image, and its coordinates are passed into the object tracking model. The object tracking model has three functions: recording the spatial features of the object in the image for a period of time, recording the ID of the object in the image, and predicting the position of the object in the next frame. For the incoming target information, it is compared with the prediction result of the target in the previous frame. If the comparison error is within the set range, the ID of the target that matches it is assigned to the current target. When the target in the image is occluded for a period of time and appears again, in order to identify the target, the spatial features of the target are compared with the features of the targets that have appeared in the model for a period of time. When the error is within the set threshold, the corresponding ID is assigned to the object.

[0094] S5. Distinguish the daily scene, the refueling scene, and the abnormal behavior scene of personnel in the fuel tank area. Based on the abnormal behavior scene of personnel in the fuel tank area, the suspicious target category, the spatial position of the suspicious target, the action category performed by the pedestrian, and the spatial position where the pedestrian performs the action, define the suspicious target holding a tubular object as a dangerous object;

[0095] It should be explained that according to the coordinates of the tubular object and the holding action, find the nearest tubular object within a certain range centered on the holding action. If there is a nearest tubular object and there is no fuel nozzle and fuel station platform in the scene, then determine the pedestrian performing the holding action as a dangerous object.

[0096] S6. Extract the ID of the dangerous object. Based on the ID of the dangerous object and the spatial coordinates of the fuel tank, judge the spatial relationship between the dangerous object and the fuel tank. If the proximity of the spatial relationship exceeds the preset threshold, make an alarm decision;

[0097] It should be explained that for the determination of the threat level of the dangerous object to the fuel tank, according to the fuel tank coordinates detected in the image, move the left vertex of the fuel tank coordinates in the negative X-axis direction by the width occupied by the pedestrian in the image. Use this point as the left vertex to draw a parallelogram with the upper base being the width occupied by the pedestrian in the image, the height being twice the height occupied by the pedestrian in the image, and the included angle between the upper base and the hypotenuse being 60° as the sensitive area. For dangerous personnel, use the cross product method to check whether the midpoint of their coordinates is in the sensitive area. If it is, count 1 time for 1 frame of the image. Calculate the time they stay in the sensitive area according to the number of frames of the video. When this time exceeds the set threshold, make an alarm decision.

[0098] Specifically, step S6 includes steps S61 to S64:

[0099] S61, set the fuel tank area as a sensitive area based on the spatial coordinates of the fuel tank;

[0100] S62, monitor the spatial coordinate points of the dangerous object in the sensitive area;

[0101] S63, calculate the residence time of the dangerous object in the sensitive area based on the monitoring video, and determine whether the residence time exceeds the residence time threshold;

[0102] S64, if so, make an alarm decision;

[0103] It can be understood that according to the spatial coordinates of the fuel tank, an area near it is set as a sensitive area. Monitor the spatial coordinate points of the dangerous personnel existing in each frame, and calculate the residence time of the dangerous personnel in the sensitive area based on the FPS of the video and the number of frames in which the dangerous personnel appear in the sensitive area. When the set residence time threshold is reached, make an alarm decision.

[0104] In addition, it is worth noting that in the specific implementation, in terms of hardware, an intel i5 13400f processor, 32G of memory, and an RTX4090 are used. In terms of software, ubantu-22.04LTS, python-3.10, cuda-11.8, pytorch-2.0.0, torchvision-0.15.0, torchaudio-2.0.0, and pycharm2021 are used.

[0105] In summary, the fuel tank safety monitoring method in the above embodiments of the present invention can record the category of the suspicious target, the spatial position of the suspicious target, the action category performed by the pedestrian, and the spatial position where the pedestrian performs through suspicious target detection and behavior detection, and can define the dangerous object. Through the spatial relationship between the dangerous object and the fuel tank, an alarm decision can be made, thereby avoiding misjudgment and missed judgment during alarm and effectively preventing the fuel tank from being stolen.

[0106] Embodiment 2

[0107] Please refer to Figure 2 , which shows the fuel tank safety monitoring system in the second embodiment of the present invention. The system includes:

[0108] A receiving module 10, configured to receive the monitoring video of the fuel tank area and preprocess the monitoring video to obtain a daily scene;

[0109] A first detection module 20, configured to perform suspicious target detection on the preprocessed monitoring video based on a target detection model to record the category of the suspicious target and the spatial position of the suspicious target;

[0110] The second detection module 30 is used to perform behavior detection on the preprocessed monitoring video based on a spatio-temporal behavior detection model to record the action categories performed by pedestrians and the spatial positions where the pedestrians perform the actions;

[0111] The extraction module 40 is used to extract key frames from the monitoring video, perform preprocessing, and track the pedestrian positions according to the changes in pedestrian positions between the current frame and the previous frame in the preprocessed key frames;

[0112] The definition module 50 is used to distinguish the daily scene, the refueling scene, and the abnormal behavior scene of personnel in the fuel tank area. Based on the abnormal behavior scene of personnel in the fuel tank area, the suspicious target categories, the spatial positions of the suspicious targets, the action categories performed by the pedestrians, and the spatial positions where the pedestrians perform the actions, a suspicious target holding a tubular object is defined as a dangerous object;

[0113] The judgment module 60 is used to extract the ID of the dangerous object, judge the spatial relationship between the dangerous object and the fuel tank based on the ID of the dangerous object and the spatial coordinates of the fuel tank. If the proximity of the spatial relationship exceeds a preset threshold, an alarm decision is made.

[0114] In some alternative embodiments, the receiving module 10 includes:

[0115] The acquisition unit is used to acquire the monitoring video of the fuel tank area based on a camera and compress the monitoring video into a video data stream encoded in the h264 standard;

[0116] The transmission unit is used to transmit the video data stream to a PC and transcoded into the mp4 format and cached in the buffer of the PC.

[0117] In some alternative embodiments, the first detection module 20 includes:

[0118] The first extraction unit is used to cyclically extract images from the monitoring video, adjust the sizes of the images, and convert the images into the two-dimensional numpy array format;

[0119] The first loading unit is used to load a target detection pre-trained model and input the images in the two-dimensional numpy array format into the target detection pre-trained model;

[0120] The first feature extraction unit is used to sequentially perform spatial feature extraction, enhanced spatial feature extraction, feature point anchor box classification, and regression on the images in the two-dimensional numpy array format based on the target detection pre-trained model to obtain the categories of the suspicious targets in the images and the spatial positions of the suspicious targets.

[0121] In some alternative embodiments, the second detection module 30 includes:

[0122] A second extraction unit, configured to cyclically extract images from the monitoring video, adjust the size of the images, and convert the overlapped images into a three-dimensional numpy array format;

[0123] A second loading unit, configured to load a pre-trained model for spatio-temporal behavior detection and input the images in the three-dimensional numpy array format into the pre-trained model for spatio-temporal behavior detection;

[0124] A second feature extraction unit, configured to sequentially perform spatial feature extraction, temporal feature extraction, spatio-temporal feature fusion, and feature point anchor box classification and regression on the images in the three-dimensional numpy array format based on the pre-trained model for spatio-temporal behavior detection, so as to obtain the categories and spatial positions of pedestrian actions in the image sequence.

[0125] In some alternative embodiments, the extraction module 40 includes:

[0126] A third extraction unit, configured to cyclically extract images from the monitoring video, adjust the size of the images, and convert the images into a two-dimensional numpy array format;

[0127] A judgment unit, configured to judge whether the image converted into the two-dimensional numpy array format is the first picture;

[0128] An allocation unit, configured to judge that if the image converted into the two-dimensional numpy array format is the first picture, then assign IDs to all pedestrians in the first picture through an object detection algorithm and record the spatial features of the pedestrians.

[0129] A prediction unit, configured to judge that if the image converted into the two-dimensional numpy array format is not the first picture, then adopt the Hungarian algorithm to match and the Kalman filter algorithm to predict the positions of the pedestrians.

[0130] In some alternative embodiments, the judgment module 60 includes:

[0131] A setting unit, configured to set the fuel tank area as a sensitive area based on the spatial coordinates of the fuel tank;

[0132] A monitoring unit, configured to monitor the spatial coordinate points of the dangerous object in the sensitive area;

[0133] A calculation and judgment unit, configured to calculate the residence time of the dangerous object in the sensitive area based on the monitoring video and judge whether the residence time exceeds a residence time threshold;

[0134] An alarm unit, configured to judge that if the residence time exceeds the residence time threshold, then make an alarm decision.

[0135] The functions or operation steps realized when the above-mentioned modules and units are executed are substantially the same as those in the above method embodiment, and will not be elaborated here.

[0136] The fuel tank safety monitoring system provided by the embodiment of the present invention has the same implementation principle and technical effects as those in the foregoing method embodiment. For a brief description, for the parts not mentioned in the system embodiment, reference may be made to the corresponding content in the foregoing method embodiment.

[0137] Embodiment III

[0138] The present invention also proposes an electronic device. Please refer to Figure 3 , which shows the electronic device in the third embodiment of the present invention.

[0139] The electronic device may include a processor 71 and a memory 72 storing computer program instructions.

[0140] Specifically, the above-mentioned processor 71 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the present application.

[0141] Among them, the memory 72 may include a mass memory for data or instructions. By way of example and not limitation, the memory 72 may include a hard disk drive (HDD), a floppy disk drive, a solid state drive (SSD), a flash memory, an optical disc, a magneto-optical disc, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. In a suitable case, the memory 72 may include removable or non-removable (or fixed) media. In a suitable case, the memory 72 may be internal or external to the data processing device. In a particular embodiment, the memory 72 is a non-volatile memory. In a particular embodiment, the memory 72 includes a read-only memory (ROM) and a random access memory (RAM). In a suitable case, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically alterable ROM (EAROM), or a flash memory (FLASH), or a combination of two or more of these. In a suitable case, the RAM may be a static random access memory (SRAM) or a dynamic random access memory (DRAM), where the DRAM may be a fast page mode dynamic random access memory (FPMDRAM), an extended date out dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0142] The memory 72 can be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 71.

[0143] The processor 71 reads and executes the computer program instructions stored in the memory 72 to implement the fuel tank safety monitoring method in the first embodiment above.

[0144] In some of the embodiments, the electronic device may further include a communication interface 73 and a bus 70. Among them, as Figure 3 shown, the processor 71, the memory 72, and the communication interface 73 are connected through the bus 70 and complete communication with each other.

[0145] The communication interface 73 is used to implement communication between the various modules, devices, units, and / or devices in the present application. The communication interface 73 can also implement data communication with other components such as external devices, image / data acquisition devices, databases, external storage, and image / data processing workstations.

[0146] The bus 70 includes hardware, software, or both, and couples components of the device together. The bus 70 includes at least one of the following, including but not limited to: a data bus, an address bus, a control bus, an expansion bus, a local bus. By way of example and not limitation, the bus 70 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable bus or a combination of two or more of these. Where appropriate, the bus 70 may include one or more buses. Although this application describes and illustrates specific buses, this application contemplates any suitable bus or interconnect.

[0147] The electronic device can obtain the fuel tank safety monitoring system and execute the fuel tank safety monitoring method of Embodiment 1.

[0148] In addition, in combination with the fuel tank safety monitoring method in Embodiment 1 above, this application can be implemented by providing a storage medium. Computer program instructions are stored on the storage medium; when the computer program instructions are executed by a processor, the fuel tank safety monitoring method of Embodiment 1 above is implemented.

[0149] In the description of this specification, the descriptions referring to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples.

[0150] The above-described embodiments merely represent several implementation manners of the present invention. The descriptions thereof are relatively specific and detailed, but should not be construed as a limitation on the scope of the patent of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can still be made, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the appended claims.

Claims

1. A fuel tank safety monitoring method, characterized in that, The method includes: Receiving the monitoring video of the fuel tank area and preprocessing the monitoring video to obtain a daily scene. This step specifically includes: Collecting the monitoring video of the fuel tank area based on a camera and compressing the monitoring video into a video data stream encoded in the h264 standard; Transmitting the video data stream to a PC and transcoding it into the mp4 format for caching in the buffer of the PC. Among them, the PC establishes a connection with the monitor at the truck rearview mirror through the TCP protocol; Performing suspicious target detection on the preprocessed monitoring video based on a target detection model to record the category of the suspicious target and the spatial position of the suspicious target. This step specifically includes: Circularly extracting images from the monitoring video, adjusting the size of the images, and converting the images into the two-dimensional numpy array format; Loading the pre-trained target detection model and inputting the images in the two-dimensional numpy array format into the pre-trained target detection model; Successively performing spatial feature extraction, enhanced spatial feature extraction, feature point anchor box classification, and regression on the images in the two-dimensional numpy array format based on the pre-trained target detection model to obtain the category of the suspicious target in the images and the spatial position of the suspicious target. Among them, tubular objects and pedestrians are marked as suspicious targets, the fuel tank is marked as a key target, and the fuel dispenser and the gas station platform are marked as safe targets; Performing behavior detection on the preprocessed monitoring video based on a spatio-temporal behavior detection model to record the action category performed by the pedestrian and the spatial position where the pedestrian performs. This step specifically includes: Circularly extracting images from the monitoring video, adjusting the size of the images, and overlapping and converting the images into the three-dimensional numpy array format; Loading the pre-trained spatio-temporal behavior detection model and inputting the images in the three-dimensional numpy array format into the pre-trained spatio-temporal behavior detection model; Successively performing spatial feature extraction, temporal feature extraction, spatio-temporal feature fusion, and feature point anchor box classification and regression on the images in the three-dimensional numpy array format based on the pre-trained spatio-temporal behavior detection model to obtain the category and spatial position of the pedestrian actions in the image sequence; Extracting the key frames in the monitoring video and performing preprocessing, and tracking the pedestrian position according to the change in the pedestrian position between the current frame and the previous frame in the preprocessed key frames. This step specifically includes: Circularly extracting images from the monitoring video, adjusting the size of the images, and converting the images into the two-dimensional numpy array format; Judging whether the image converted into the two-dimensional numpy array format is the first picture; If so, assigning IDs to all pedestrians in the first picture through a target detection algorithm and recording the spatial features of the pedestrians; Differentiate the daily scenarios, refueling scenarios, and abnormal behavior scenarios of personnel in the fuel tank area. Based on the abnormal behavior scenarios of personnel in the fuel tank area, the suspicious target categories, the spatial positions of the suspicious targets, the action categories performed by the pedestrians, and the spatial positions where the pedestrians perform the actions, define the suspicious target holding a tubular object as a dangerous object. Among them, the abnormal behavior scenarios of personnel in the fuel tank area specifically include: According to the coordinates of the tubular object and the holding action, search for the tubular object closest to the holding action within a preset range centered on the holding action. If there is a closest tubular object and there is no fueling gun and gas station platform in the scenario, then determine the pedestrian performing the holding action as the dangerous object; Extract the ID of the dangerous object, and based on the ID of the dangerous object and the spatial coordinates of the fuel tank, judge the spatial relationship between the dangerous object and the fuel tank. If the proximity of the spatial relationship exceeds a preset threshold, then make an alarm decision. This step specifically includes: Set the fuel tank area as a sensitive area based on the spatial coordinates of the fuel tank; Monitor the spatial coordinate points of the dangerous object in the sensitive area; Calculate the residence time of the dangerous object in the sensitive area based on the surveillance video, and judge whether the residence time exceeds the residence time threshold. Among them, calculate the residence time of the dangerous person in the sensitive area according to the number of frames in which the dangerous person appears in the sensitive area recorded by the video FPS; If so, make an alarm decision.

2. The fuel tank safety monitoring method according to claim 1, characterized in that After the step of judging whether the image in the format of a two-dimensional numpy array is the first picture, the method further includes: If the image in the format of a two-dimensional numpy array is not the first picture, then use the Hungarian algorithm to match and the Kalman filter algorithm to predict the position of the pedestrian.

3. A fuel tank safety monitoring system, characterized in that, The system includes: A receiving module, used to receive the surveillance video of the fuel tank area and preprocess the surveillance video to obtain the daily scenario; The receiving module includes: An acquisition unit, used to acquire the surveillance video of the fuel tank area based on a camera and compress the surveillance video into a video data stream encoded in the h264 standard; A transmission unit, used to transmit the video data stream to a PC and transcode it into the mp4 format and cache it in the buffer of the PC. Among them, the PC establishes a connection with the monitor at the truck rearview mirror through the TCP protocol; A first detection module, used to detect suspicious targets in the preprocessed surveillance video based on a target detection model to record the categories of the suspicious targets and the spatial positions of the suspicious targets; The first detection module includes: A first extraction unit, used to extract images from the surveillance video in a loop, adjust the size of the images, and convert the images into the format of a two-dimensional numpy array; A first loading unit, used to load the target detection pre-trained model and input the images in the format of a two-dimensional numpy array into the target detection pre-trained model; The first feature extraction unit is used to sequentially perform spatial feature extraction, enhanced spatial feature extraction, feature point anchor box classification and regression on the image in the format of a two-dimensional numpy array based on the target detection pre-trained model, so as to obtain the category of the suspicious target and the spatial position of the suspicious target in the image, wherein the tubular object and the pedestrian are marked as suspicious targets, the fuel tank is marked as a key target, and the fuel dispenser and the gas station platform are marked as safe targets; The second detection module is used to perform behavior detection on the pre-processed surveillance video based on the spatio-temporal behavior detection model, so as to record the action category performed by the pedestrian and the spatial position where the pedestrian performs the action; The second detection module includes: The second extraction unit is used to cyclically extract images from the surveillance video, adjust the size of the images, and convert the overlapping images into the format of a three-dimensional numpy array; The second loading unit is used to load the spatio-temporal behavior detection pre-trained model and input the image in the format of a three-dimensional numpy array into the spatio-temporal behavior detection pre-trained model; The second feature extraction unit is used to sequentially perform spatial feature extraction, temporal feature extraction, spatio-temporal feature fusion, and feature point anchor box classification and regression on the image in the format of a three-dimensional numpy array based on the spatio-temporal behavior detection pre-trained model, so as to obtain the category and spatial position of the pedestrian action in the image sequence; The extraction module is used to extract the key frames in the surveillance video, perform pre-processing, and track the pedestrian position according to the change in the pedestrian position between the current frame and the previous frame in the pre-processed key frames; The extraction module includes: The third extraction unit is used to cyclically extract images from the surveillance video, adjust the size of the images, and convert the images into the format of a two-dimensional numpy array; The judgment unit is used to judge whether the image converted into the format of a two-dimensional numpy array is the first picture; The allocation unit is used to judge that if the image converted into the format of a two-dimensional numpy array is the first picture, then assign IDs to all pedestrians in the first picture through the target detection algorithm and record the spatial features of the pedestrians; The definition module is used to distinguish the daily scene, the refueling scene, and the abnormal behavior scene of personnel in the fuel tank area. Based on the abnormal behavior scene of personnel in the fuel tank area, the suspicious target category, the spatial position of the suspicious target, the action category performed by the pedestrian, and the spatial position where the pedestrian performs the action, the suspicious target holding a tubular object is defined as a dangerous object, wherein the abnormal behavior scene of personnel in the fuel tank area specifically includes: according to the coordinates of the tubular object and the holding action, find the tubular object closest to the holding action within a preset range centered on the holding action. If there is the closest tubular object and there is no fuel dispenser and gas station platform in the scene, then the pedestrian performing the holding action is determined to be the dangerous object; The judgment module is used to extract the ID of the dangerous object, judge the spatial relationship between the dangerous object and the fuel tank based on the ID of the dangerous object and the spatial coordinates of the fuel tank. If the proximity of the spatial relationship exceeds the preset threshold, then make an alarm decision; The judgment module includes: A setting unit, configured to set the fuel tank area as a sensitive area based on the spatial coordinates of the fuel tank; A monitoring unit, configured to monitor the spatial coordinate points of the dangerous object in the sensitive area; A calculation and judgment unit, configured to calculate the residence time of the dangerous object in the sensitive area based on the monitoring video, and judge whether the residence time exceeds a residence time threshold, wherein the residence time of the dangerous person in the sensitive area is calculated according to the number of frames in which the dangerous person appears in the sensitive area recorded by the video FPS; An alarm unit, configured to make an alarm decision if the residence time exceeds the residence time threshold.

4. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the fuel tank safety monitoring method according to any one of claims 1 to 2.

5. A storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the fuel tank safety monitoring method according to any one of claims 1 to 2.

Images