Well site monitoring video processing method and device, well site monitoring system and equipment

Abstract

The embodiment of the present application relates to a kind of well site monitoring video processing method, device, well station monitoring system and equipment, the method comprises: obtaining the well site monitoring video to be detected;Determine the motion trajectory information of target object in the well site monitoring video to be detected based on the trained Yolov5 neural network model, the target object includes well site equipment and extraneous object;In the case where it is determined that extraneous object is in the preset well site area according to the motion trajectory information of extraneous object, the behavior category of target object is determined based on the trained Slowfast neural network model;In the case where the behavior category of well site equipment is preset working condition, and the behavior category of extraneous object is to enter preset well site area, output early warning information;That is, the neural network model of the embodiment of the present application can quickly identify unsafe state of well site and give early warning, reduce artificial field supervision workload, improve supervision effect, realize the purpose of enterprise cost reduction operation, quality improvement and efficiency, safety control.

Description

Technical Field

[0001] This invention relates to the field of oilfield development, and in particular to a method, apparatus, well station monitoring system and equipment for processing well site monitoring videos. Background Technology

[0002] In recent years, with the promotion and operation of technologies such as video surveillance, the Internet of Things, and mobile Internet, it has been possible to monitor oilfield construction sites and production areas in a comprehensive, all-process, all-round, and all-weather manner. How to effectively utilize these video surveillance technologies is of great significance to achieving safe oil well development.

[0003] In existing technologies, manual judgment is usually required, meaning that staff monitor the well site remotely in real time, which is labor-intensive and resource-intensive, and the accuracy of identifying dangerous behaviors in the well site is low. Summary of the Invention

[0004] The embodiments of the present invention provide a method, apparatus, well station monitoring system and equipment for processing well site monitoring videos, in order to solve the technical problem that the current supervision of safe oil well development relies on manual labor and has low accuracy in identifying dangerous behaviors.

[0005] In a first aspect, embodiments of the present invention provide a method for processing well site monitoring videos, comprising: acquiring a well site monitoring video to be detected; determining the motion trajectory information of a target object in the well site monitoring video to be detected based on a trained Yolov5 neural network model, wherein the target object includes well site equipment and external objects; determining the behavior category of the target object based on a trained Slowfast neural network model when the motion trajectory information of the external object indicates that the external object is in a preset well site area; and outputting a warning message when the behavior category of the well site equipment is a preset operating condition and the behavior category of the external object is entering the preset well site area.

[0006] As an embodiment of the present invention, the Slowfast neural network model includes a first branch and a second branch; the step of determining the behavior category of the target object based on the trained Slowfast neural network model when the external object is determined to be in a preset well site area based on the motion trajectory information of the external object includes: extracting a well site monitoring video segment from the well site monitoring video when the external object is located in the preset well site area based on the motion trajectory information of the external object; sampling the well site monitoring video segment based on a first sampling frame rate and a second sampling frame rate to obtain a first sampled image frame and a second sampled image frame; inputting the first sampled image frame into the first branch to obtain a corresponding first output result; inputting the second sampled image frame into the second branch to obtain a corresponding second output result; performing global average pooling operation on the first output result and the second output result to obtain a corresponding first feature vector and a second feature vector; and inputting the first feature vector and the second feature vector into a fully connected classifier after horizontal concatenation to output the behavior category of the target object.

[0007] As an embodiment of the present invention, before determining the motion trajectory information of the target object in the well site monitoring video to be detected based on the trained Yolov5 neural network model, the method further includes: acquiring historical well site monitoring videos or real-time acquired well site monitoring videos; performing frame extraction processing on the historical well site monitoring videos or real-time acquired well site monitoring videos to obtain multiple frames of images; labeling the target objects and their behavior categories on the multiple frames of images, wherein the behavior category of the well site equipment is one of normal operation, shutdown, or maintenance; the behavior category of the external object is entering or leaving a preset entry area; preprocessing the labeled multiple frames of images to obtain sample images; and iteratively training the Yolov5 neural network model and the Slowfast neural network model to be trained based on the sample images.

[0008] As an embodiment of the present invention, the iterative training of the Yolov5 neural network model and the Slowfast neural network model to be trained based on the sample images includes: dividing the sample images into a training set and a test set; iteratively training the Yolov5 neural network model and the Slowfast neural network model to be trained based on the training set to obtain the trained Yolov5 neural network model and the Slowfast neural network model; and verifying the trained Yolov5 neural network model and the Slowfast neural network model based on the test set.

[0009] As an embodiment of the present invention, the preprocessing includes at least one of the following: data cleaning, data augmentation, and data normalization.

[0010] Secondly, embodiments of the present invention provide a processing device for well site monitoring video, comprising: an acquisition module for acquiring well site monitoring video to be detected; a detection module for determining the motion trajectory information of a target object in the well site monitoring video to be detected based on a trained Yolov5 neural network model, wherein the target object includes well site equipment and external objects; and, when it is determined that the external object is in a preset well site area based on the motion trajectory information of the external object, determining the behavior category of the target object based on a trained Slowfast neural network model; and an output module for outputting warning information when the behavior category of the well site equipment is a preset working condition and the behavior category of the external object is entering the preset well site area.

[0011] Thirdly, embodiments of the present invention provide a well site monitoring system, including at least one camera, a well site monitoring video processing device, and a monitoring interface; wherein, the well site monitoring video processing device is used to implement the steps of the well site monitoring video processing method as described in any of the first aspects; the at least one camera is used to collect well site monitoring video and send the well site monitoring video to the well site monitoring video processing device; the monitoring interface is used to receive and display the warning information output by the well site monitoring video processing device.

[0012] As an embodiment of the present invention, the camera is used to collect monitoring videos of different well site equipment; the system also includes a monitoring and scheduling module; the monitoring and scheduling module is used to obtain the monitoring videos of the well site equipment collected by the corresponding camera according to the collection instruction, and forward the monitoring videos of the well site equipment to the well site monitoring video processing device.

[0013] As an embodiment of the present invention, the system further includes an event recording module, which is used to record the processing measures for the early warning information.

[0014] Fourthly, embodiments of the present invention provide an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; the memory is used to store computer programs; and the processor is used to execute the program stored in the memory to implement the steps of the well site monitoring video processing method described in any one of the first aspects.

[0015] Fifthly, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, characterized in that, when the computer program is executed by a processor, it implements the steps of the well site monitoring video processing method described in any one of the first aspects.

[0016] The embodiments of the present invention provide a method, apparatus, well site monitoring system, and equipment for processing well site monitoring videos. The method includes: acquiring a well site monitoring video to be detected; determining the motion trajectory information of a target object in the well site monitoring video based on a trained YOLOv5 neural network model, wherein the target object includes well site equipment and external objects; determining the behavior category of the target object based on a trained Slowfast neural network model when the external object is determined to be in a preset well site area according to the motion trajectory information of the external object; and outputting a warning message when the behavior category of the well site equipment is a preset working condition and the behavior category of the external object is entering the preset well site area. That is, the embodiments of the present invention can quickly identify unsafe conditions at the well site and issue warnings through the YOLOv5+Slowfast neural network model, reduce the workload of manual on-site supervision, improve supervision effectiveness, and achieve the goals of cost reduction, efficiency improvement, and safety management for enterprises. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A flowchart illustrating a method for processing well site monitoring video provided in an embodiment of the present invention;

[0020] Figure 2 for Figure 1 A detailed flowchart of step S103 in the method for processing well site monitoring videos;

[0021] Figure 3 A flowchart illustrating another method for processing well site monitoring videos provided in an embodiment of the present invention;

[0022] Figure 4 A schematic diagram of model training provided in an embodiment of the present invention;

[0023] Figure 5 A flowchart illustrating a method for processing well site monitoring videos provided in an embodiment of the present invention;

[0024] Figure 6 A schematic diagram of a well site monitoring video processing device provided in an embodiment of the present invention;

[0025] Figure 7 This is a schematic diagram of the structure of a well station monitoring system provided in an embodiment of the present invention;

[0026] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] In recent years, with the advancement of the "four modernizations" construction in oilfields, digital and intelligent oilfields have become the future development trend. The promotion and operation of technologies such as video surveillance, the Internet of Things, and mobile internet have enabled comprehensive, all-process, all-round, and 24 / 7 monitoring of construction sites and production areas. Video surveillance data has also become increasingly diversified, massive, and varied. Currently, the number of oil well monitoring cameras in oilfields is growing exponentially. This data contains crucial information about the daily operations of oil wells, and effectively utilizing this data is of great significance for achieving oil well safety supervision and process control.

[0029] In existing technologies, manual judgment is usually required, meaning that staff monitor the well site remotely in real time, which is labor-intensive and resource-intensive, and the accuracy of identifying dangerous behaviors in the well site is low.

[0030] To address the aforementioned technical problems, the technical concept of this invention is as follows: based on the Yolov5+Slowfast neural network model, the operating status of well site equipment and whether there are any foreign objects intruding into key areas of the well site are quickly identified, and an early warning message is issued when the well site is determined to be in an unsafe state.

[0031] Figure 1 This is a flowchart illustrating a method for processing well site monitoring videos according to an embodiment of the present invention. The executing entity is a well site monitoring video processing device, an electronic device equipped with a well site monitoring video processing device, or a well station monitoring system equipped with a well site monitoring video processing device. Figure 1 As shown, the processing method for the well site monitoring video includes:

[0032] Step S101: Obtain the monitoring video of the well site to be inspected.

[0033] Specifically, multiple cameras are installed at different locations in the well site to collect video data of the well site, such as video data of the pumping unit area.

[0034] Step S102: Determine the motion trajectory information of the target object in the well site monitoring video to be detected based on the trained Yolov5 neural network model. The target object includes well site equipment and external objects.

[0035] Specifically, the trained Yolov5 neural network model performs image processing on the well site monitoring video to obtain the location information and category of the target object detection boxes in the keyframe images of the well site monitoring video. The category is determined based on the target object, divided into two types: well site equipment and external objects. Taking the monitoring video of the pumping unit area as an example, target objects include the crank and donkey head of the pumping unit, while external objects include people, animals, and vehicles. The motion trajectory of the target object can be determined based on the location information and category of the detection boxes in the keyframe image sequence.

[0036] Step S103: After determining that the external object is in the preset well site area based on the movement trajectory information of the external object, determine the behavior category of the target object based on the trained Slowfast neural network model.

[0037] Specifically, taking an oil pumping unit as an example, the preset area is a range of 30m × (30m to 50m), which requires accurate identification of the behavior category of foreign objects entering this area. Based on the movement trajectory information of the foreign object, video clips of the foreign object within the preset well site area are extracted from the well site monitoring video. These video clips are then input into a trained Slowfast neural network model, which outputs the behavior category of the target object. The behavior category includes whether the well site equipment is operating normally, and whether the foreign object has entered or left the preset well site area.

[0038] Figure 2 for Figure 1 A detailed flowchart of step S103 in the method for processing well site monitoring videos.

[0039] In some embodiments, such as Figure 2 As shown, the Slowfast neural network model includes a first branch and a second branch; in step S103, when it is determined that the external object is in the preset well site area based on the motion trajectory information of the external object, the behavior category of the target object is determined based on the trained Slowfast neural network model, including the following steps:

[0040] Step S201: Extract a well site monitoring video segment from the well site monitoring video when the external object is located in the preset well site area based on the movement trajectory information of the external object.

[0041] Step S202: Sample the well site monitoring video clips based on the first sampling frame rate and the second sampling frame rate to obtain the first sampled image frame and the second sampled image frame.

[0042] Step S203: Input the first sampled image frame into the first branch to obtain the corresponding first output result.

[0043] Step S204: Input the second sampled image frame into the second branch to obtain the corresponding second output result.

[0044] Step S205: Perform global average pooling on the first output result and the second output result respectively to obtain the corresponding first feature vector and second feature vector.

[0045] Step S206: The first feature vector and the second feature vector are concatenated horizontally and then input into a fully connected classifier to output the behavior category of the target object.

[0046] Specifically, the SlowFast neural network model includes a Slow path (the first path) and a Fast path (the second path). It samples video segments at a first sampling frame rate (low frame rate) and a second sampling frame rate (high frame rate), and inputs the corresponding sampled image frames into the Slow path and Fast path, respectively, to capture low-frame-rate spatial semantics and high-frame-rate temporal information. Global average pooling is performed on the output of each path. Finally, the two merged feature vectors are horizontally concatenated and used as the input of a fully connected classifier to output the behavior category of the target object.

[0047] Step S104: When the behavior category of the well site equipment is the preset working condition and the behavior category of the external object is entering the preset well site area, output the warning information.

[0048] Specifically, the preset operating conditions can be normal operating conditions. Taking the pumping unit as an example, when the pumping unit is operating normally, if an external object enters the preset well site area of ​​the pumping unit, it may cause damage to the external object, or the entry of the external object may affect the normal operation of the pumping unit. In this case, an early warning message can be output so that the staff can understand the situation as soon as possible and take appropriate measures to deal with it. When the pumping unit is in a shutdown or maintenance state, if an external object enters, no early warning is required.

[0049] The well site monitoring video processing method provided by the embodiments of the present invention acquires the well site monitoring video to be detected; determines the motion trajectory information of target objects in the well site monitoring video based on a trained YOLOv5 neural network model, the target objects including well site equipment and external objects; when it is determined that the external object is in a preset well site area based on the motion trajectory information of the external object, the behavior category of the target object is determined based on a trained Slowfast neural network model; when the behavior category of the well site equipment is a preset working condition and the behavior category of the external object is entering the preset well site area, an early warning information is output. This realizes intelligent and proactive early warning of risks such as unsafe conditions, poor environment and management defects in the well site through the YOLOv5+Slowfast neural network model, assisting managers in achieving convenient remote supervision and rapid handling of violations, thereby reducing the workload of manual on-site supervision, improving supervision effectiveness, and achieving the goals of cost reduction, quality improvement and efficiency enhancement, and safety management for enterprises.

[0050] Based on the above embodiments, Figure 3 This is a flowchart illustrating another method for processing well site monitoring videos provided in an embodiment of the present invention, as shown below. Figure 3 As shown, the processing method for the well site monitoring video includes:

[0051] Step S301: Obtain historical well site monitoring videos or real-time well site monitoring videos.

[0052] Step S302: Perform frame extraction processing on the historical well site monitoring video or the real-time acquired well site monitoring video to obtain multiple frames of images.

[0053] Step S303: Label the target objects and their behavior categories on the multi-frame images.

[0054] The behavior category of the well site equipment is one of normal operation, shutdown, or maintenance; the behavior category of the external object is entering or leaving the preset entry area.

[0055] Step S304: Preprocess the labeled multi-frame images to obtain sample images.

[0056] Step S305: Iteratively train the Yolov5 neural network model and the Slowfast neural network model to be trained based on the sample images.

[0057] Step S306: Obtain the monitoring video of the well site to be inspected.

[0058] Step S307: Determine the motion trajectory information of the target object in the well site monitoring video to be detected based on the trained Yolov5 neural network model.

[0059] The target objects include well site equipment and external objects.

[0060] Step S308: If the external object is determined to be in the preset well site area based on the motion trajectory information of the external object, the behavior category of the target object is determined based on the trained Slowfast neural network model.

[0061] Step S309: When the behavior category of the well site equipment is the preset working condition and the behavior category of the external object is entering the preset well site area, output the warning information.

[0062] The implementation of steps S306-S309 in this embodiment is similar to that of steps S101-S104 in the above embodiment, and will not be described again here.

[0063] The difference from the above embodiments lies in that this embodiment further defines the training process of the Yolov5+Slowfast neural network model. In this embodiment, historical well site monitoring videos or real-time acquired well site monitoring videos are acquired; frame extraction processing is performed on the historical well site monitoring videos or real-time acquired well site monitoring videos to obtain multi-frame images; target objects and their behavior categories on the multi-frame images are labeled, wherein the behavior category of the well site equipment is one of normal operation, shutdown, or maintenance; the behavior category of the external object is entering or leaving a preset entry area; the labeled multi-frame images are preprocessed to obtain sample images; and the Yolov5 neural network model and Slowfast neural network model to be trained are iteratively trained based on the sample images.

[0064] Specifically, historical or on-site monitoring videos of oil and gas production operations are obtained from major oilfields; then, these videos are processed by frame extraction to obtain multi-frame images of well site equipment or external objects, forming a sample material library; then, each frame image is labeled, including the target object and the target object's behavior category; then, the Yolov5+Slowfast neural network model is iteratively trained based on the labeled sample images.

[0065] In some embodiments, the preprocessing includes at least one of the following: data cleaning, data augmentation, and data normalization. Specifically, after obtaining the labeled sample images, the input data size (1920Pixel × 1080Pixel) can be automatically adjusted according to the network model to suit the network input; useful information in the images can be retained while removing useless noise and interference, deleting abnormal or blurry images, performing supplementary lighting processing for images with low exposure, and defogging processing for images from rainy or foggy days, etc.; data augmentation can also be performed to expand the image set; and the pixels in the image can be normalized from the 0-255 pixel range to the 0-1 range to facilitate calculation.

[0066] In some embodiments, step S303 includes: dividing the sample images into a training set and a test set; iteratively training the Yolov5 neural network model and the Slowfast neural network model to be trained based on the training set to obtain the trained Yolov5 neural network model and the Slowfast neural network model; and validating the trained Yolov5 neural network model and the Slowfast neural network model based on the test set.

[0067] Specifically, the preprocessed dataset can be divided into training and test sets in an 8:2 ratio. The dataset is labeled using AVA format to create the training set for the SlowFast neural network, and the VOC-labeled dataset is converted to YOLO format to create the training set for the YOLOv5 neural network. The SlowFast+YOLOv5 deep learning model is iteratively trained using the training sets. To improve the model's feature learning ability, the Adam optimization function is used as the loss function, which effectively addresses excessive fluctuations and accelerates the model's convergence. Simultaneously, the learning rate of each parameter is dynamically adjusted using first-moment and second-moment estimations of the gradient, achieving adaptive learning rate. The YOLOv5+YOLOv5 neural network is iteratively trained using the above steps, and the trained model is saved. The model performance is then validated on the test set. If the validation results meet expectations, the network model is saved as the final model; otherwise, the model is retrained.

[0068] Figure 4 This is a schematic diagram of model training provided in an embodiment of the present invention, such as... Figure 4As shown, the process begins by acquiring sample library materials, specifically multi-frame images from well site monitoring videos of major oil plants, after frame extraction. Next, sample annotation is performed, identifying the target objects and behavioral categories on each frame. Then, preprocessing is carried out, including data cleaning, data augmentation, and data normalization. The preprocessed data is then divided into training and testing sets at an 8:2 ratio. An initial Yolov5+Slowfast neural network model (to be trained) is constructed. The network model is trained using the training set, and its parameters are tuned. The trained network model is saved. The saved network model is tested using the testing set, and the test set performance is evaluated to determine if it meets expectations. If not, the network model is retrained and its parameters are tuned again; if yes, the saved model is considered the final model.

[0069] Figure 5 A flowchart illustrating a method for processing well site monitoring videos provided in an embodiment of the present invention is shown below. Figure 5 As shown, the system can acquire well site monitoring videos in real time, and perform frame extraction and annotation on the videos to obtain new training data. The original training data consists of sample images from previously annotated historical monitoring videos. Preprocessing, including data cleaning, data augmentation, and data normalization, is performed on both the new and original training data to obtain training and testing sets. The training set is used to iteratively train the Yolov5+Slowfast neural network model, and the test set is used to determine whether the model has achieved the expected results. If not, retraining is performed; otherwise, the trained model is deployed online. The video to be detected is then input into the model to detect any intrusion of foreign objects, and the corresponding decision is determined based on the detection results and sent to the response unit.

[0070] Based on the aforementioned embodiments, historical well site monitoring videos or real-time acquired well site monitoring videos are obtained; frame extraction processing is performed on the historical well site monitoring videos or real-time acquired well site monitoring videos to obtain multi-frame images; target objects and their behavior categories on the multi-frame images are labeled, wherein the behavior category of the well site equipment is one of normal operation, shutdown, or maintenance; the behavior category of the external object is entering or leaving the preset entry area; the labeled multi-frame images are preprocessed to obtain sample images; and the Yolov5 neural network model and Slowfast neural network model to be trained are iteratively trained based on the sample images, thus realizing the training of the Yolov5 and Slowfast neural network models.

[0071] Figure 6 This is a schematic diagram of the structure of a well site monitoring video processing device provided in an embodiment of the present invention, as shown below. Figure 6 As shown, the device includes:

[0072] The acquisition module 601 is used to acquire the well site monitoring video to be detected; the detection module 602 is used to determine the motion trajectory information of the target object in the well site monitoring video to be detected based on the trained Yolov5 neural network model, wherein the target object includes well site equipment and external objects; when it is determined that the external object is in a preset well site area based on the motion trajectory information of the external object, the behavior category of the target object is determined based on the trained Slowfast neural network model; the output module 603 is used to output early warning information when the behavior category of the well site equipment is a preset working condition and the behavior category of the external object is entering the preset well site area.

[0073] As an embodiment of the present invention, the Slowfast neural network model includes a first branch and a second branch. The detection module 602 is specifically used for: extracting a well site monitoring video segment from the well site monitoring video when the external object is located in a preset well site area based on the motion trajectory information of the external object; sampling the well site monitoring video segment based on a first sampling frame rate and a second sampling frame rate to obtain a first sampled image frame and a second sampled image frame; inputting the first sampled image frame into the first branch to obtain a corresponding first output result; inputting the second sampled image frame into the second branch to obtain a corresponding second output result; performing global average pooling operation on the first output result and the second output result to obtain a corresponding first feature vector and a second feature vector; and inputting the first feature vector and the second feature vector into a fully connected classifier after horizontal concatenation to output the behavior category of the target object.

[0074] As an embodiment of the present invention, the device further includes a training module 604, configured to: acquire historical well site monitoring videos or real-time acquired well site monitoring videos; perform frame extraction processing on the historical well site monitoring videos or real-time acquired well site monitoring videos to obtain multi-frame images; label the target objects and their behavior categories on the multi-frame images, wherein the behavior category of the well site equipment is one of normal operation, shutdown, or maintenance; the behavior category of the external object is entering or leaving a preset entry area; preprocess the labeled multi-frame images to obtain sample images; and iteratively train the Yolov5 neural network model and the Slowfast neural network model to be trained based on the sample images.

[0075] In one embodiment of the present invention, the training module 604 is specifically used for: dividing the sample images into a training set and a test set; iteratively training the Yolov5 neural network model and the Slowfast neural network model to be trained based on the training set to obtain the trained Yolov5 neural network model and the Slowfast neural network model; and verifying the trained Yolov5 neural network model and the Slowfast neural network model based on the test set.

[0076] As an embodiment of the present invention, the preprocessing includes at least one of the following: data cleaning, data augmentation, and data normalization.

[0077] The well site monitoring video processing device provided in this embodiment of the invention has a similar implementation principle and technical effect to the above embodiments, and will not be described again here.

[0078] Figure 7 This is a schematic diagram of the structure of a well station monitoring system provided in an embodiment of the present invention, as shown below. Figure 7 As shown, the well site monitoring system includes at least one camera, a well site monitoring video processing device, and a monitoring interface;

[0079] The well site monitoring video processing device is used to implement the steps of the well site monitoring video processing method described above; the at least one camera is used to collect well site monitoring video and send the well site monitoring video to the well site monitoring video processing device; the monitoring interface is used to receive and display the warning information output by the well site monitoring video processing device.

[0080] Specifically, the well site monitoring system includes multiple cameras, each capable of capturing video images of different scenes, such as external pumping stations, sewage tanks, external power distribution rooms, and oil pumping units. The well site monitoring video processing device contains pre-trained Yolov5+Slowfast neural network models corresponding to different scenes, which can issue warnings for foreign objects entering the scene and send the warning information to the monitoring interface for display.

[0081] As an embodiment of the present invention, the camera is used to collect monitoring videos of different well site equipment; the system also includes a monitoring and scheduling module; the monitoring and scheduling module is used to obtain the monitoring videos of the well site equipment collected by the corresponding camera according to the collection instruction, and forward the monitoring videos of the well site equipment to the well site monitoring video processing device.

[0082] As an embodiment of the present invention, the system further includes an event recording module, which is used to record the processing measures for the early warning information.

[0083] refer to Figure 7As shown, the working process of the well site monitoring system is as follows: Each camera is used to collect monitoring videos from different locations in the well site (such as the external pumping station, sewage tank, external power distribution room, pumping unit, etc.); if the monitoring interface receives a user's collection command for the pumping unit, it will send the collection command to the monitoring and dispatching module. The monitoring and dispatching module will obtain the monitoring video of the pumping unit area from the corresponding camera and forward the video to the well site monitoring video processing device for detection to determine whether there is any intrusion. If an intrusion is found, an early warning message will be output to the monitoring interface for display. After the user sees the early warning message through the monitoring interface, they can input control commands through the monitoring interface to determine the corresponding handling measures; the monitoring video, early warning message, and handling measures of the entire process are recorded through the event logging module.

[0084] The well site monitoring system provided in this embodiment of the invention embeds the well site monitoring video processing method provided in the above embodiment into the well site monitoring system, thereby completing the automatic identification and alarm function for intrusions within the well site area, eliminating the need for manual judgment and related work for subsequent monitoring and maintenance.

[0085] like Figure 8 As shown, this embodiment of the invention provides an electronic device, including a processor 801, a communication interface 802, a memory 803, and a communication bus 804, wherein the processor 801, the communication interface 802, and the memory 803 communicate with each other via the communication bus 804.

[0086] Memory 803 is used to store computer programs;

[0087] In one embodiment of the present invention, when the processor 801 executes the program stored in the memory 803, it implements the steps of the well site monitoring video processing method provided in any of the foregoing method embodiments.

[0088] The electronic device provided in this embodiment of the invention has a similar implementation principle and technical effect to the above embodiments, and will not be described again here.

[0089] The aforementioned memory 803 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Memory 803 has storage space for program code used to perform any of the method steps described above. For example, the storage space for program code may include individual program codes for implementing the various steps in the methods described above. This program code can be read from or written to one or more computer program products. These computer program products include program code carriers such as hard disks, optical discs (CDs), memory cards, or floppy disks. Such computer program products are typically portable or fixed storage units. The storage unit may have storage segments or storage spaces arranged similarly to the memory 803 in the aforementioned electronic device. The program code may be compressed, for example, in a suitable form. Typically, the storage unit includes programs for performing the method steps according to embodiments of the invention, i.e., code that can be read by a processor such as 801, which, when run by the electronic device, causes the electronic device to perform the various steps in the methods described above.

[0090] Embodiments of the present invention also provide a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the well site monitoring video processing method described above.

[0091] The computer-readable storage medium may be included in the device / apparatus described in the above embodiments; or it may exist independently and not assembled into the device / apparatus. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of the present invention.

[0092] According to embodiments of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, such as including, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0093] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0094] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for processing well site monitoring video, characterized in that, include: Obtain monitoring videos of the well site to be inspected; The motion trajectory information of the target object in the well site monitoring video to be detected is determined based on the trained Yolov5 neural network model. The target object includes well site equipment and external objects. Based on the trajectory information of the external object, it is determined that the external object is in the preset well site area. Then, the behavior category of the target object is determined based on the trained Slowfast neural network model. When the behavior category of the well site equipment is a preset working condition, and the behavior category of the external object is entering the preset well site area, an early warning message is output.

2. The method according to claim 1, characterized in that, The Slowfast neural network model includes a first branch and a second branch; the step of determining the behavior category of the target object based on the trained Slowfast neural network model, when the trajectory information of the external object indicates that the external object is in a preset well site area, includes: Based on the movement trajectory information of the external object, extract the well site monitoring video segment when the external object is located in the preset well site area from the well site monitoring video; The well site monitoring video clips are sampled based on the first sampling frame rate and the second sampling frame rate to obtain the first sampled image frame and the second sampled image frame. The first sampled image frame is input into the first branch to obtain the corresponding first output result; The second sampled image frame is input into the second branch to obtain the corresponding second output result; Perform global average pooling on the first output result and the second output result respectively to obtain the corresponding first feature vector and second feature vector; The first and second feature vectors are concatenated laterally and then input into a fully connected classifier to output the behavior category of the target object.

3. The method according to claim 2, characterized in that, Before determining the motion trajectory information of the target object in the well site monitoring video to be detected based on the trained Yolov5 neural network model, the method further includes: Obtain historical well site monitoring videos or real-time well site monitoring videos; The historical well site monitoring video or the real-time acquired well site monitoring video is subjected to frame extraction processing to obtain multiple frames of images; The target objects and their behavior categories on multiple frames of images are labeled. The behavior category of the well site equipment is one of normal operation, shutdown, or maintenance. The behavior category of the external object is entering or leaving the preset entry area. Preprocess the labeled multi-frame images to obtain sample images; The sample images are used to iteratively train the Yolov5 neural network model and the Slowfast neural network model to be trained.

4. The method according to claim 3, characterized in that, The iterative training of the Yolov5 neural network model and the Slowfast neural network model to be trained based on the sample images includes: The sample images are divided into a training set and a test set; Based on the training set, iterative training is performed on the Yolov5 neural network model and the Slowfast neural network model to be trained to obtain the trained Yolov5 neural network model and the Slowfast neural network model. The trained Yolov5 neural network model and Slowfast neural network model are validated based on the test set.

5. The method according to claim 3, characterized in that, The preprocessing includes at least one of the following: Data cleaning, data augmentation, and data normalization.

6. A processing device for well site monitoring video, characterized in that, include: The acquisition module is used to acquire the monitoring video of the well site to be inspected; The detection module is used to determine the motion trajectory information of the target object in the well site monitoring video to be detected based on the trained Yolov5 neural network model. The target object includes well site equipment and external objects. When it is determined that the external object is in a preset well site area based on the motion trajectory information of the external object, the behavior category of the target object is determined based on the trained Slowfast neural network model. The output module is used to output early warning information when the behavior category of the well site equipment is a preset working condition and the behavior category of the external object is entering the preset well site area.

7. A well site monitoring system, characterized in that, It includes at least one camera, a well site monitoring video processing device, and a monitoring interface; The well site monitoring video processing device is used to implement the steps of the well site monitoring video processing method as described in any one of claims 1-5; The at least one camera is used to collect well site monitoring video and send the well site monitoring video to the well site monitoring video processing device; The monitoring interface is used to receive and display the early warning information output by the processing device of the well site monitoring video.

8. The system according to claim 7, characterized in that, The camera is used to collect monitoring videos of different well site equipment; the system also includes a monitoring and scheduling module. The monitoring and scheduling module is used to acquire the monitoring video of the well site equipment captured by the corresponding camera according to the acquisition instruction, and forward the monitoring video of the well site equipment to the well site monitoring video processing device.

9. The system according to claim 8, characterized in that, The system also includes an event logging module, which is used to record the handling measures for early warning information.

10. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in a memory, implements the steps of the well site monitoring video processing method according to any one of claims 1-5.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the well site monitoring video processing method as described in any one of claims 1-5.

Images