Pumping unit motion state monitoring system based on deep learning

Abstract

The invention discloses a pumping unit motion state monitoring system based on deep learning. A data acquisition unit uses a camera to acquire pumping unit data; the data processing unit processes the collected data based on a multi-target tracking model carried by the development board; the visual platform receives the real-time data flow and provides instant operation state information for the user; the service scheduling system generates a control instruction according to the operation state information and monitoring data from non-camera equipment in the monitoring system, the instruction is sent to the weak current controller to control the strong current controller, and then accurate scheduling of well site equipment is achieved through the strong current controller; the model updating subsystem is used for carrying out online analysis on data and training a multi-target tracking model; the monitoring system monitors various indexes of operation of the oil pumping unit in real time. The system realizes real-time tracking of the motion state of the oil pumping unit, provides accurate data for efficient operation of various green electricity control oil pumping units provided by a coordination control platform, and achieves the purposes of energy conservation and efficiency improvement.

Description

Technical Field

[0001] The present disclosure belongs to the field of deep learning and artificial intelligence technology, and particularly relates to a pumping unit motion state monitoring system based on deep learning. Background Art

[0002] The petrochemical industry, a key pillar of the national economy, is currently a major source of carbon emissions. Emissions from this industry account for approximately 14% of the nation's total, and the cost of reducing emissions is estimated to be as high as 300 yuan per ton or more.

[0003] To improve production efficiency and reduce carbon emissions, oil well sites deploy numerous monitoring and testing devices, including motors, water distribution pumps, angular displacement meters, load meters, dynamic liquid level analyzers, gateways, switches, and more, to track the operating status of pumping units and wellsite equipment. This equipment is typically manufactured by different manufacturers, serving different functions and users. Consequently, its data is multi-source and heterogeneous, encompassing data such as electrical parameters, dynamic liquid level depth, water pressure, air pressure, oil and water content, oil and gas, video images, angular displacement, donkey head travel, ball pitching, and chemical dosing.

[0004] Therefore, in order to address the heterogeneous data problem at oil well sites, it is urgently necessary to combine AI technology to upgrade and transform oil field production and optimize business processes. Summary of the Invention

[0005] In view of this, the present disclosure provides a deep learning-based pumping unit motion state monitoring system, including a data acquisition unit, a data processing unit, a model update subsystem, a monitoring system, a visualization platform and a business scheduling system, wherein:

[0006] The data acquisition unit uses a camera to collect pumping unit data;

[0007] The data processing unit processes the collected data based on the multi-target tracking model carried by the development board;

[0008] The visualization platform receives real-time data streams and provides users with immediate job status information;

[0009] The business scheduling system generates control instructions based on the operation status information and monitoring data from non-camera equipment in the monitoring system. The instructions are sent to the weak current controller, which then controls the strong current controller through the weak current controller to achieve accurate scheduling of well site equipment;

[0010] The model updating subsystem performs online data analysis and trains the multi-target tracking model;

[0011] The monitoring system monitors various indicators of the operation of the oil pumping unit in real time.

[0012] Developed based on the Nezha series IoT development kit, this system consists of data acquisition and processing modules, as well as a model update service subsystem. The data acquisition component uses a camera to capture image data of the "donkey head" motion within the well site. Multi-target tracking technology is then used to track the "donkey head"'s trajectory, and its start and stop status is monitored based on a designed "donkey head" deviation calculation algorithm. This "donkey head" motion information is then provided to a central control system, which coordinates and controls the operation of multiple well pumping units based on oilfield environmental data (such as three-phase electrical parameters, load, and oil production rate) and green power. The model update service subsystem periodically trains and updates the tracking model online using video data. Overall, this system, based on a multi-target tracking algorithm and a proposed proprietary "donkey head" deviation motion algorithm, achieves real-time tracking of the "donkey head" motion of the pumping units. This provides accurate data for the efficient operation of the various green power-controlled pumping units provided by the coordinated control platform, ultimately achieving energy savings and efficiency improvements. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 1 is a schematic structural diagram of a deep learning-based pumping unit motion state monitoring system provided in one embodiment of the present disclosure;

[0014] Figure 2 This is a schematic diagram of the Nezha development board provided in one embodiment of the present disclosure;

[0015] Figure 3 is a basic flow chart of FairMOT provided in one embodiment of the present disclosure;

[0016] Figure 4 This is a demonstration diagram of tracking results provided in one embodiment of the present disclosure;

[0017] Figure 5 It is a model framework diagram of a multi-target tracking model provided in one embodiment of the present disclosure. DETAILED DESCRIPTION

[0018] In order to make those skilled in the art understand the technical solutions disclosed in the present invention, the following will be combined with the embodiments and related appended Figures 1 to 5 , the technical solutions of various embodiments are described, and the described embodiments are part of the embodiments of the present invention, but not all of the embodiments.

[0019] References to "embodiments" herein mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present invention. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. Those skilled in the art will appreciate that the embodiments described herein may be combined with other embodiments.

[0020] See also Figure 1 In one embodiment, the present invention discloses a deep learning-based pumping unit motion state monitoring system, comprising a data acquisition unit, a data processing unit, a model update subsystem, a monitoring system, a visualization platform, and a business scheduling system, wherein:

[0021] The data acquisition unit uses a camera to collect pumping unit data;

[0022] The data processing unit processes the collected data based on the multi-target tracking model carried by the development board;

[0023] The visualization platform receives real-time data streams and provides users with immediate job status information;

[0024] The business scheduling system generates control instructions based on the operation status information and monitoring data from non-camera equipment in the monitoring system. The instructions are sent to the weak current controller, which then controls the strong current controller through the weak current controller to achieve accurate scheduling of well site equipment;

[0025] The model updating subsystem performs online data analysis and trains the multi-target tracking model;

[0026] The monitoring system monitors various indicators of the operation of the oil pumping unit in real time.

[0027] In this embodiment, to further enhance the carbon reduction and efficiency improvement of smart, unmanned wellsites, we utilize advanced multi-target tracking algorithms. Real-time camera footage is captured, and computers perform data processing and analysis to identify and track the movement trajectory of the pumping units within the wellsite. This system determines the start and stop status of the pumping units and, combined with monitoring results from other equipment within the wellsite, optimizes the operation of the pumping units and green electricity usage within the wellsite. The goal is to develop and build an easy-to-use pumping unit tracking system using deep learning technology and embedded devices. This system will enable low-carbon, efficient, and intelligent management at the wellsite, reduce human resource investment, and achieve low-carbon management goals. This system enables efficient and intelligent management, improves wellsite management efficiency, and provides important support and assurance for the sustainable development of the petroleum and petrochemical industries.

[0028] This system uses advanced image tracking technology to enable real-time online monitoring and intelligent analysis of the operating status of pumping units at a wellsite. A deep learning-based multi-target tracking algorithm is employed to precisely track the motion trajectory of the pumping unit's head, thereby monitoring the equipment's operational status, including startup and shutdown. By combining head motion data with other key well parameters, this system achieves efficient and energy-saving operation of the pumping unit without human intervention. The specific goal is to develop an intelligent control system that autonomously determines the start and shutdown timing of the well, based on the well's optimal production schedule. The core objectives are as follows: 1) Intelligently adjust the well's operating hours to achieve the optimal balance between production and efficiency without reducing liquid production. 2) Significantly reduce the well's operating hours, thereby directly reducing power consumption and maximizing energy savings. 3) By reducing unnecessary operating time, equipment life is extended and maintenance costs are reduced. 4) Automation and intelligent technology are used to improve well operation safety, preventing accidents caused by human error or oversights. 5) Enhance the flexibility of well management, ensuring energy management is more aligned with market dynamics and environmental policies. 6) Provide real-time data and analysis results to decision makers to support a more scientific decision-making process.

[0029] The monitoring system includes displacement sensors, cameras, angular positioners, and dynamic fluid level gauges. Monitoring data primarily comes from various testing equipment, such as angular displacement, oil level testers, oil-water ratio detectors, wind / photovoltaic power voltage / current, utility power peak and valley time periods, and pumping unit operating status (running / stopped / speed / stroke). The monitoring system provides real-time monitoring of various pumping unit operating indicators, including stroke, stroke frequency, suspension point load, balance, motor current, power consumption, and pump efficiency.

[0030] Non-camera equipment and the data they detect include: oil level depth data detected by the echo sounder, oil-water ratio data from the oil-water ratio tester, working voltage and current of three electrical parameters, maximum travel distance of angular displacement, etc., as well as determination of whether the current oil pump is in a high energy consumption and low efficiency moment.

[0031] The command is sent to the pumping unit to stop or start operation. The specific process is as follows:

[0032] The dispatching system combines data analysis and sends a weak current signal to the weak current control to trigger the strong current control (normal working current). After receiving the weak current signal, the strong current control controls the movement of the pumping unit (stop or run) by cutting off or restoring the normal working current.

[0033] The model update subsystem receives real-time video data and other device data to retrain the original model to improve tracking accuracy. The model update subsystem performs online data analysis, which involves cleaning, enhancing, and labeling camera video. For example, it identifies which pumping units in the current video frame need to be marked as running and which need to be marked as stopped. The training method for the model update subsystem is as follows:

[0034] Preliminary preparation: FairMOT environment, Donkey Head video.

[0035] (1) Use DarkLabel to label the original video and generate a gt file. The content format of the gt file is as follows:

[0036] , <id>,<bb_left>,<bb_top>,<bb_width>,<bb_height>, <cos>

[0037] Among them, indicates in which frame the target appears, <id>Indicates the tracklet ID to which the target belongs. The next four values ​​indicate the position of the target's bounding box in 2D frame coordinates, represented by the coordinates of the upper left corner and the width and height of the bounding box. <cos>Indicates whether the integrity of the target should be considered (1) or ignored (0).

[0038] (2) Use ffmpeg to split the video into frames and rename them. The command is as follows:

[0039] ffmpeg-iinput_video.mp4-vf'fps=30'%06d.jpg

[0040] (3) Refer to the FairMOT project to create the dataset file, seqinfo.ini file, and path file.

[0041] (4) Run the gen_labels.py file to generate labels for each image based on gt.txt, that is, to generate the data format required for training. The data format required for training is as follows:

[0042] <class> <id><x_center / img_width><y_center / img_height><w / img_width><h / img_height>

[0043] class: target category

[0044] id: target id

[0045] x_center / img_width: normalized center column coordinates

[0046] y_center / img_height: normalized center row coordinates

[0047] w / img_width: normalized width

[0048] h / img_height: normalized height

[0049] (5) Run the script for training

[0050] python train.py mot--exp_id horsehead--gpus 0--batch_size 6

[0051] --load_model your_model.pth

[0052] In another embodiment, the operational status information includes the operating status of the oil well, efficiency indicators, and any immediate alarms or notifications.

[0053] In this embodiment, monitoring includes oil pressure, casing pressure, and temperature (abnormalities may indicate wax deposition or blockage), casing pressure, and wellhead tightness. Alarm indicators and notifications include wellhead pressure exceeding limits, combustible gas leaks, pumping unit current fluctuations, and water content accumulation. Notifications include scheduled maintenance, such as lubricant changes. Users can use this information to quickly respond and optimize operational decisions.

[0054] In another embodiment, the monitoring data includes dynamic liquid level, oil-water mixing ratio, and touchdown result.

[0055] In this embodiment, the dynamic liquid level reflects the parameters of the liquid flow in the oil well. The oil-water mixing ratio refers to the mixing ratio of crude oil and water. The touch result may refer to the detected position or status of the downhole equipment.

[0056] The sensors and specific detection contents are as follows:

[0057] 1) Dynamic liquid level monitoring

[0058] Pressure sensor: used to measure pressure changes at the bottom of the well or in the pipeline, and indirectly calculate the liquid level.

[0059] Liquid level sensor (such as radar level meter, ultrasonic level meter): directly measures the liquid level.

[0060] Float level gauge: monitors the liquid level by changing the position of the float.

[0061] Capacitive liquid level sensor: uses capacitance changes to detect liquid level position.

[0062] 2) Oil-water mixing ratio monitoring

[0063] Capacitance or conductivity sensors: Measure the mixing ratio using the difference in dielectric constant or conductivity between oil and water.

[0064] Microwave moisture meter: detects moisture content through microwave signal attenuation.

[0065] 3) Touch result monitoring

[0066] Limit switches (mechanical or magnetic): detect when a mechanical part has reached a specific position.

[0067] etc.

[0068] In another embodiment, the precise scheduling of wellsite equipment includes starting, stopping and adjusting pumps.

[0069] In another embodiment, the model updating subsystem continuously updates and optimizes the algorithm model through the collected historical and real-time data.

[0070] This embodiment can improve the efficiency and accuracy of the system. The optimization process is as follows: by periodically sampling and labeling historical video data and other non-monitoring video data (i.e., real-time), the labeled data is used to train the historical model and improve the tracking effect.

[0071] In another embodiment, the model updating subsystem performs long-term data storage to provide resources for future data analysis and knowledge extraction.

[0072] In another embodiment, the specific steps of using the monitoring system to complete the motion status monitoring of the oil pumping unit are as follows:

[0073] S100: Equipment in the monitoring system continuously monitors the well site, capturing key data about the well's operating status;

[0074] S200: The processor processes the key data and converts it into a format that can be used by the monitoring system;

[0075] S300: The processed data is distributed to the visualization platform, the control system, and the model update subsystem, wherein the control system includes the business scheduling system, the weak current control system, and the strong current control system;

[0076] S400: The visualization platform provides users with real-time information to assist with daily monitoring and management;

[0077] S500: The control system generates a control command based on the received data and preset parameters, and transmits the command to the high-voltage controller via the low-voltage controller;

[0078] S600: The high-voltage controller executes control commands and operates well site equipment;

[0079] S700: The model updating subsystem uses the received data to perform deep learning and model training to continuously optimize the control strategy;

[0080] S800: The optimized control strategy and parameters are fed back to the control system.

[0081] In this example, the optimized control strategy and parameters are fed back to the control system, forming a self-perfecting closed loop. Through this highly integrated and automated system design, we expect to significantly improve oil well efficiency, reduce energy consumption, and ensure the safety and stability of the production process.

[0082] Key data include reservoir dynamics (production rate, pressure, and water content), equipment health (current, power diagram, and vibration), and production (stroke frequency, pump depth, and water injection plan).

[0083] The conversion formats are as follows:

[0084] Reservoir and equipment health data are structured data, mainly in JSON and CSV formats.

[0085] 1) Extract peak load, minimum load, and area (reflecting pump efficiency) from the dynamometer diagram.

[0086] Dynamometer Diagram Diagnosis

[0087] Raw data: load-displacement curve (binary or 2D array).

[0088] After conversion:

[0089] Extract characteristic parameters (such as maximum load MaxLoad = 45kN).

[0090] Classified into fault types (e.g. "air lock").

[0091] The output format is json and the content is as follows:

[0092] {

[0093] "diagnosis":"gas_lock",

[0094] "confidence":0.92,

[0095] "features":{"max_load":45,"min_load":12}

[0096] }

[0097] 2) Dynamic liquid level calculation

[0098] Original data: acoustic echo time series ([t1=1.2s, t2=1.3s,...]).

[0099] After conversion:

[0100] The depth is converted to liquid depth using the speed of sound formula (liquid depth = speed of sound × time / 2).

[0101] Output: {"liquid_level":856,"unit":"m"}.

[0102] The details of the diversion process are as follows:

[0103] The original video captured by the well site pumping unit, the tracked video, and the tracking results are diverted to the visualization platform; the tracking results, oil-water ratio, three electrical parameters, and the oil well status are diverted to the control system to generate control instructions; the original video and tracking results judged to be excellent are diverted to the update subsystem as new training data.

[0104] In another embodiment, the core goal of this system is to build an intelligent system based on image recognition and analysis technology, capable of real-time online monitoring and analysis of oil well operating status. By accurately identifying the start and stop status of the oil well, continuously monitoring the environment and operating conditions at the oil well site, and combining it with a reasonable oil well production system, a program can be developed to intelligently control the start and stop of the oil well.

[0105] The overall tracking design is as follows: cameras are installed in the well site to monitor the entire well site environment in real time. The cameras are connected to the main control board, which processes the video data received from the camera and then passes the processing results through the model to the control system, so that the control system can optimize business plans and issue optimization instructions.

[0106] The hardware design is as follows: The Nezha development kit is based on a credit card-sized (85x56mm) development board - Nezha. N97 processor (AlderLake-N), maximum turbo frequency 3.6GHz, UHD Graphics core GPU can achieve high-resolution display; onboard LPDDR5 memory, eMMC storage and TPM2.0, equipped with GPIO interface, support Windows and Linux operating systems, these features combined with fanless cooling method to build efficient solutions for various applications, such as automation, IoT gateway, digital signage and robotics. Figure 2 shown.

[0107] The software is designed to receive data from the camera and transmit it back to the main control board via WiFi.

[0108] The model design is as follows: We will use FairMOT as the basic model and make some modifications on it. The DLA-34 backbone network originally used by FairMOT is replaced with the YOLOv8s backbone network, while retaining the feature extraction capabilities of FairMOT, thereby comprehensively achieving a speed increase during tracking while inheriting the efficient feature extraction capabilities of FairMOT.

[0109] The basic process of implementing the multi-target tracking algorithm using the multi-target tracking model is as follows: Figure 3 As shown in the figure, the multi-target tracking algorithm performs the following steps during operation:

[0110] Data preprocessing: To ensure fairness, it is important to ensure that the training data is unbiased. This includes using diverse datasets and avoiding bias during the data collection process.

[0111] Balanced object detection: Select or design detection algorithms that can treat different types of objects equally, ensuring that all objects can be detected fairly.

[0112] Fair tracking allocation: When tracking multiple targets, computing resources and tracking attention are reasonably allocated so that each target can get a fair tracking opportunity.

[0113] Dynamic target priority adjustment: Dynamically adjust the tracking priority of targets based on scenario requirements and target behavior to maintain the fairness and effectiveness of the tracking process.

[0114] In addition, we will use the loss function to perform secondary processing on the results.

[0115] The loss function value used consists of four parts: hm_loss(L hm )、wh_loss(L wh )、offset_loss(L offset ) and id_loss(L identity ). The total loss function L total is the weighted sum of the losses of the target detection branch and the target re-identification branch, that is,

[0116] L total =1 / 2[exp(-w1)L detection +exp(-w2)L identity +w1+w2]

[0117] L detection =λ1L hm +L box

[0118] L box =λ2L offset +λ3L wh

[0119] Loss function meaning:

[0120] L total : The overall loss function value, including the loss values ​​of the two major branches of target detection and target re-identification;

[0121] L detection : The total loss of the target detection branch, including the heat map loss L hm and target regression bounding box loss L box ;

[0122] L hm : Target detection heat map loss, which is used to measure the gap between the predicted center points of all targets in the entire image and the corresponding true values;

[0123] L box : The total loss of the target detection regression bounding box, including the predicted center point offset loss L offset and the predicted bounding box aspect ratio loss L wh ;

[0124] L offset : Target detection prediction center point offset loss, used to measure the gap between the predicted center point of each target and the corresponding true value;

[0125] L wh : Target detection predicted bounding box aspect ratio loss, used to measure the gap between the predicted aspect ratio of the target and the corresponding true value;

[0126] L identity : Inter-frame object matching loss for re-identification.

[0127] Model training begins by reading data from a dataset containing video sequences and annotations, including the location of the donkey head's bounding box, the class label, and a unique ID for each object. These images are then preprocessed, such as scaling, cropping, and normalization, to ensure they are suitable for use in deep learning models. We employ the replaced backbone network to extract image features, which are then fed into an object detection head such as YOLO or Faster R-CNN for accurate object detection. During the model training phase, we use the extracted feature vectors to train our model, ensuring it can effectively identify and track objects in the video. Using a multi-object tracking model, we track the trajectory of each donkey head in the video sequence. We then overlay the tracking results on the original video frames for visualization and calculate evaluation metrics such as Mean On-Time (MOTA) and Indirect Detection Frequency (IDF1) to measure system performance. Finally, we save the tracking results as text, image, and video files. Based on the performance evaluation, we further optimize and adjust the model parameters and the entire system to improve tracking accuracy and efficiency.

[0128] Model deployment is: The environment configuration is to configure the OpenVINO environment on the Nezha development board. Model compression and optimization: Import the trained Pytorch model, use OpenVINO to convert the Pytorch model into an ONNX file, and based on the trained network topology, weights, and bias values, convert the ONNX model into a model that generates an optimized intermediate representation (IR), which contains two files: .xml and .bin. In this process, the model is converted to FP16 format to support half-precision reasoning; new inputs and outputs of the conversion model are defined to remove unnecessary parts of the model. On the OpenVINO platform, import camera data, use the compressed and optimized model, and perform inference to obtain tracking results. Tracking results, such as Figure 4 shown.

[0129] The returned result is sent to the business scheduling system. Direct data results are sent to the visualization platform, where the business scheduling system predicts parameters and the visualization platform responds in real time. Combined with PV system power generation and operation data and power forecasts, oil well production strategies are further optimized. Intelligent scheduling algorithms prioritize the use of PV power while ensuring stable and increased oil well production. When utility power is needed, production is conducted within peak electricity price ranges as much as possible, reducing production costs and enabling oilfields to implement renewable energy substitution, contributing to achieving carbon peak and carbon neutrality goals.

[0130] In another embodiment, the multi-target tracking model adopts a proportional coefficient judgment method, which solves the recognition problem caused by the characteristics of the camera capture image by using the target center point information fed back by the multi-target tracking model.

[0131] As far as this embodiment is concerned, when a camera is used to track an object, due to the perspective effect, objects closer to the camera appear larger in the picture, while objects farther away from the camera appear smaller. In addition, the instability of the camera or the jitter of the picture caused by external factors may cause slight changes in the picture position between consecutive frames, which makes it difficult to compare the position of objects. In the field of computer vision and image processing, especially in the task of multiple object tracking (MOT), the proportional coefficient judgment method is a method for solving recognition problems caused by the characteristics of the camera's captured picture. This method is particularly suitable for dealing with the phenomenon of near-large and far-small objects due to the perspective effect, as well as the effects of picture jitter. This method uses the target center point information fed back by the multi-target tracking model, specifically including

[0132] 1) Assume that the coordinates of the i-th donkey head center point at the starting time t are (x i t ,y i t )

[0133] 2) After k moments, the maximum positive moving position coordinate (x i (t+k.max) ,y i (t+k，max) )

[0134] 3) After k moments, the coordinates of the most negative moving position (x i (t+k′，min) ,y i (t+k′，min) ), k' are all within K;

[0135] 4) To unify the differences in near-field and far-field sensor head movement distances, the maximum travel distance in the axial direction is normalized:

[0136] 5) Similarly, the x-axis is normalized to the maximum travel:

[0137] 6) Finally, set the moving offset threshold a=0.02 (this value comes from the maximum value of the moving offset error predicted by the moving donkey head).

[0138] Regarding the prediction offset error of the immobile donkey head:

[0139] For the same static donkey head, during the data annotation phase, due to errors in manual or automatic annotation, the position of the annotated box may fluctuate and shift between consecutive frames. This dataset error affects model training and inference.

[0140] During inference, the model's predicted position for the same static donkey head frame can fluctuate and deviate between consecutive frames. This is known as the static donkey head prediction offset error.

[0141] For the same stationary donkey head, the center point offset algorithm is applied to calculate the normalized maximum travel of the x-axis and y-axis. The statistical method is used to obtain that the maximum value does not exceed 0.02, so it is set as the movement offset threshold.

[0142] In another embodiment, the data acquisition unit adopts dual-camera monitoring based on multi-target tracking.

[0143] As for this embodiment, during the development of the project, multiple solutions were provided for comparison. However, because each solution had certain problems, it was iterated many times. The following is a detailed analysis of the solutions.

[0144] Sensor Monitoring Solution: In the early stages of the project, we evaluated the feasibility of using sensors for real-time monitoring. This approach involves installing sensors on the beam pumping units to collect data on their operating status. While theoretically feasible, this approach presented several practical challenges. First, the cost of each sensor was relatively high, which could result in prohibitive overall costs for large-scale deployment. Second, sensors require complex communication infrastructure to transmit data, which not only increases the technical difficulty of implementation but also potentially affects the stable operation of the system. Most importantly, since each sensor can only monitor a single pumping unit, this approach has significant efficiency limitations and cannot achieve comprehensive monitoring coverage. Based on these considerations, we decided not to pursue this solution.

[0145] Camera monitoring solution based on target detection: Another solution is to use a visual monitoring system with deep learning-based target detection. By installing cameras to collect scene image data, and using target detection algorithms to identify and track the operating status of each donkey head. This method has a wider field of view and better scalability than sensor monitoring. However, this method is not without flaws. The target detection algorithm may make errors when dealing with occlusion problems, causing the prediction box (bounding boxes) to drift, thereby reducing the accuracy of detection. More importantly, it cannot take advantage of the correlation between the previous and next frames. Moreover, this method cannot distinguish between instances and has no ID number, which may lead to blurred object recognition. In view of this, we believe that although this solution has potential, it still needs further optimization to meet the project requirements.

[0146] Camera monitoring based on multi-target tracking: To overcome the challenges of target detection methods, we propose an improved solution: camera monitoring based on multi-target tracking. The key to this solution lies in its ability to not only rely on target detection within a single frame but also incorporate temporal information from video sequences. By tracking the historical motion of targets, the system can more accurately associate the identities of individual targets. This means that even in the presence of occlusion, the system can maintain accurate identification and tracking of individual targets.

[0147] Furthermore, this approach allowed us to comprehensively monitor the entire well site with fewer cameras, effectively utilizing resources and improving the efficiency of "one-to-many" monitoring. However, we did identify some potential issues, such as the potential loss of system recognition accuracy in low light or inclement weather. Therefore, we decided to further refine this solution.

[0148] Dual-camera monitoring solution based on multi-target tracking: To address the performance degradation that can occur with single-camera systems in certain environments, we have designed a more robust monitoring solution: dual-camera monitoring based on multi-target tracking. This solution combines the advantages of infrared and conventional cameras to provide stable monitoring capabilities in various lighting conditions. Infrared cameras excel in night vision or low-light environments, while conventional cameras are suitable for everyday lighting conditions. By using both types of cameras, we can ensure that the system maintains high-precision target tracking in a variety of environmental conditions. This dual-camera system not only improves monitoring reliability but also enhances the robustness of the entire monitoring system. Ultimately, we expect this comprehensive solution to meet all our real-time monitoring needs and achieve optimal monitoring results in a variety of environments.

[0149] In another embodiment, the dual camera monitoring refers to using an infrared camera in night vision or low light environments and using a normal camera in daily light conditions.

[0150] In another embodiment, a feature-based security mechanism: In security monitoring systems, using computer vision technology for real-time monitoring and automatic identification of people and objects in the footage is a key means of improving security. The multi-target tracking technology of the present invention can accurately track multiple targets in a video stream and extract their features. These features typically include information such as the target's location, size, color, and motion trajectory. By integrating the feature values ​​returned by the present invention, a trusted database can be constructed to store and identify targets marked as safe. The algorithm identifies and tracks people or objects in the video and extracts features for each target, such as appearance, shape, size, color, and speed. Based on this, we select which targets (people or objects) should be deemed trustworthy based on the scenario to be protected and specific requirements. The feature information of these trusted targets is entered into the database, creating a "whitelist" dataset. When new footage enters the monitoring system, the algorithm developed by the present invention based on the improved FairMOT is used again for feature extraction, and the extracted features are compared with those in the trusted database. If the system detects a target not in the database, it is considered an unsafe unit and an alarm is immediately triggered. Once a non-safe unit is detected entering the collection screen, the system can issue an alarm through sound, light, message push, etc.

[0151] In another embodiment, the data processing unit processes the collected data based on the multi-target tracking model carried by the development board, and the model framework is as follows: Figure 5 shown.

[0152] The network's connections to the output layer are fully connected, while the connections between hidden layers are residual. Features of different dimensions are acquired through upsampling and downsampling. Detection and tracking are performed simultaneously, with the output layer outputting four branch tasks: RE-ID for tracking, and offsets, heatmaps, and bounding box size for detection.

[0153] RE-ID (Re-Identification): refers to the technology of identifying the same target (such as pedestrians and vehicles) across cameras or scenes.

[0154] Offsets: In object detection, it refers to the small position adjustment of the predicted box (Bbox) relative to a reference point (such as the grid center or anchor box).

[0155] Heatmaps: A two-dimensional probability map that represents the probability distribution of objects (such as key points and object centers) in an image.

[0156] Bbox Size (Bounding Box Size): The width and height of the rectangular box used in target detection.

[0157] Finally, it should be noted that, under the guidance of this specification and without departing from the scope of protection of the claims of the present invention, ordinary technicians in this field can also make many kinds of changes in form, which are all protected by the present invention.< / id> < / class> < / cos> < / id> < / cos> < / id>

Claims

1. A deep learning-based pumping unit motion state monitoring system, comprising a data acquisition unit, a data processing unit, a model update subsystem, a monitoring system, a visualization platform, and a business scheduling system, wherein: The data acquisition unit uses a camera to collect pumping unit data; The data processing unit processes the collected data based on the multi-target tracking model carried by the development board; The visualization platform receives real-time data streams and provides users with immediate job status information; The business scheduling system generates instructions based on the operation status information and monitoring data from non-camera equipment in the monitoring system. The instructions are sent to the weak current controller, which then controls the strong current controller through the weak current controller to achieve accurate scheduling of well site equipment; The model updating subsystem performs online data analysis and trains the multi-target tracking model; The monitoring system monitors various indicators of the operation of the oil pumping unit in real time.

2. The monitoring system according to claim 1, wherein the operation status information preferably includes the operating status of the oil well, efficiency indicators and any immediate alarms or notifications.

3. The monitoring system according to claim 1, wherein the monitoring data includes a dynamic liquid level, an oil-water mixture ratio, and a touch result. 4 . The monitoring system according to claim 1 , wherein the precise scheduling of well site equipment includes starting, stopping and adjusting pumps.

5. The monitoring system according to claim 1, wherein the model updating subsystem continuously updates and optimizes the multi-target tracking model through the collected historical and real-time data.

6. The monitoring system according to claim 1, wherein the model updating subsystem performs long-term data storage to provide resources for future data analysis and knowledge extraction.

7. The monitoring system according to claim 1, wherein the multi-target tracking model adopts a proportional coefficient judgment method, which solves the recognition problem caused by the characteristics of the camera acquisition image by utilizing the target center point information fed back by the multi-target tracking model.

8. The monitoring system according to claim 1, wherein the data acquisition unit adopts dual-camera monitoring based on multi-target tracking.

9. The monitoring system according to claim 8, wherein the dual-camera monitoring refers to using an infrared camera in night vision or low-light environments and a normal camera in daily lighting conditions.

10. The monitoring system according to claim 1, wherein the specific steps for completing the motion status monitoring of the oil pumping unit using the monitoring system are as follows: S100: Equipment in the monitoring system continuously monitors the well site, capturing key data about the well's operating status; S200: The processor processes the key data and converts it into a format that can be used by the monitoring system; S300: The processed data is distributed to the visualization platform, the control system, and the model update subsystem, wherein the control system includes the business scheduling system, the weak current control system, and the strong current control system; S400: The visualization platform provides users with real-time information to assist with daily monitoring and management; S5 00: The control system generates control commands based on the received data and preset parameters, and transmits them to the strong current controller through the weak current controller; S600: The high-voltage controller executes control commands and operates well site equipment; S700: The model updating subsystem uses the received data to perform deep learning and model training to continuously optimize the control strategy; S800: The optimized control strategy and parameters are fed back to the control system.

Images