An aerial-ground cooperative oilfield inspection intelligent agent global visual target tracking and closed-loop disposal method and device

Abstract

The application discloses an air-ground cooperative oil field inspection intelligent agent global visual target tracking and closed-loop treatment method and system, belongs to the technical field of oil field inspection, computer vision and intelligent processing, and aims to solve the problems of non-uniform coordinate system in existing air-ground inspection, discontinuous tracking of cross-device targets, low target matching accuracy and lack of closed-loop management of abnormal treatment process. The application proposes the following solutions: constructing unified format air-ground inspection data, establishing the mapping relationship between pixel space and geographic space to realize cross-device space alignment, constructing a target identification system based on unified space information to realize global continuous tracking, fusing visual features and geographic space features to complete cross-device target association, identifying abnormalities based on target association results and executing the closed-loop treatment process of detection, reporting, scheduling, treatment and feedback, and finally realizing the visualization and traceability of inspection and treatment results, which is suitable for unmanned aerial vehicle and ground device cooperative inspection and abnormal automatic treatment in an oil field.

Description

Technical Field

[0001] It involves the fields of oilfield safety production inspection, computer vision, cross-equipment target tracking, geographic coordinate transformation, intelligent closed-loop processing, and human-computer interaction, specifically involving air-ground collaborative oilfield inspection intelligent agents for full-domain visual target tracking and closed-loop processing. Background Technology

[0002] As oilfield safety production management moves towards intelligent and unmanned operations, air-ground collaborative inspection technology is gradually becoming an important means of oilfield monitoring. Existing technologies typically use drones to acquire large-scale overhead images, combined with fixed ground cameras or inspection robots for detailed monitoring of specific areas, thereby achieving comprehensive perception of oilfield equipment operating status, environmental anomalies, and personnel behavior. In target detection and tracking, existing solutions generally employ deep learning-based visual detection algorithms to identify targets such as oil leaks, equipment damage, and personnel intrusion, and complete target tracking within a single equipment viewpoint. For spatial positioning, some technologies incorporate GPS information to locate inspection equipment, achieving rough target location marking. Regarding anomaly handling, some systems use monitoring platforms to implement anomaly alarms and manual dispatch, assisting maintenance personnel in on-site handling. Furthermore, at the data level, some research improves model training effects by constructing datasets or introducing simulation data augmentation methods.

[0003] However, existing technologies are still mainly based on "single-device independent operation" or "weakly coupled collaboration," with target tracking, spatial positioning, and anomaly handling often scattered across different modules or systems, lacking a unified global collaborative mechanism. For example, in cross-device target tracking, most solutions only maintain target IDs within their respective devices, failing to establish a unified global identification system, resulting in the inability to continuously associate the same target across different devices; in spatial coordinate processing, there is a lack of a high-precision bidirectional conversion mechanism between visual pixel coordinates and geographic coordinates, making it difficult to uniformly calibrate target positions across devices; in target matching, matching typically relies solely on visual features, which is easily affected by factors such as lighting and occlusion in complex environments, resulting in low matching accuracy; in anomaly handling, most systems remain at the "detection-alarm" stage, lacking a standardized closed-loop process covering reporting, scheduling, execution, and feedback, leading to insufficient handling efficiency and traceability; in terms of data support, obtaining real inspection data is difficult and annotation costs are high, and existing solutions lack a mechanism for fusing synthetic and real data, making it difficult to meet the needs of model training and system testing.

[0004] In summary, existing technologies suffer from several drawbacks, including the lack of a unified coordinate system and high-precision conversion mechanism between air and ground equipment, the inability to establish a globally consistent continuous tracking relationship between cross-device targets, insufficient robustness due to the reliance on a single visual feature for target matching, the lack of standardized closed-loop management in anomaly handling processes, and insufficient supply of inspection data with a single processing method. Summary of the Invention

[0005] To address the shortcomings of existing technologies, such as the lack of a unified coordinate system and high-precision conversion mechanism between air and ground equipment, the inability to establish globally consistent continuous tracking relationships across equipment targets, insufficient robustness due to reliance on single visual features for target matching, the lack of standardized closed-loop management in anomaly handling procedures, and insufficient and simplistic processing methods for inspection data, the technical solution provided by this invention is as follows: A method for full-domain visual target tracking and closed-loop processing of an intelligent agent for oilfield inspection, comprising: The steps include: acquiring inspection area parameters, air and ground inspection equipment parameters, and inspection data; generating air and ground inspection data in a unified format; and performing standardized processing. Based on the air-ground inspection data and equipment parameters, the steps are as follows: establish a mapping relationship between pixel space and geographic space and perform cross-device spatial alignment to obtain target location information with unified spatial representation. Based on the target location information expressed in a unified spatial manner, the steps are as follows: target identification is performed on inspection data and a target identification system is constructed. Cross-device target identification association is established to obtain continuous target tracking results across the entire domain. Based on the target tracking results, the steps are as follows: extract multi-source features of the target and perform fusion processing, perform cross-device target association determination and calibrate the target identifier to obtain consistent target association results across devices; Based on the target association results, the steps are as follows: identify abnormal targets and generate abnormal events, execute the abnormal handling process according to preset rules and complete the status update to obtain the abnormal closed-loop handling results. Based on the target tracking results and anomaly closed-loop handling results, the execution results are displayed and data is recorded to achieve visualization and traceability of the inspection and handling processes.

[0006] Furthermore, in a preferred embodiment, the air-to-ground inspection data is constructed by fusing synthetic data and real data to form a unified data format.

[0007] Furthermore, in a preferred embodiment, pixel space and geographic space are bidirectionally converted through camera parameters and spatial mapping relationships.

[0008] Furthermore, in a preferred embodiment, cross-device spatial alignment is based on device positioning and time synchronization and dynamic calibration.

[0009] Furthermore, in a preferred embodiment, the target identifier is unified across devices through a mapping between local and global identifiers.

[0010] Furthermore, in a preferred embodiment, target association is achieved based on the fusion of visual features and geospatial features.

[0011] Based on the same inventive concept, this invention also provides an air-ground collaborative oilfield inspection intelligent agent system for full-domain visual target tracking and closed-loop processing, comprising: This module acquires inspection area parameters, air and ground inspection equipment parameters, and inspection data, generates air and ground inspection data in a unified format, and performs standardized processing. Based on the air-ground inspection data and equipment parameters, a module is established to create a mapping relationship between pixel space and geographic space and to perform cross-device spatial alignment to obtain target location information with a unified spatial representation. Based on the target location information expressed in a unified spatial format, this module identifies targets in inspection data, constructs a target identification system, establishes cross-device target identification associations, and obtains continuous target tracking results across the entire domain. Based on the target tracking results, a module is used to extract multi-source features of the target and perform fusion processing, perform cross-device target association determination and calibrate the target identifier to obtain a consistent target association result across devices; Based on the target association results, the module identifies abnormal targets and generates abnormal events, executes the abnormal handling process according to preset rules and completes the status update to obtain the abnormal closed-loop handling result; Based on the target tracking results and the abnormal closed-loop handling results, the module displays the execution results and records data to realize the visualization and traceability of the inspection and handling process.

[0012] Based on the same inventive concept, the present invention also provides a computer storage medium for storing a computer program, wherein when the computer program is read by a computer, the computer executes the method described thereon.

[0013] Based on the same inventive concept, the present invention also provides a computer, including a processor and a storage medium, wherein when the processor reads a computer program stored in the storage medium, the computer executes the method described thereon.

[0014] Based on the same inventive concept, the present invention also provides a computer program product, which, when executed, implements the method described.

[0015] Compared with the prior art, the advantages of the technical solution provided by the present invention are as follows: The dual-source inspection data processing mechanism, through the collaborative design of synthetic data generation and standardized loading of real data, originates from the data generation and preprocessing process in the dual-source inspection data processing module. This ensures that the inspection data is unified in terms of type, format, and annotation. At the same time, it constructs diverse training samples through environmental parameter modeling. Compared with the dataset construction method that relies solely on real data, it effectively solves the problems of sample scarcity and uneven distribution, thereby improving the model training stability and generalization ability and providing a high-quality data foundation for subsequent full-domain tracking and anomaly handling.

[0016] The global coordinate unification mechanism, through pixel coordinate distortion correction, bidirectional conversion between pixel coordinates and GPS geographic coordinates, and cross-device coordinate calibration process, originates from the intrinsic parameter calibration and projection transformation design in the global coordinate unification module. It unifies the target location information obtained by different devices under the same spatial reference. Compared with the existing methods that only perform device self-positioning or unidirectional coordinate mapping, it significantly improves spatial positioning accuracy and cross-device position consistency, thereby providing reliable spatial constraints for target cross-device association and reducing position error propagation.

[0017] The cross-device global target tracking mechanism outputs unified results through local ID and global ID mapping rules and cross-device tracking results. It originates from the global identifier construction and tracking execution process in the cross-device global target tracking module, ensuring that the same target maintains a consistent identity across different devices. Compared with the target ID fragmentation problem caused by traditional independent tracking by each device, it realizes the continuous expression of the target in multi-source air and ground data, thereby significantly improving tracking continuity and data fusion.

[0018] The visual and GPS fusion feature matching mechanism, through the weighted fusion of visual depth features and spatial features and the association determination based on similarity calculation, originates from the feature extraction and fusion design in the cross-device feature matching module. This allows target matching to rely on both appearance and spatial information. Compared with matching methods based solely on visual features, it maintains higher robustness in complex environments such as sandstorms and changes in lighting, thereby improving the accuracy of cross-device target matching and enhancing the stability of tracking results.

[0019] The closed-loop process for handling anomalies employs a standardized five-stage execution mechanism encompassing detection, reporting, scheduling, handling, and feedback. This mechanism stems from the process construction and state transition design within the closed-loop anomaly handling module, creating a complete chain from anomaly discovery to resolution. Compared to existing methods that only have alarm or semi-manual handling processes, this approach automates and standardizes the handling process, significantly improving anomaly handling efficiency and ensuring full traceability.

[0020] The intelligent scheduling mechanism for handling resources, through the matching rules between anomaly types and handling resources, is derived from the scheduling strategy design in the anomaly closed-loop handling module. This enables different types of anomalies to automatically match the optimal execution subject. Compared with the manual experience-driven scheduling method, it improves resource utilization efficiency and response speed, thereby reducing handling delays and improving processing accuracy.

[0021] The status statistics and real-time update mechanism, through status indication functions and periodic update strategies, originates from the status statistics and monitoring design in the anomaly closed-loop handling module. This enables all anomalies to have quantifiable status identifiers at different stages. Compared to systems lacking unified status management, this achieves transparent monitoring of the entire process, thereby enhancing the system's controllability and manageability in the anomaly handling process.

[0022] The cross-device real-time coordinate calibration and error closed-loop control mechanism, through inter-frame deviation correction and error threshold control, originates from the dynamic calibration design in the global coordinate unified module. This enables the device to maintain spatial consistency during dynamic inspection. Compared with static calibration methods, it effectively suppresses error accumulation, thereby ensuring the stability of target positioning and tracking results in the time dimension.

[0023] The lightweight target detection mechanism for oilfield scenarios, through category weight optimization and algorithm structure simplification, originates from the detection optimization design in the cross-device target tracking module. This enables the algorithm to adapt to the computing power limitations of UAVs and ground equipment. Compared with general detection models, it balances detection accuracy and operating efficiency, thereby achieving real-time target recognition and tracking in edge device environments.

[0024] The integrated visualization and interaction mechanism integrates data management, tracking and display, and status visualization. It originates from the interface and function integration design in the visualization and interaction module, which enables the system operation results to be presented intuitively and supports operation interaction. Compared with the decentralized interface system, it improves the efficiency of operation and information acquisition, thereby reducing the cost of manual intervention and improving the level of inspection management.

[0025] The system initialization and parameter configuration mechanism, by uniformly loading device parameters, threshold rules, and mapping table structures, originates from the initialization process design in the implementation steps. This ensures that the system has a complete configuration environment before running. Compared with systems without a unified initialization mechanism, this reduces operational uncertainty and thus ensures the stability of subsequent collaborative execution of various modules.

[0026] The data-driven frame-level association mechanism organizes and associates dual-source inspection data frame by frame. This is derived from the data organization design in the data processing steps, which keeps the air and ground data consistent in both time and space dimensions. Compared with loose data processing methods, this improves data utilization efficiency and enhances cross-device analysis and processing capabilities.

[0027] The results visualization and logging mechanism, through tracking results drawing and processing log storage, and the data recording and output design from the results display steps, makes the entire inspection process and anomaly handling traceable. Compared with systems without records or with incomplete records, it improves the system's auditing capabilities and historical analysis capabilities, thereby providing data support for subsequent optimization and decision-making.

[0028] It is suitable for use in oilfield scenarios for full-area inspection, continuous tracking of abnormal targets and automated closed-loop handling of drones and ground equipment. Attached Figure Description

[0029] Figure 1 This is a schematic diagram of the overall system architecture; Figure 2This is a schematic diagram of the core processing flow. Detailed Implementation

[0030] To make the advantages and benefits of the technical solution provided by the present invention clearer, the technical solution provided by the present invention will now be described in further detail with reference to the accompanying drawings, specifically: Implementation Method 1: This implementation method provides a method for all-domain visual target tracking and closed-loop processing of an intelligent oilfield inspection agent based on air-ground collaboration, including: The steps include: acquiring inspection area parameters, air and ground inspection equipment parameters, and inspection data; generating air and ground inspection data in a unified format; and performing standardized processing. Based on the air-ground inspection data and equipment parameters, the steps are as follows: establish a mapping relationship between pixel space and geographic space and perform cross-device spatial alignment to obtain target location information with unified spatial representation. Based on the target location information expressed in a unified spatial manner, the steps are as follows: target identification is performed on inspection data and a target identification system is constructed. Cross-device target identification association is established to obtain continuous target tracking results across the entire domain. Based on the target tracking results, the steps are as follows: extract multi-source features of the target and perform fusion processing, perform cross-device target association determination and calibrate the target identifier to obtain consistent target association results across devices; Based on the target association results, the steps are as follows: identify abnormal targets and generate abnormal events, execute the abnormal handling process according to preset rules and complete the status update to obtain the abnormal closed-loop handling results. Based on the target tracking results and anomaly closed-loop handling results, the execution results are displayed and data is recorded to achieve visualization and traceability of the inspection and handling processes.

[0031] The air-to-ground inspection data is constructed by fusing synthetic data and real data to form a unified data format.

[0032] Pixel space and geographic space can be converted bidirectionally through camera parameters and spatial mapping.

[0033] Cross-device spatial alignment is based on device positioning and time synchronization and dynamic calibration.

[0034] The target identifier achieves cross-device consistency through mapping between local and global identifiers.

[0035] Target association is achieved by fusing visual features and geospatial features.

[0036] Based on the same inventive concept, this invention also provides an air-ground collaborative oilfield inspection intelligent agent system for full-domain visual target tracking and closed-loop processing, comprising: This module acquires inspection area parameters, air and ground inspection equipment parameters, and inspection data, generates air and ground inspection data in a unified format, and performs standardized processing. Based on the air-ground inspection data and equipment parameters, a module is established to create a mapping relationship between pixel space and geographic space and to perform cross-device spatial alignment to obtain target location information with a unified spatial representation. Based on the target location information expressed in a unified spatial format, this module identifies targets in inspection data, constructs a target identification system, establishes cross-device target identification associations, and obtains continuous target tracking results across the entire domain. Based on the target tracking results, a module is used to extract multi-source features of the target and perform fusion processing, perform cross-device target association determination and calibrate the target identifier to obtain a consistent target association result across devices; Based on the target association results, the module identifies abnormal targets and generates abnormal events, executes the abnormal handling process according to preset rules and completes the status update to obtain the abnormal closed-loop handling result; Based on the target tracking results and the abnormal closed-loop handling results, the module displays the execution results and records data to realize the visualization and traceability of the inspection and handling process.

[0037] A computer storage medium is also provided for storing a computer program, which, when read by the computer, executes the method.

[0038] A computer is also provided, including a processor and a storage medium, wherein the computer executes the method when the processor reads a computer program stored in the storage medium.

[0039] A computer program product is also provided, which, when executed, implements the method described.

[0040] Implementation Method Two: This implementation method is a further detailed description of the technical solution provided in Implementation Method One, specifically: This embodiment provides a method for air-ground collaborative oilfield inspection intelligent agent full-domain visual target tracking and closed-loop handling. This method is designed to meet the needs of collaborative perception, unified positioning, continuous tracking and automated anomaly handling among UAVs, ground fixed cameras and ground inspection robots in oilfield inspection. The whole system is executed in the following order: system initialization and parameter configuration, dual-source inspection data generation or loading and preprocessing, full-domain coordinate unified parameter calibration and initialization, cross-device full-domain target tracking execution, cross-device target feature matching and association, fully closed-loop automated execution of anomaly handling, and visualization and data recording of tracking and handling results. The data, parameters or intermediate results output by the previous step are used as inputs for the next step, thus forming a complete air-ground collaborative inspection closed loop.

[0041] The first step is system initialization and parameter configuration. This step aims to establish the basic operating environment for the entire air-ground collaborative inspection system, set core parameters, and initialize various operational objects, providing a unified software and business foundation for subsequent dual-source data processing, coordinate transformation, target tracking, and anomaly closed-loop handling. In specific implementation, a data processing module, coordinate unification module, target tracking module, feature matching module, anomaly closed-loop handling module, and visualization interaction module are deployed in the oilfield inspection control system. Communication interfaces, calling sequences, and message passing rules between these modules are established. The control system can be deployed on the oilfield control center server or on edge computing nodes with graphics processing capabilities. In practical applications, a centralized deployment or edge collaborative deployment method can be selected based on the inspection scale. After completing module deployment, the hardware environment required for system operation is configured, including the central processing unit, graphics processor, storage devices, network communication components, and data access interfaces for receiving data streams from UAVs and ground equipment. For UAVs, their flight control information, onboard camera image streams, positioning information, and attitude information are accessed. For ground-based fixed equipment, their video streams, equipment installation coordinates, and orientation parameters are accessed. For ground inspection robots, their mobile positioning information, attitude information, and vehicle-mounted camera image streams are accessed. Subsequently, the geographical boundary information of the oilfield inspection area is defined. This geographical boundary information can be represented using latitude and longitude fences, polygonal work areas, or station / reservoir partition boundaries to constrain the working scope of subsequent data generation, coordinate mapping, anomaly screening, and resource scheduling. Further, the intrinsic camera parameters corresponding to the UAVs and ground equipment are defined, including focal length parameters, principal point parameters, and distortion parameters. Attributes such as equipment type identifier, equipment number, equipment installation height, equipment orientation, image resolution, and whether the equipment is movable are also configured. To ensure the stable execution of subsequent target detection, feature matching, and anomaly recognition processes, target detection confidence thresholds, cross-device target similarity thresholds, anomaly judgment thresholds, scheduling strategy rules, status update cycles, and log storage rules also need to be set. The system includes several key components: a target detection confidence threshold to filter out low-confidence targets; a cross-device target similarity threshold to determine whether objects detected from different perspectives (air and ground) belong to the same real target; an anomaly detection threshold to map tracked targets as risk targets such as oil leaks, equipment damage, or unauthorized personnel intrusion; and scheduling strategy rules to automatically match resources for handling based on anomaly type, risk level, and spatial location. Finally, it initializes a global target identifier mapping table, an anomaly event queue, an anomaly ledger, a handling work order cache, a handling status statistician, and a log database. The global target identifier mapping table stores the correspondence between local target identifiers and globally unique target identifiers; the anomaly event queue caches event objects to be reported, scheduled, and handled in chronological order; the anomaly ledger records the entire event flow information; and the handling status statistician periodically counts the number of events in each status.After completing this step, output the system's basic configuration parameter set, device parameter set, threshold rule set, as well as the initialized global mapping table and event management object, which will serve as the input basis for the next step of generating or loading and preprocessing dual-source inspection data.

[0042] The next step involves generating or loading and preprocessing dual-source inspection data. The purpose of this step is to obtain air-ground dual-source inspection data in a unified format, directly usable for subsequent coordinate unification, target detection, cross-device tracking, and anomaly handling. In practice, the data source method is first selected through a visual interactive interface or configuration file. When the system is in the algorithm training, functional integration, or lacks sufficient real data, synthetic data generation can be selected; when the system enters the actual deployment or effect verification stage, real inspection data loading can be selected; when it is necessary to balance training generalization ability and real-scene adaptation ability, a mixed processing method of synthetic and real data can also be used. If synthetic data generation is selected, the system automatically generates inspection samples from both the UAV's perspective and the ground equipment's perspective, based on the oilfield area boundary, number of devices, device type, flight altitude range, ground view parameters, and environmental interference parameters set in the first step. Specifically, target objects are randomly generated or distributed according to a preset density within the oilfield operation area. These target objects include normal facility targets and abnormal facility targets, where abnormal facility targets may include oil leak areas, equipment damage areas, pipeline anomaly points, and areas where personnel have illegally entered. Then, image samples, equipment location information, target bounding box information, and target type annotation information corresponding to their respective viewpoints are generated for both the UAV and ground equipment. During the generation process, common external interference factors in oilfield inspection scenarios are simulated simultaneously, including strong light, low light, sand and dust, rain and fog, partial occlusion, lens shake, and equipment installation errors, to enhance the adaptability of subsequent models to complex environments. For each frame of synthesized sample, the frame number, equipment number, equipment type, image data, equipment geographic coordinates, equipment attitude parameters, target bounding box, target category label, and the geographic coordinate label corresponding to the target are output, thus forming a standardized inspection sample. If the real data loading method is selected, the system automatically reads images captured by the UAV, video frames from the fixed camera, images collected by the inspection robot, and their corresponding positioning metadata and annotation files from the preset data directory. During loading, the image files and metadata files are parsed first, then the target annotation files are read, the validity of the annotation bounding boxes is checked, abnormal annotation formats are corrected or removed, the image size is normalized, pre-correction processing is performed on the original images with obvious lens distortion, and annotation information from different sources and in different formats is converted into a unified data structure. For drone images, it is also necessary to correlate information such as latitude and longitude, altitude, pitch angle, heading angle, and roll angle at the time of flight; for fixed ground equipment, it is also necessary to read its installation location, installation height, and shooting direction; for ground inspection robots, it is also necessary to read its displacement trajectory, vehicle attitude, and inspection path information. When using a hybrid data processing method, synthetic data and real data are fused according to a preset ratio to ensure that the output dataset maintains a balance in terms of target type distribution, interference condition distribution, and equipment perspective distribution.After all processing is complete, the dual-source data is encapsulated according to unified fields to form a frame-level air-to-ground collaborative inspection dataset. Each record in this dataset includes at least the frame number, device type, device number, acquisition time, image data, device geographical location, device attitude, target annotation information, and subsequent expandable fields. After completing this step, a standardized frame-level air-to-ground collaborative inspection dataset is output as input for the next step of global coordinate unified parameter calibration and initialization.

[0043] A second global coordinate unified parameter calibration and initialization is performed. The purpose of this step is to establish a unified spatial reference between the UAV and ground equipment, enabling target information perceived by different devices to be expressed and aligned in the same geographic space, thus laying the spatial foundation for subsequent cross-device target association and continuous tracking. In practice, the device parameters, image data, device positioning information, and device attitude information from the frame-level air-to-ground collaborative inspection dataset output in the previous step are first read, and the corresponding camera parameter library is loaded for each device. For UAVs and different types of ground equipment, dedicated parameter templates can be established, containing lens focal length, image center position, distortion correction parameters, installation angle compensation parameters, and device height information. Then, based on the correspondence between pixel positions in the image and the device imaging model, distortion correction processing is performed on the target positions in the image. This distortion correction processing is not expressed through explicit formulas, but rather by using the device focal length, principal point position, and distortion correction parameters to reverse-correct the target pixels, making them correspond to a normalized position closer to the actual optical imaging path. After distortion correction, the device's own geographic coordinates, altitude, and attitude parameters are used as spatial references to convert the target pixel position in the image into a planar offset of the target relative to the device. For UAVs, this offset also needs to be mapped to the ground in conjunction with the flight altitude; for fixed ground equipment, its altitude can be considered as zero altitude or a fixed installation altitude, and the target's projected position on the ground plane is determined by the device's orientation and installation tilt angle. Then, through a geographic coordinate projection transformation mechanism, the device's latitude and longitude and the target's planar offset information relative to the device are mapped to a unified planar coordinate system, and further converted to standard geographic coordinates to obtain the target's geographic location. In addition to performing the forward conversion from pixel to geographic coordinates, a reverse conversion capability from geographic coordinates to pixel position also needs to be established. This reverse conversion capability is mainly used to map the target position identified by the UAV to the ground equipment image, or to map the target position identified by the ground equipment to the UAV image, to achieve mutual calibration of perspectives and regional linkage tracking between air and ground equipment. To ensure real-time performance during air-ground collaboration, this step also configures a cross-device coordinate real-time calibration mechanism. Specifically, the observation results from different devices are time-series aligned based on device timestamps, positioning times, attitude update times, and image acquisition times. Deviations in target geographic locations at the same or adjacent times are estimated, and the coordinate conversion results for the current frame are continuously corrected based on historical frame deviation trends. When a reprojection error exceeds a preset threshold during the first frame or periodic calibration, a secondary calibration of device intrinsic parameters or an attitude compensation update is triggered to ensure subsequent mapping accuracy. Through this step, all targets detected by all devices can be uniformly expressed as global geographic coordinates, and the system also has the ability to project these global geographic coordinates back to the device image location.After completing this step, output inspection frame data with unified geographic location identifiers, equipment coordinate transformation model, and cross-equipment coordinate calibration parameters as input for the next step of cross-equipment global target tracking.

[0044] Then, cross-device global target tracking is performed. The purpose of this step is to identify oilfield targets in inspection images collected by UAVs and ground equipment, and to achieve continuous cross-device tracking based on a globally unique identifier mechanism, avoiding the splitting of the same target into multiple unrelated objects in different devices. In specific implementation, the image data and equipment information corresponding to the current processing frame are first read from the inspection frame data with unified geographical location identifiers output in the previous step, and lightweight target detection of the oilfield scene is performed on the UAV image and the ground equipment image respectively. The lightweight target detection can optimize the category weights for common target types in oilfield inspections, making the model pay more attention to high-value targets such as oil leak areas, equipment damage points, personnel intrusion behavior, and abnormal facility components. In actual execution, the model input is a standardized image, and the output is a set of detection results. Each detection result includes at least the target bounding box, target category, target confidence, and local location in the image. To ensure the stability of subsequent tracking, candidate results below the detection confidence threshold are first filtered out, and then local identifiers are assigned to the remaining detected targets. Local identifiers can be generated using a combination of device number and local serial number, ensuring each device maintains a unique local target ID within its own time frame. Next, the system combines the global geographic coordinates obtained in the previous step to determine the spatial continuity, category consistency, and temporal proximity of targets in the current frame and historical frames. When a current target is spatially close to a known historical target, matches its category, and is temporally continuous, and meets the cross-device similarity association condition, it is considered an existing global target and inherits that global target identifier. When a current target cannot establish a reliable correspondence with any existing global target, a new globally unique target identifier is assigned to it and written into the global target identifier mapping table. In this way, local targets from both UAVs and ground equipment are uniformly mapped to the same global target identifier system, thus ensuring consistency of identity for the same real target from both aerial and ground perspectives. To avoid target fragmentation due to tracking loss or short-term occlusion, the system can also set a short-term memory window to retain its global identifier and most recent state when a target is not observed for a short period. When a matching target is subsequently detected again, the original global identifier is reused. After processing each frame, output cross-device tracking results with a global target identifier. The results include at least the global target identifier, device type, device number, frame number, image bounding box, target category, target confidence score, target geographic coordinates, and timestamp. After completing this step, output a set of cross-device tracking results with a globally unified identifier, which will serve as input for the next step of cross-device target feature matching and association.

[0045] Next, cross-device target feature matching and association are performed. The purpose of this step is to further improve the accuracy of target association between different devices by utilizing the fusion of visual and geospatial features, building upon existing global target tracking, especially improving the reliability of target identification under complex conditions such as lighting changes, dust occlusion, and significant viewing angle differences. In practice, candidate associated target pairs are first extracted from the cross-device tracking results set output in the previous step. These candidate pairs are initially screened based on temporal synchronization range, spatial proximity, and target category consistency. For example, only drone targets and ground device targets that are temporally close, geographically similar, and of the same or compatible category are considered for subsequent association. For each candidate associated target pair, the target region is cropped from the original image based on the corresponding bounding box and input into a feature extraction network to obtain the target's visual depth features. Visual depth features are used to describe the target's texture, contour, color distribution, local structure, and appearance pattern. Simultaneously, the target's global geographic coordinates, device location, and relative spatial relationship are read to construct a spatial feature representation of the target. The spatial features can include the target's latitude and longitude, its planar distance relative to the equipment, its relationship with the inspection path, its continuity with historical observation locations, and its relationship with the topological location of oilfield facilities. Subsequently, the visual depth features and spatial features are input into a fusion module for joint processing. The fusion module dynamically adjusts the importance weights of the two types of features based on current environmental conditions. For example, it enhances the influence of visual features in clear weather and with clear images, and increases the influence of spatial features in dusty, low-light, or blurry conditions. The fusion process yields a normalized joint feature representation of the target. Then, a similarity calculation is performed on the joint features between the UAV target and the ground equipment target. The higher the similarity, the greater the likelihood that the two belong to the same real target. To further improve the accuracy of the determination, a calibration layer can be added after the similarity calculation to perform scenario-based corrections on the original similarity results. For example, compensation can be made based on the current equipment status, weather conditions, target category confusion risk, and historical association success rate. Finally, when the similarity result exceeds a preset threshold, the two candidate targets are determined to belong to the same real target, and their global target identification is confirmed or corrected; when the similarity result is below the threshold, the independent identification relationship in the original tracking result is retained. This step further eliminates misassociations caused by relying solely on spatial location or appearance features, improving the stability of continuous cross-device target tracking. After this step, the cross-device target association results, validated through fusion matching and calibration, are output as input for the next step of fully automated, closed-loop anomaly handling.

[0046] Subsequently, a fully automated closed-loop anomaly handling process is implemented. The purpose of this step is to automatically identify abnormal objects during oilfield inspections based on the globally tracked and precisely correlated target results, and to complete closed-loop management from discovery to handling feedback according to standardized procedures. In specific implementation, the cross-device target correlation results output from the previous step are first subjected to anomaly screening. Anomaly screening can be completed based on target category, target appearance characteristics, target behavior patterns, location, duration, risk level, and business rules. For example, when the target category is an oil leak area and its location is in a critical area of ​​the oil pipeline, it can be identified as an oil leak anomaly; when the target category is an equipment damage area and it persists for multiple frames, it can be identified as an equipment damage anomaly; when the target category is personnel and its trajectory enters a restricted or dangerous area, it can be identified as a personnel trespassing anomaly. For each target identified as abnormal, the system generates a standardized anomaly event form. The anomaly event form includes at least the anomaly event number, corresponding global target identifier, anomaly type, risk level, detection time, detection equipment, target geographical location, on-site image evidence, detection confidence level, and suggested handling resource type. After an event ticket is generated, it is immediately written to the abnormal event queue and abnormal ledger, and its handling status is set to "detected". Next, the abnormal reporting process is executed. The system automatically sends the abnormal event ticket to the pending queue of the oilfield inspection control system, completing information registration, message notification, and queue caching without manual intervention, and updating the status to "reported". After reporting, the system enters the handling scheduling phase. During handling scheduling, the system automatically matches the most suitable handling resources based on the abnormality type, risk level, event geographical location, current available resource distribution, and response priority rules. For example, for oil leak anomalies, explosion-proof ground inspection robots can be prioritized for on-site verification and initial isolation; for equipment damage anomalies, the nearest maintenance team can be prioritized; for personnel intrusion anomalies, on-site security personnel can be notified first, and adjacent camera equipment can be linked for continuous tracking. After matching, the system generates a handling work order and sends it to the corresponding handling terminal. The work order includes the abnormality location, abnormality type, risk description, on-site images, route suggestions, target identification, and handling requirements, and updates the status to "scheduled". Upon receiving the work order, the handling resource executes the on-site handling operation. If a robotic terminal is used, it can automatically plan the path from the current location to the target location and perform actions such as arrival, image capture, confirmation, and processing. If a human terminal is used, personnel will arrive on-site according to the terminal's prompts to perform the handling operation and transmit the processing progress and results back via handheld devices, mobile terminals, or the central control system. During the handling process, the system continuously receives progress information, verification images, personnel feedback, and additional detection data, and keeps the status updated synchronously. When the on-site handling is completed, the handling terminal uploads the handling result, verification images, handling personnel information, and completion time, and the central control system updates the status to "handled".Finally, in the feedback and closure phase, the central control system automatically verifies and manually confirms the returned results. When it is confirmed that the anomaly has been eliminated and the on-site status has returned to normal, the event status is updated to "feedback completed" or "closed-loop completed," and the entire process information is archived and saved. For high-risk events that fail to complete closure within the specified time limit, the system can trigger continuous highlighting, timeout alarms, superior-level escalation scheduling, or secondary dispatch strategies to ensure that the anomaly handling process remains under control. This step also requires periodic statistics on the handling status, calculating the number of currently detected, reported, scheduled, handled, and closed events based on the event status in the anomaly log, and refreshing the status panel within the statistical period. After completing this step, the anomaly closure handling results, status statistics, and complete anomaly log records are output as input for the next step of tracking and visualizing the handling results and recording data.

[0047] Finally, the tracking and handling results are visualized and recorded. The purpose of this step is to present the overall tracking results, abnormal event status, and handling process generated in the preceding steps to maintenance personnel in an intuitive way, while simultaneously creating complete logs and archives for subsequent review, auditing, and system optimization. In specific implementation, the visual interactive interface includes an image display area, a map display area, a status display area, an event list area, and a log query area. In the image display area, the images currently collected by the drone and ground equipment are displayed in real time, with target bounding boxes, target categories, and global target identifiers overlaid on the images, allowing maintenance personnel to directly observe the continuous tracking of the same target from different air and ground perspectives. In the map display area, based on an oilfield area map or electronic fence base map, the current location of each device, inspection path, abnormal event location, and distribution of handling resources are displayed, with different colors or icons used to distinguish normal targets from abnormal targets. In the status display area, the number of current abnormal events at each stage, the number of high-risk events that have not been closed, the average handling time, and resource usage are displayed in real time, enabling management personnel to have a comprehensive understanding of the inspection and handling progress. Within the event list area, event entries are displayed by time, risk level, or anomaly type, and users can click on a single event to view its entire process from detection, reporting, scheduling, handling, to feedback. In the log query area, historical records can be retrieved by time range, device type, target type, anomaly type, event status, and handling personnel. In addition to the interface display, the system automatically records inspection logs and handling logs. Inspection logs store frame number, device number, image path, target detection result, target global identifier, geographical location, and time information; handling logs store anomaly event number, anomaly type, risk level, reporting time, scheduled resources, processing progress, review result, and closure time. These logs can be stored in a local database, network database, or distributed file system to support long-term archiving and subsequent data analysis. For critical events, on-site images, before-and-after comparison images, trajectory playback information, and work order execution records can be packaged and saved to form a complete event archive. Through this step, the system not only provides an intuitive presentation of inspection and anomaly handling results but also provides reliable data support for subsequent algorithm evaluation, business assessment, accountability, and rule optimization.

[0048] Based on the continuous execution of the above steps, this implementation method forms a complete intelligent processing flow for air-ground collaborative oilfield inspection. Its inputs include equipment parameters, inspection area parameters, and dual-source inspection data. Its core intermediate results sequentially include standardized inspection frame data, unified geographic coordinate results, global target tracking results, fused feature correlation results, and abnormal event closure-loop results. Its final outputs are a visualized tracking screen, abnormal handling status, and a full-process log archive. Since the output of each step always serves as the input for the next, the entire solution not only has a clear implementation logic but also achieves true data and business closure in engineering.

[0049] In one alternative implementation, the dual-source data processing can be prioritized during the system development phase, where synthetic data is used to expand the sample coverage and real data is used to calibrate model biases. The combination of the two makes the detection model and the matching model more consistent with the actual oilfield scenario.

[0050] In one optional implementation, the global coordinate unification process can establish equipment parameter templates and geographic projection templates for different oilfield areas to adapt to the actual needs of different station layouts, different equipment installation conditions and different inspection radii.

[0051] In one optional implementation, the cross-device target tracking process can store the global target identifier along with the device type, time segment, and spatial trajectory, thereby improving trajectory recovery and target backtracking capabilities during long-term inspections.

[0052] In one optional implementation, the visual and spatial feature fusion matching process can automatically switch weight strategies according to the real-time environmental conditions, so that the system can maintain high correlation stability under sunny, nighttime, sandstorm, or partial occlusion conditions.

[0053] In one optional implementation, the abnormal closed-loop handling process can be set with different handling priorities and escalation rules according to the abnormal risk level. For high-risk oil spill events, robot review and manual intervention can be triggered first, while for low-risk events, they can be placed in the observation queue first, so as to improve the overall resource utilization efficiency.

[0054] In one optional implementation, the visual interactive system, in addition to supporting real-time monitoring, can also support historical playback, event replay, handling path review, and rule configuration modification, so that the system can be used not only for daily inspection and operation, but also for management analysis and solution optimization.

[0055] Implementation Method 3: This implementation method is described in detail with reference to the accompanying drawings. Specific embodiments are provided to further illustrate the technical solutions offered above. Specifically: This implementation method is specifically designed for oilfield air-ground collaborative inspection scenarios. It centers on unified global coordinates and cross-device target tracking, utilizes closed-loop anomaly handling as its implementation vehicle, and employs an integrated visual interaction platform. It constructs a six-layer closed-loop architecture for data exchange and result feedback. Each layer is specifically designed to address the pain points and business needs of oilfield inspection scenarios. The functions from bottom to top are as follows: Dual-source data processing layer: Addressing the pain points of scarce real inspection data and high labeling difficulty in oilfields, it enables adaptive generation of synthetic data for air-ground collaborative inspections in oilfields and standardized loading of real inspection data, outputting air-ground dual-source inspection data in a unified format, providing standardized data source support for full-domain tracking and closed-loop handling.

[0056] Global Coordinate Unification Layer: Addressing the pain point of inconsistent spatial positioning between air and ground equipment in oilfields, it completes the bidirectional accurate conversion between visual pixel coordinates and GPS geographic coordinates, as well as cross-device real-time coordinate calibration between UAVs and ground equipment, unifying the spatial location information of targets across the entire domain, and providing an accurate spatial benchmark for cross-device target tracking.

[0057] Cross-device target tracking layer: Addressing the pain point of fragmented cross-device tracking in oilfields, it achieves lightweight target detection specific to oilfield scenarios, establishes a unique mapping mechanism from local ID to global ID, completes continuous cross-device full-domain target tracking between UAVs and ground equipment, and outputs target tracking results with globally unique identifiers.

[0058] Cross-device feature matching layer: To address the pain point of low target matching accuracy in the complex environment of oilfields, the visual features of the target and GPS spatial features are extracted and weighted and fused. By calculating the similarity of the fused features, the accurate association of targets across devices is achieved, providing reliable feature support for continuous tracking across the entire domain and enhancing the stability of tracking results.

[0059] Anomaly Closed-Loop Handling Layer: Addressing the pain point of fragmented anomaly handling processes in oilfields, a standardized closed-loop process of five stages—detection, reporting, scheduling, handling, and feedback—is constructed to adapt to oilfield safety production standards. This enables automated handling of oilfield anomalies and real-time statistics and full-process traceability of handling status.

[0060] Integrated Visual Interaction Layer: Designed to meet the operational needs of oilfield inspection and maintenance personnel, it integrates dual-source data management, full-domain tracking and monitoring, anomaly handling and scheduling, and result traceability and query, enabling visualized interaction of the entire operation process, reducing the threshold for manual operation, and improving inspection and management efficiency.

[0061] Core module design (including core improvements and core formulas / key designs) The six core modules of this implementation are: dual-source inspection data processing module, global coordinate unification module, cross-device global target tracking module, vision-GPS fusion cross-device feature matching module, anomaly handling closed-loop management module, and integrated visualization interaction module. The design ideas, core improvements, and core formulas / key designs of each module are as follows: Dual-source inspection data processing module Design Concept: A dual-source supply mechanism is constructed, integrating synthetic data generation and real data loading. Synthetic data simulates the inspection perspective of drones and ground equipment, generating labeled data with target type, pixel bounding boxes, and GPS coordinates. Real data supports automated parsing and preprocessing of standardized label formats, achieving seamless switching between the two data sources and unified format output, providing data support for comprehensive tracking and closed-loop management. Core Improvement: Addressing the industry pain point of scarce real inspection data in oilfields, synthetic data can batch generate air-ground dual-source inspection data for different anomaly types and locations. Real data achieves standardized parsing and coordinate transformation of labeled information, improving data processing efficiency by ≥50%. Key Design: Adaptive generation of synthetic data: Pre-defined oilfield geographical boundaries and equipment operating parameters are used to randomly generate GPS coordinates and attitude parameters for UAVs (flying at altitudes of 50-100 meters) and ground-based fixed equipment / inspection robots, generating labeled inspection data adapted to oilfield inspection scenarios. The number of samples generated per batch ranges from 5000 to 20000 frames, with UAV perspective samples accounting for 40%-60% and ground equipment perspective samples accounting for 40%-60%. Simultaneously, complex oilfield environmental parameters are simulated quantitatively, with light intensity simulation ranging from 500 lux to 120000 lux (covering all scenarios including dawn / dusk, noon, and nighttime supplemental lighting). Dust concentration is divided into three levels (light dust with visibility ≥10km, moderate dust with visibility 2-10km, and heavy dust with visibility ≤2km). Simultaneous simulation of interference scenarios such as rain / fog, target obstruction, and equipment vibration is also performed, with abnormal target samples accounting for no less than 30% in a single scenario. The final output contains a unified format of data including frame ID, device type, GPS coordinates, inspection image, target pixel bounding box, and type label. The training fusion ratio of synthetic data and real data is 7:3 to 9:1, and the fusion ratio in the test scenario is 3:7 to 5:5, which can be adaptively adjusted according to the sample size of real data.

[0062] Real data standardization loading: Automatically parses real inspection data images, GPS metadata and target annotation files, completes the format verification of annotation bounding boxes, image size normalization, distortion correction preprocessing, and outputs inspection data with a format completely consistent with the synthetic data, supporting mainstream image formats such as JPG / PNG and mainstream annotation formats such as VOC / COCO.

[0063] Global coordinate unification module Design concept: Based on camera intrinsic parameter calibration and geographic coordinate projection transformation, the system realizes the forward transformation from pixel coordinates to GPS geographic coordinates and the reverse transformation from GPS geographic coordinates to pixel coordinates. At the same time, it completes cross-device coordinate calibration between UAVs and ground equipment, unifies the spatial position information of targets across the entire domain, and provides a coordinate basis for cross-device target tracking.

[0064] Key improvements: For the first time, a dedicated camera intrinsic parameter library for UAVs and ground equipment was designed for collaborative air-ground inspection in oil fields. Coordinate distortion correction and planar offset calculation were introduced, along with a closed-loop error control method for bidirectional conversion between pixel coordinates and GPS geographic coordinates, keeping the bidirectional conversion error stably controlled within ≤0.5 meters. At the same time, the cross-device coordinate calibration process between UAVs and ground equipment was optimized, keeping the cross-device coordinate calibration delay within ≤100ms, enabling real-time mutual calibration of target coordinates between air and ground equipment. Compared with traditional coordinate conversion methods, the conversion accuracy is improved by ≥40%.

[0065] Core formula: Pixel coordinate distortion correction formula To address camera distortion in drones / ground equipment, distortion correction is performed on pixel coordinates:

[0066]

[0067] in, , Normalized coordinates for pixels; For camera focal length, As the camera's main point, Configure camera distortion coefficients (specific parameters for drones / ground equipment); The square of the modulus of the pixel's normalized coordinates; These are the normalized coordinates after distortion removal.

[0068] Core formula for converting pixel coordinates to GPS geographic coordinates The conversion from pixel coordinates to GPS coordinates is achieved through planar offset calculation and UTM-GPS projection transformation:

[0069]

[0070] Where h is the equipment's working altitude (50-100 meters for drones, 0 meters for ground equipment); The target is offset relative to the ground plane of the equipment; Projection transformation function from WGS84 GPS coordinates to UTM plane coordinates This is the inverse transformation function from UTM plane coordinates to WGS84 GPS coordinates; For the device's GPS coordinates, The target's GPS coordinates.

[0071] Cross-device coordinate calibration rules Based on real-time GPS positioning data and attitude angle data collected by the device, the spatiotemporal reference alignment between the UAV and the ground equipment is completed; the target GPS coordinates detected by the UAV are converted into the pixel coordinates of the ground equipment through the above reverse formula, realizing the target coordinate alignment between the air and ground equipment and completing cross-device coordinate calibration; through the inter-frame coordinate deviation closed-loop correction, the cross-device coordinate calibration delay is controlled to ≤100ms, the reprojection error of single-frame coordinate calibration is ≤2 pixels, and the corresponding geospatial error is ≤0.5 meters.

[0072] Cross-device global target tracking module Design concept: First, perform lightweight target detection on the inspection images of UAVs and ground equipment in the oilfield scene, and output the pixel bounding box, confidence score and type of the target; then, based on the mapping mechanism from local ID to global ID, assign a unique global ID to all detected targets of air and ground equipment; finally, realize cross-device global target continuous tracking of UAVs and ground equipment, and output the tracking results with global ID.

[0073] Core improvements: Replacing the traditional independent device tracking mode, a global target ID mapping table is established to achieve ID unification for the same target in air and ground devices, solving the problem of fragmented tracking across devices and improving tracking continuity by ≥60%; at the same time, the target detection algorithm is optimized for oilfield scenarios to balance detection accuracy (≥80%) and real-time performance.

[0074] Core Design / Formula: Lightweight target detection in oilfield scenarios: Optimizes the category weights of the detection algorithm for different target types during oilfield inspections (oil leaks, equipment damage, personnel intrusion), and outputs detection results that conform to the input format of the tracking algorithm. ,in s represents the bounding box of the target pixels, s represents the detection confidence, and c represents the target type.

[0075] Global ID Mapping Rules: Add a device type identifier to the local target ID of the drone / ground device to form a locally unique identifier:

[0076] like If not in the global ID mapping table, then assign a unique global ID. (N is the current maximum ID in the mapping table + 1), and update the mapping table.

[0077] Cross-device tracking result output: Outputs cross-device tracking results with a global ID, including the global ID, device type, frame ID, pixel bounding box, target type, and detection confidence, enabling continuous tracking of the target across the entire domain.

[0078] A cross-device feature matching module that integrates vision and GPS Design concept: Extract visual depth features and GPS spatial features of targets across devices, achieve deep fusion of the two types of features through a feature fusion layer, and then combine cosine similarity and network calibration to calculate the similarity of targets across devices, so as to achieve accurate association of the same target and enhance the reliability of cross-device tracking results.

[0079] Core Improvements: Replacing the traditional single visual feature matching method, this system is the first to design an adaptive weighted fusion mechanism for visual depth features and GPS spatial features specifically for the complex environment of oilfields. It defines a fusion ratio range of 60%-80% for visual features and 20%-40% for GPS spatial features, and can adaptively adjust the weights based on environmental conditions such as sandstorms and lighting. The system leverages the environmental anti-interference capabilities of GPS spatial features to compensate for the distorted nature of visual features. Combined with cosine similarity and network calibration mechanisms, a feature similarity threshold of 0.75-0.85 is set, resulting in a ≥35% improvement in cross-device target matching accuracy.

[0080] Core formula: Fusion Feature Extraction Formula To address the complex inspection environment in oilfields, an adaptive weighted fusion mechanism for visual features and GPS spatial features is designed. The basic fusion weight is 60%-80% for visual features and 20%-40% for GPS spatial features. The weights can be dynamically adjusted according to the real-time environmental interference level. The fusion feature extraction formula is as follows:

[0081]

[0082] in, For target visual depth features, GPS spatial features of the target (after normalization); For the weights and biases of the fusion layer, For activation functions; The normalized fusion characteristics It is an L2 norm.

[0083] Cross-device target similarity calculation formula

[0084]

[0085] in, These are the normalized fusion characteristics of drones and ground equipment targets, respectively; Cosine similarity; For similarity calibration layer weights and biases, Use the Sigmoid activation function; To determine the final cross-device target similarity, and considering the need to balance false positives and false negatives in oilfield inspection scenarios, the similarity judgment threshold is set to a range of 0.75-0.85. The baseline threshold is 0.8 in normal scenarios, and can be lowered to 0.75 in scenarios with strong interference such as sandstorms and low light. If the final similarity is greater than the set threshold, it is determined to be the same target.

[0086] Anomaly Handling Closed-Loop Management Module Design concept: Based on cross-device full-domain target tracking results, a five-stage standardized closed-loop anomaly handling process is constructed, consisting of detection, reporting, scheduling, handling, and feedback, to automate the handling of high-risk anomalies (oil leaks, equipment damage) in the oilfield; at the same time, an anomaly status statistics mechanism is designed to realize real-time statistics and traceability of the status of the entire handling process, providing data support for oilfield safety production decisions.

[0087] Core improvements: The discrete steps of oilfield anomaly handling are integrated into a five-stage automated closed-loop process adapted to oilfield safety production standards, encompassing detection, reporting, dispatching, handling, and feedback. The triggering conditions, execution standards, and status transition rules for each stage are clearly defined, and intelligent matching rules between anomaly target types and handling resources are designed. A real-time status update mechanism is established, with an update frequency of no less than once every 5 minutes, enabling full-process status monitoring and traceability in anomaly handling. Compared to traditional manual handling processes, anomaly handling efficiency is improved by ≥50%.

[0088] Core Design / Formula: Intelligent scheduling rules for disposal resources: Dedicated disposal resources are matched according to the type of abnormal target. Oil leaks are matched with ground inspection robots, and equipment damage is matched with professional maintenance teams, so as to achieve precise scheduling of disposal resources.

[0089] Statistical formula for abnormal handling status:

[0090] Where M is the total number of abnormal events; For indicator functions S represents the handling status (detected / reported / scheduled / handled / feedback received); Real-time statistics on the number of abnormal events in a corresponding state are provided to track the handling status.

[0091] Five-stage closed-loop process: Anomaly detection phase: Based on cross-device tracking and feature matching results, high-risk targets (oil leaks, equipment damage, unauthorized personnel entry, etc.) that meet the oilfield anomaly judgment criteria are screened in real time, and a standardized anomaly event form is generated, which includes the anomaly global ID, target type, GPS coordinates, detection equipment, confidence level, and on-site image information. The handling status is updated to "detected" and entered into the anomaly event log. Anomaly reporting phase: The generated anomaly event form will be automatically reported to the oilfield inspection control system, added to the waiting queue, and the handling status will be updated to "reported" synchronously. The reporting process does not require manual intervention and the reporting delay is ≤1 second. Disposal and scheduling phase: Based on the type, risk level and geographical location of the abnormal target, and according to the preset intelligent scheduling rules, the optimal disposal resources are matched for the abnormal event (oil leaks are matched with explosion-proof ground inspection robots, equipment damage is matched with local maintenance teams, and personnel intrusion is matched with on-site security personnel), a disposal work order is generated and sent to the corresponding disposal terminal, and the disposal status is updated to "dispatch" simultaneously. Abnormal handling phase: After receiving the work order, the handling resources will perform on-site handling operations. During the handling process, the handling progress will be transmitted back in real time through the terminal. After the handling is completed, the handling results, on-site verification images and handling personnel information will be uploaded, and the central control system will update the handling status to "handled". Feedback and Closure Phase: The central control system reviews the handling results. Once the anomaly is confirmed to have been eliminated, the anomaly event closure is completed, and the handling status is updated to "Feedback Completed / Closed." All process information is archived in the anomaly handling ledger to support subsequent traceability and statistical analysis. Throughout the process, the handling status of all anomalies is updated at least once every 5 minutes. Anomalies that are not closed will be continuously highlighted in the central control system to ensure that the entire anomaly handling process is controllable.

[0092] Core Implementation Steps The air-ground collaborative oilfield inspection intelligent agent's full-domain visual target tracking and closed-loop processing method of this embodiment is specifically implemented in steps S1-S7. Each step is executed sequentially and data is exchanged to form a complete "data-coordinate-tracking-matching-processing-display" closed loop. It can be directly deployed to the central control system of oilfield inspection. The specific steps are as follows: S1: System Initialization and Parameter Configuration Configure the hardware environment and software framework of the oilfield inspection control system, determine the computing device (CPU / GPU) for algorithm execution, and initialize the communication interfaces of each module of the system; The core parameters for oilfield inspection are set as follows: oilfield geographical boundaries, camera intrinsic parameters of UAVs / ground equipment (focal length, principal point, distortion coefficient), target detection confidence threshold, cross-equipment similarity matching threshold, and anomaly handling resource scheduling rules, etc. Initialize the global target ID mapping table, the exception event queue, and the handling status statistician; complete the initialization of the projection transformation function for global coordinate transformation.

[0093] S2: Generation, Loading, and Preprocessing of Dual-Source Inspection Data Select your data source: synthetic data or real data through a visual interactive system; If you choose to synthesize data: set the number of frames to generate, batch generate dual-source inspection data of UAV and ground equipment, including frame ID, GPS coordinates, inspection images, target annotations, and complete data format standardization; If real data is selected: Select the root directory of real inspection data, automatically load images, GPS metadata and target annotation files, complete annotation format verification and image size normalization, and output dual-source inspection data with the same format as the synthetic data; The preprocessed dual-source inspection data are correlated frame by frame to construct a frame-level air-ground collaborative inspection dataset.

[0094] S3: Global Coordinate Unified Parameter Calibration and Initialization Based on the hardware parameters of the drone / ground equipment, load the corresponding device-specific camera intrinsic parameter library and complete the initialization of pixel coordinate distortion correction parameters; Initialize the projection transformation function between WGS84 GPS and UTM planar coordinates, complete the parameter configuration for bidirectional conversion between pixels and GPS coordinates, and preset the bidirectional conversion error threshold to ≤0.5 meters; Based on the device's GPS coordinates and attitude data from the first frame of inspection data, the cross-device coordinate calibration parameters between the UAV and ground equipment are initialized, the coordinates of the target in the first frame are aligned, the calibration error in the first frame is verified, and if the error exceeds the threshold, the internal parameter secondary calibration is triggered. Configure a cross-device coordinate real-time calibration mechanism, set inter-frame calibration trigger rules, and control the cross-device coordinate calibration delay to ≤100ms to ensure the consistency of coordinate reference during dynamic inspection.

[0095] S4: Cross-device global target tracking execution Select the inspection frame to be processed, perform lightweight target detection in the oilfield scene on the inspection images of the UAV and ground equipment respectively, and output the pixel bounding box, confidence and type of the target; Detected targets are identified by local IDs, and a unique global ID is assigned to all targets based on the global ID mapping rules. The global ID mapping table is then updated. Perform cross-device target tracking between UAVs and ground equipment, and output cross-device tracking results with a global ID, including information such as global ID, device type, pixel bounding box, and target type.

[0096] S5: Cross-device target feature matching and association The visual region of the target is extracted from the cross-device tracking results, and the visual depth features of the target are obtained through a feature extraction network. At the same time, the GPS coordinates of the target are obtained through a global coordinate unification module, and the GPS spatial features are obtained after normalization. Visual features and GPS features are spliced ​​and fused to generate normalized fused features; Calculate the similarity of fusion features between UAVs and ground equipment targets, identify targets with similarity greater than a threshold as the same target, complete the accurate association of cross-device targets, and enhance the reliability of tracking results.

[0097] S6: Fully automated closed-loop exception handling Anomaly Detection: Based on the oilfield safety production anomaly judgment standards, high-risk anomaly targets (oil leaks, equipment damage, unauthorized personnel entry, etc.) are screened from the cross-device tracking results after association. A standardized anomaly event form containing the global anomaly ID, target type, GPS coordinates, detection equipment, confidence level, and on-site image is generated, added to the anomaly event queue, and the handling status is updated to "detected" synchronously. The anomaly is also entered into the anomaly log. Anomaly Reporting: Automatically report anomaly events in the anomaly queue to the oilfield inspection control system without manual intervention. The reporting delay is ≤1 second. Simultaneously, add the anomaly to the waiting queue and update the handling status to "reported". Dispatch and handling: Based on the type, risk level and geographical location of the abnormal target, the preset intelligent dispatch rules are executed to match dedicated disposal resources for each abnormal event (oil leaks are matched with explosion-proof ground inspection robots, equipment damage is matched with professional maintenance teams, and personnel intrusion is matched with on-site security personnel), generate disposal work orders and send them to the corresponding disposal terminals, and update the disposal status to "dispatch" in a synchronous manner; Abnormal Handling: After receiving a work order, the handling resource executes on-site handling operations. During the handling process, the handling progress is transmitted back in real time through the terminal. After the handling is completed, the handling results, verification images and handling personnel information are uploaded, and the central control system updates the handling status to "handled" simultaneously. Handling feedback and closed loop: The central control system automatically reviews and manually confirms the handling results. After confirming that the abnormality has been eliminated, the abnormal event is closed, the handling status is updated to "feedback / closed loop", and the entire process information is archived to the handling ledger. Real-time statistics and updates of handling status: Based on the abnormal event ledger, the number of abnormal events in each handling status is counted in real time. The global update frequency of handling status is no less than once every 5 minutes. High-risk abnormal events that have not been closed are continuously highlighted to achieve full monitoring and traceability of the handling process.

[0098] S7: Visualization and Data Recording of Tracking and Handling Results In the image display area of ​​the visual interactive system, the target tracking bounding boxes of the drone and ground equipment are drawn in real time and labeled with global IDs, intuitively displaying the cross-device global target tracking results; The status display area shows the number of statuses at each stage of the anomaly handling process in real time, clearly presenting the progress of the closed-loop handling. Automatically record inspection and handling logs, including frame ID, tracking target information, abnormal event information, handling resources, handling results, handling time, etc. It supports local saving of tracking result images and disposal logs, enabling traceability of inspection and disposal data.

[0099] The core innovations of this implementation method are reflected in seven aspects. All innovations are specifically designed to address the scenario characteristics and technical pain points of full-domain target tracking and anomaly closed-loop handling in oilfield air-ground collaborative inspection, possessing significant technical originality and practical application value. Furthermore, all designs are adapted to the on-site computing power and application requirements of oilfield inspection. The specific innovations are as follows: A method for unifying coordinates across the entire oilfield scenario is proposed: A dedicated camera intrinsic parameter library is designed for both UAVs and ground equipment to achieve precise bidirectional conversion between visual pixel coordinates and GPS geographic coordinates. A corresponding error closed-loop control method is implemented to keep the bidirectional conversion error within ≤0.5 meters. The cross-device coordinate calibration process is optimized to keep the calibration delay within ≤100ms, enabling real-time coordinate calibration of both UAVs and ground equipment. This unifies the spatial location information of targets across the entire domain, fundamentally solving the core problem of inconsistent location benchmarks between air and ground equipment.

[0100] Design a cross-device global target tracking mechanism based on global ID mapping: Establish a unique mapping rule from local ID to global ID, assign a globally unique ID to all targets of air and ground devices, realize continuous global tracking of the same target in different devices, and maintain the global ID retention rate of the same target ≥98%, completely solving the problem of result fragmentation caused by traditional independent tracking of devices.

[0101] A cross-device feature matching method integrating visual and GPS spatial features is proposed: For the first time, an adaptive weighted fusion mechanism of visual depth features and GPS spatial features is designed for the complex environment of oilfields. The fusion range is defined as 60%-80% for visual features and 20%-40% for GPS spatial features. The environmental anti-interference ability of GPS features is used to compensate for the susceptibility of visual features to distortion. Combined with cosine similarity and network calibration mechanism, the cross-device target matching accuracy is improved by ≥35%.

[0102] Construct a five-stage closed-loop process for handling anomalies that is adapted to oilfield safety production standards: design a fully automated closed-loop process for detection, reporting, scheduling, handling, and feedback, clarify the execution standards and status flow rules for each stage, establish intelligent matching rules for anomaly types and handling resources, and set the handling status update frequency to be no less than once every 5 minutes, so as to realize the fully automated handling of oilfield anomalies, improve handling efficiency by ≥50%, and enable real-time monitoring and traceability of the entire process status.

[0103] Design a dual-source air-ground inspection data processing mechanism: Construct a dual-source data system of adaptive generation of synthetic data and standardized loading of real data, clarify the environmental simulation quantification parameters of synthetic data, the sample generation range and the fusion ratio of real data, realize seamless switching and unified format output of the two types of data, make up for the industry shortcomings of scarce real inspection data in oil fields and high labeling difficulty, and provide sufficient data support for algorithm training and functional testing.

[0104] Lightweight optimization of target detection and tracking algorithms for oilfield scenarios: Based on the target types and computing power constraints of edge devices in oilfield inspections, the target detection and tracking algorithms are optimized for lightweight scenarios, balancing detection accuracy (≥80%) and tracking real-time performance, and adapting to the computing power requirements of various inspection equipment such as oilfield drones, ground inspection robots, and fixed cameras.

[0105] Develop an integrated visual interactive system specifically for oilfield inspections: integrate the full-process functions of dual-source data management, full-domain tracking and monitoring, anomaly closed-loop handling, and result traceability query, realize real-time visual display of tracking results and handling status, support local storage of handling logs and full-process traceability, reduce the operation cost of operation and maintenance personnel by ≥60%, and significantly improve the intelligent management level of oilfield inspections.

[0106] In the embodiment: To verify the technical advantages of this implementation method, a simulated test environment and a real oilfield field test environment were built for typical application scenarios of air-ground collaborative inspection in oilfields. Comparative tests were conducted with existing mainstream technical solutions, and the test results are as follows: Comparative test with existing air-ground cooperative path optimization schemes (compare with document 4, improved ant colony and chaotic whale optimization algorithms) This test targeted a full-area inspection scenario at oilfield stations and storage facilities. The path planning efficiency and target positioning accuracy of the proposed implementation scheme were compared with those in Comparative Document 4. The test results are as follows:

[0107] Test results show that, compared with the path optimization scheme in Comparative Document 4, this implementation method, through a global coordinate unification mechanism and cross-device real-time calibration method, significantly improves the target positioning accuracy and coordinate calibration real-time performance. At the same time, it greatly optimizes the path planning efficiency of air-ground collaboration and solves the shortcomings of existing schemes that only focus on static path coverage and ignore the accurate positioning and continuous tracking of dynamic targets.

[0108] Comparison test with existing hierarchical path planning schemes (Comparison document 5, hierarchical strategy + improved wolf pack and particle swarm optimization algorithms) This test targeted a routine inspection scenario in a complex oilfield environment. The core indicators of target tracking continuity and anomaly handling efficiency were compared between the proposed implementation scheme and the scheme in Comparative Document 5. The test results are as follows:

[0109] Test results show that, compared with the hierarchical path planning scheme in Comparative Document 5, this implementation method, through the cross-device tracking mechanism of global ID mapping, the visual-GPS fusion feature matching method, and the five-stage standardized closed-loop handling process, has made a qualitative improvement in the continuity of cross-device target tracking and the efficiency of anomaly handling. It solves the core defects of the existing scheme that only focuses on inspection path coverage and lacks the ability to continuously track targets across the entire domain and to handle automated closed-loop processes.

[0110] In response to the complex environments commonly encountered in oilfields, such as dust storms, low light levels, and target obstruction, an anti-interference test was conducted on the proposed implementation scheme. The test results showed that under moderate dust storm conditions and low light conditions at dawn and dusk, the cross-device target matching accuracy of this implementation scheme remained above 90%. Compared with the traditional single visual feature matching scheme (accuracy <65% under strong interference conditions), the environmental anti-interference capability was significantly improved, fully meeting the application needs of complex field inspection scenarios in oilfields.

[0111] Combination Figure 1 The structure, from bottom to top, consists of a dual-source data processing layer, a global coordinate unification layer, a cross-device target tracking layer, a cross-device feature matching layer, an anomaly closed-loop handling layer, and an integrated visualization interaction layer. Combination Figure 2This diagram illustrates the core implementation steps of full-domain target tracking and closed-loop handling, in the following order: S1 System initialization and parameter configuration → S2 Generation / loading and preprocessing of dual-source inspection data → S3 Unified parameter calibration and initialization of full-domain coordinates → S4 Execution of cross-device full-domain target tracking → S5 Cross-device target feature matching and association → S6 Automated execution of anomaly handling in a fully closed loop → S7 Visualization and data recording of tracking and handling results.

[0112] The above description of several specific embodiments further details the technical solution provided by the present invention in order to highlight the advantages and benefits of the technical solution provided by the present invention. However, the above-described specific embodiments are not intended to limit the present invention. Any reasonable modifications and improvements to the present invention, combinations of embodiments, and equivalent substitutions based on the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for all-domain visual target tracking and closed-loop processing of an intelligent oilfield inspection agent with air-ground collaboration, characterized in that, include: The steps include: acquiring inspection area parameters, air and ground inspection equipment parameters, and inspection data; generating air and ground inspection data in a unified format; and performing standardized processing. Based on the air-ground inspection data and equipment parameters, the steps are as follows: establish a mapping relationship between pixel space and geographic space and perform cross-device spatial alignment to obtain target location information with unified spatial representation. Based on the target location information expressed in a unified spatial manner, the steps are as follows: target identification is performed on inspection data and a target identification system is constructed. Cross-device target identification association is established to obtain continuous target tracking results across the entire domain. Based on the target tracking results, the steps are as follows: extract multi-source features of the target and perform fusion processing, perform cross-device target association determination and calibrate the target identifier to obtain consistent target association results across devices; Based on the target association results, the steps are as follows: identify abnormal targets and generate abnormal events, execute the abnormal handling process according to preset rules and complete the status update to obtain the abnormal closed-loop handling results. Based on the target tracking results and anomaly closed-loop handling results, the execution results are displayed and data is recorded to achieve visualization and traceability of the inspection and handling processes.

2. The method for all-domain visual target tracking and closed-loop processing of an air-ground collaborative oilfield inspection intelligent agent according to claim 1, characterized in that, The air-to-ground inspection data is constructed by fusing synthetic data and real data to form a unified data format.

3. The method for all-domain visual target tracking and closed-loop processing of an intelligent oilfield inspection agent with air-ground collaboration as described in claim 1, characterized in that, Pixel space and geographic space can be converted bidirectionally through camera parameters and spatial mapping.

4. The method for all-domain visual target tracking and closed-loop processing of an intelligent oilfield inspection agent with air-ground collaboration as described in claim 1, characterized in that, Cross-device spatial alignment is based on device positioning and time synchronization and dynamic calibration.

5. The method for all-domain visual target tracking and closed-loop processing of an intelligent oilfield inspection agent with air-ground collaboration as described in claim 1, characterized in that, The target identifier achieves cross-device consistency through mapping between local and global identifiers.

6. The method for all-domain visual target tracking and closed-loop processing of an air-ground collaborative oilfield inspection intelligent agent according to claim 1, characterized in that, Target association is achieved by fusing visual features and geospatial features.

7. A fully-domain visual target tracking and closed-loop processing system for an intelligent oilfield inspection agent with air-ground collaboration, characterized in that, include: This module acquires inspection area parameters, air and ground inspection equipment parameters, and inspection data, generates air and ground inspection data in a unified format, and performs standardized processing. Based on the air-ground inspection data and equipment parameters, a module is established to create a mapping relationship between pixel space and geographic space and to perform cross-device spatial alignment to obtain target location information with a unified spatial representation. Based on the target location information expressed in a unified spatial format, this module identifies targets in inspection data, constructs a target identification system, establishes cross-device target identification associations, and obtains continuous target tracking results across the entire domain. Based on the target tracking results, a module is used to extract multi-source features of the target and perform fusion processing, perform cross-device target association determination and calibrate the target identifier to obtain a consistent target association result across devices; Based on the target association results, the module identifies abnormal targets and generates abnormal events, executes the abnormal handling process according to preset rules and completes the status update to obtain the abnormal closed-loop handling result; Based on the target tracking results and the abnormal closed-loop handling results, the module displays the execution results and records data to realize the visualization and traceability of the inspection and handling process.

8. A computer storage medium for storing computer programs, characterized in that, When the computer program is read by the computer, the computer executes the method of claim 1.

9. A computer, comprising a processor and a storage medium, characterized in that, When the processor reads the computer program stored in the storage medium, the computer executes the method of claim 1.

10. A computer program product, as a computer program, is characterized by: When the computer program is executed, it implements the method of claim 1.

Images