Oil field unmanned aerial vehicle inspection method and system

Abstract

The invention relates to an oil field unmanned aerial vehicle inspection method and system. The method comprises the following steps: planning by using a Dijkstra algorithm to obtain an initial inspection path of an unmanned aerial vehicle among oil field equipment in an inspection area; performing routing inspection based on the initial routing inspection path, and identifying an oil field equipment image captured in real time by using a pre-established anomaly identification model; if the oil field equipment with the abnormal condition exists, planning an inspection sub-path from the current position to the oil field equipment with the abnormal condition by using an A algorithm, and re-planning an inspection sub-path from the oil field equipment with the abnormal condition to a target point by using an A * algorithm to obtain an updated inspection path; and performing routing inspection according to the updated routing inspection path, and repeating the identification process and the routing inspection path re-planning process until all oil field equipment is subjected to routing inspection. Through dynamic path planning, the unmanned aerial vehicle can adjust the inspection path in real time in the inspection process, and the inspection adaptability and flexibility are improved.

Description

Technical Field

[0001] The invention relates to an oilfield unmanned aerial vehicle inspection method and system. Background Art

[0002] With the continuous deepening of oilfield development, inspection tasks are becoming more and more important for the safe operation and fault diagnosis of oilfield equipment. Traditional manual inspection methods are time-consuming and labor-intensive, and require high skills of inspectors. With the development of drone technology, drones have great application potential in oilfield inspections. Summary of the invention

[0003] In order to enrich the types of oilfield drone inspection methods in the prior art and realize real-time remote monitoring of oilfield equipment and environment, the present invention proposes an oilfield drone inspection method and system. The technical solution proposed by the present invention is as follows:

[0004] In a first aspect, the present invention provides an oilfield drone inspection method, comprising:

[0005] The Dijkstra algorithm is used to plan the initial inspection path between the oilfield equipment in the inspection area and the target point of the initial inspection path of the UAV;

[0006] Performing inspection based on the initial inspection path, and during the inspection process, using a pre-established abnormality recognition model to recognize the real-time captured oilfield equipment image to determine whether there is any oilfield equipment in an abnormal condition;

[0007] If yes, use the A algorithm to plan the inspection sub-path from the current position to the oilfield equipment with abnormal conditions, and use the A* algorithm to re-plan the inspection sub-path from the oilfield equipment with abnormal conditions to the target point to obtain an updated inspection path;

[0008] The inspection is carried out according to the updated inspection path, and the above identification process and the process of replanning the inspection path are repeated until all the oilfield equipment are inspected.

[0009] In some embodiments, the use of the Dijkstra algorithm to plan and obtain an initial inspection path between oilfield equipment in an inspection area and a target point of the initial inspection path by the UAV includes:

[0010] The inspection area is divided into multiple grid cells, and the weight of each grid cell is determined according to the distance between oilfield equipment and the predetermined terrain level;

[0011] According to the weight of each grid unit, a first distance from the starting point to each oilfield equipment and a second distance between each oilfield equipment are calculated;

[0012] Determine the initial inspection path using the Dijkstra algorithm according to the first distance from the starting point to each oilfield device and the second distance between each oilfield device.

[0013] In some embodiments, the determining the initial inspection path using the Dijkstra algorithm according to the first distance from the starting point to each oilfield device and the second distance between each oilfield device includes:

[0014] Set the distance of the starting point to 0, create set S and set Q, and add all oilfield devices to set Q;

[0015] Select the oilfield device with the smallest first distance from set Q to obtain oilfield device u, and add oilfield device u to set S;

[0016] If the oilfield device v in set Q satisfies Equation 1, then update dist[v] based on Equation 2 to generate a sub-path from oilfield device u to oilfield device v;

[0017] dist[u] + dist_between(u, v) < dist[v], Equation 1;

[0018] dist[v] = dist[u] + dist_between(u, v), Equation 2;

[0019] Where dist[u] represents the shortest path length from the starting point to oilfield device u, dist_between(u, v) represents the shortest path length from oilfield device u to oilfield device v, and dist[v] represents the shortest path length from the starting point to oilfield device v;

[0020] If set Q is not empty, then use oilfield device v as the new starting point, then obtain the oilfield device with the smallest second distance from set Q to the new starting point, add oilfield device v to set S, and repeat the above update process and the process of obtaining oilfield devices from set Q until set Q is empty to obtain the initial inspection path.

[0021] In some embodiments, the re-planning the inspection sub-path from the oilfield device with an abnormal condition to the target point using the A* algorithm includes:

[0022] Add the oilfield device with an abnormal condition and all oilfield devices to the open list, use the oilfield device with an abnormal condition as the starting point, and use the target point as the target node;

[0023] Determine the evaluation function value f(n) of each oilfield device based on Equation 3, and screen the oilfield device with the smallest evaluation function value from the open list to obtain the current node;

[0024] f(n)=g(n)+h(n), Formula 3;

[0025] Among them, g(n) represents the actual distance from the starting point to the oilfield equipment n, and h(n) represents the heuristic estimated distance from the oilfield equipment n to the target node;

[0026] Determine whether the current node is a target node;

[0027] If not, taking the current node as a new starting point, traverse all oilfield equipment, calculate the new evaluation function value f(n) of each oilfield equipment by formula 3, and update the oilfield equipment in the open list based on the new evaluation function value f(n) to obtain an updated open list;

[0028] The current node is added to the closed list, and the process of screening the oilfield equipment with the smallest evaluation function value from the open list and the process of judging are repeated until the target node is found or the open list is empty, thereby obtaining the inspection sub-path from the current position to the oilfield equipment where the abnormal condition occurs.

[0029] In some embodiments, the anomaly recognition model is trained in the following manner:

[0030] Preprocessing the collected image data including normal devices and abnormal devices to obtain a processed data set;

[0031] Dividing the processed data set into a first training set and a first validation set, and using the first training set to train a pre-established CNN model to obtain a trained model; wherein the CNN model includes a plurality of convolutional layers, a pooling layer, and a fully connected layer;

[0032] The performance of the trained model is evaluated using the first validation set, and the model parameters of the trained model are adjusted or the model structure is changed according to indicators including accuracy, recall rate and F1 score until the model performance meets the preset requirements, thereby obtaining the anomaly recognition model.

[0033] In some embodiments, the method of using the first training set to train the pre-established CNN model to obtain a trained model includes:

[0034] Dividing the first training set into several batches of sub-training sets according to the batch size, inputting each batch of sub-training sets into the CNN model in batches to obtain prediction results, determining a loss function value according to the prediction results and the true labels, and calculating the gradient of the loss function value with respect to the model parameters;

[0035] Based on the gradient of the loss function with respect to the model parameters, an optimizer is used to update the model parameters of the CNN model according to a preset learning rate and a smoothing term to obtain a trained model.

[0036] In some embodiments, based on the gradient of the loss function with respect to the model parameters, using an optimizer to update the model parameters of the CNN model according to a preset learning rate and a smoothing term to obtain a trained model, comprises:

[0037] The two cumulative variables m_t and v_t of the model parameters are updated respectively by equations 4 and 5; among them, m_t is used to estimate the first-order moment of the gradient, and v_t is used to estimate the second-order moment of the gradient. m_t and v_t are updated as follows:

[0038] m_t=β_1*m_t-1+(1-β_1)*g_t, formula 4;

[0039] v_t=β_2*v_t-1+(1-β_2)*(g_t)^2, formula 5;

[0040] Among them, β_1 and β_2 are hyperparameters, m_t-1 and v_t-1 are the accumulated variables of the previous iteration, and g_t is the gradient of the loss function with respect to the model parameters;

[0041] The deviations of the first-order moment and the second-order moment are corrected by equations 6 and 7 respectively, and the corrected first-order moment and second-order moment are obtained:

[0042] m_t_hat=m_t / (1-(β_1)^t), Formula 6;

[0043] v_t_hat=v_t / (1-(β_2)^t), Formula 7;

[0044] Among them, t is the current iteration number, m_t_hat and v_t_hat are the corrected first-order moment and second-order moment respectively;

[0045] Using the modified first-order moment and second-order moment, combined with the learning rate α and the smoothing term ε, the model parameter θ is updated through equation 8 to obtain the trained model:

[0046] θ=θ-α*m_t_hat / (sqrt(v_t_hat)+ε), equation 8.

[0047] In some embodiments, the anomaly recognition model is trained in the following manner:

[0048] Use the ImageNet dataset to pre-train the ResNet-50 model, obtain the pre-trained ResNet-50 model, and deploy it to the drone inspection system;

[0049] The anomaly recognition model is obtained by performing transfer learning on the pre-trained ResNet-50 model using multiple oilfield equipment images.

[0050] In some embodiments, the pre-training of the ResNet-50 model using the ImageNet dataset to obtain the pre-trained ResNet-50 model includes:

[0051] Preprocessing the ImageNet dataset to obtain a preprocessed dataset;

[0052] The ResNet-50 model is trained using the preprocessed data set to obtain a pretrained ResNet-50 model.

[0053] In some embodiments, the use of multiple oilfield equipment images to perform transfer learning on the pre-trained ResNet-50 model to obtain the anomaly recognition model includes:

[0054] Import the pre-trained ResNet-50 model and remove the last fully connected layer of the pre-trained ResNet-50 model;

[0055] According to the number of categories of oilfield equipment, adding a new fully connected layer to the pre-trained ResNet-50 model; wherein the number of output units of the new fully connected layer corresponds to the number of categories;

[0056] The weights of the convolutional layer of the pre-trained ResNet-50 model remain unchanged, and the weights of the newly added fully connected layer are updated to obtain an updated ResNet-50 model;

[0057] The updated ResNet-50 model is trained using multiple oilfield equipment images to obtain the anomaly recognition model.

[0058] In some embodiments, the using of multiple oilfield equipment images to train the updated ResNet-50 model to obtain the anomaly recognition model includes:

[0059] Dividing the plurality of oilfield equipment images into a training set and a validation set according to a preset ratio;

[0060] According to the preset learning rate, batch size and training cycle, the updated ResNet-50 model is trained using the training set, and the model performance is evaluated using the validation set to obtain the anomaly recognition model.

[0061] In some embodiments, the method further comprises:

[0062] According to the sensor data collected in real time, the sensor data is updated on the map to obtain updated map information; wherein the sensor data includes the location and size of the obstacle;

[0063] If a new obstacle is encountered, the updated map information is used to replan the obstacle avoidance path using a pre-built obstacle avoidance model to obtain an optimal obstacle avoidance path around the obstacle; wherein the obstacle avoidance model is built based on the A* algorithm;

[0064] The updated inspection path is integrated with the optimal obstacle avoidance path to generate a new inspection path, and inspection is performed according to the new inspection path.

[0065] In some embodiments, if a new obstacle is encountered, the updated map information is used to replan the obstacle avoidance path using a pre-built obstacle avoidance model to obtain an optimal obstacle avoidance path around the obstacle, including:

[0066] If a new obstacle is encountered, the current node where the new obstacle is located in the updated map information is obtained, multiple neighboring nodes of the current node are accessed, and the total cost f'(next_node) of each neighboring node is determined by Formula 9:

[0067] f'(next_node)=g'(next_node)+h'(next_node), equation 9;

[0068] Among them, g'(next_node) represents the cumulative cost from the starting point to the neighboring node n, and h'(n) represents the estimated cost from the neighboring node n to the target position;

[0069] The neighboring nodes with the smallest total cost are screened to obtain the optimal neighboring node, and the optimal neighboring node is used as a new starting point to visit the new node, and the above-mentioned process of determining the total cost and the screening process are repeated until the obstacle is bypassed and the optimal obstacle avoidance path is obtained.

[0070] In some embodiments, the method further comprises:

[0071] During the inspection process, weather conditions are acquired in real time, the updated inspection path is adjusted in real time according to the weather conditions to obtain an adjusted inspection path, and the inspection is performed according to the adjusted inspection path.

[0072] In some embodiments, the method further comprises:

[0073] A pre-established multi-task model is used to perform real-time anomaly detection and structured information extraction on the identified abnormal conditions to obtain structured anomaly data.

[0074] In some embodiments, the method further includes: after the inspection task is completed, generating an inspection report based on the structured abnormality data; wherein the inspection report includes equipment abnormality classification statistics, abnormal equipment detailed information, an inspection route map and drone flight data.

[0075] In a second aspect, the present invention provides an oilfield UAV inspection system, comprising:

[0076] A static path planning module is used to plan the initial inspection path between the oilfield equipment in the inspection area and the target point of the initial inspection path by using the Dijkstra algorithm;

[0077] An identification module is used to perform inspection based on the initial inspection path, and during the inspection process, use a pre-established abnormality identification model to identify the real-time captured oilfield equipment image to determine whether there is any oilfield equipment in an abnormal condition;

[0078] The dynamic path planning module is used to use the A algorithm to plan the inspection sub-path from the current position to the oilfield equipment where the abnormal condition occurs, and use the A* algorithm to re-plan the inspection sub-path from the oilfield equipment where the abnormal condition occurs to the target point to obtain an updated inspection path; and, perform inspections according to the updated inspection path, repeat the above-mentioned identification process and the process of re-planning the inspection path until all oilfield equipment are inspected.

[0079] In some embodiments, the oilfield drone inspection system further includes:

[0080] The intelligent obstacle avoidance module is used to update the sensor data on the map according to the sensor data collected in real time to obtain updated map information; wherein the sensor data includes the location and size of the obstacle; if a new obstacle is encountered, the obstacle avoidance path is replanned using the updated map information and a pre-built obstacle avoidance model to obtain an optimal obstacle avoidance path that bypasses the obstacle; wherein the obstacle avoidance model is built based on the A* algorithm; the updated inspection path is integrated with the optimal obstacle avoidance path to generate a new inspection path, and inspection is performed according to the new inspection path.

[0081] In some embodiments, the oilfield drone inspection system further includes:

[0082] The adjustment module is used to obtain weather conditions in real time during the inspection process, adjust the updated inspection path in real time according to the weather conditions to obtain an adjusted inspection path, and perform inspections according to the adjusted inspection path.

[0083] In some embodiments, the oilfield drone inspection system further includes:

[0084] The structuring module is used to perform structured processing on the identified abnormal conditions using a pre-established multi-task model to obtain structured abnormal data.

[0085] In some embodiments, the oilfield drone inspection system further includes:

[0086] The inspection report generation module is used to generate an inspection report based on the structured abnormal data after the inspection task is completed; wherein the inspection report includes equipment abnormality classification statistics, abnormal equipment detailed information, inspection route map and drone flight data.

[0087] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the oilfield UAV inspection method as described in the first aspect.

[0088] In a fourth aspect, the present invention provides an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0089] Memory, used to store computer programs;

[0090] The processor is used to implement the oilfield UAV inspection method as described in the first aspect when executing the program stored in the memory.

[0091] Based on the above technical solution, the beneficial effects of the present invention compared with the prior art are as follows:

[0092] The oilfield UAV inspection method provided by the present invention utilizes the Dijkstra algorithm to plan the inspection path of the UAV between the oilfield equipment in the inspection area to obtain the initial inspection path; the inspection is performed based on the initial inspection path, and during the inspection process, the oilfield equipment is identified in real time using the ResNet-50-based anomaly recognition model to determine whether there is any oilfield equipment in an abnormal condition; the oilfield equipment image captured in real time can be identified through the anomaly recognition model, and the abnormal conditions in the oilfield equipment and the environment can be identified in real time, and the recognition accuracy of the abnormal conditions of the equipment can be improved through the intelligent image recognition technology, thereby reducing the risk of missed detection and misjudgment. When an abnormal situation occurs in the oilfield equipment, dynamic path planning is introduced, and the A algorithm is used to plan the inspection path from the current position to the oilfield equipment with the abnormal situation. After the abnormal situation is handled, the oilfield equipment with the abnormal situation is used as a new starting point, and the Dijkstra algorithm is used to re-plan the inspection path from the new starting point to the target point to obtain an updated inspection path, thereby realizing automatic inspection route planning and improving inspection efficiency. Through dynamic path planning, the unmanned aerial vehicle can adjust the inspection path in real time during the inspection process, thereby improving the adaptability and flexibility of the inspection. Moreover, the inspection is carried out according to the updated inspection path, and the above-mentioned identification process and the process of re-planning the inspection path are repeated until all the oilfield equipment are inspected. The present invention can quickly adjust the inspection path in real time when an emergency occurs during the unmanned aerial vehicle inspection process, thereby reducing the safety risks of personnel during the on-site inspection process and ensuring the smooth progress of the inspection task.

[0093] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0094] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0095] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0096] Figure 1 It is a flow chart of an oilfield UAV inspection method provided by an embodiment of the present invention;

[0097] Figure 2It is a schematic diagram of a training process of an abnormality recognition model provided by an embodiment of the present invention;

[0098] Figure 3 It is a schematic diagram of a training process of another abnormality recognition model provided by an embodiment of the present invention;

[0099] Figure 4 It is an overall framework diagram of the multi-task model provided by an embodiment of the present invention;

[0100] Figure 5 It is a structural schematic diagram of an oilfield UAV inspection system provided by an embodiment of the present invention;

[0101] Figure 6 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0102] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0103] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0104] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0105] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0106] As oilfield development continues to deepen, inspection tasks are becoming increasingly important for the safe operation and fault diagnosis of oilfield equipment. Traditional manual inspection methods are time-consuming and labor-intensive, and require high skills from inspectors. With the development of drone technology, drones have great application potential in oilfield inspections. The drone inspection system in the prior art usually performs inspections through the following steps:

[0107] (1) Planning the inspection route: Before the inspection, the system needs to pre-plan the inspection route of the drone, including the take-off point, inspection point, flight altitude, etc., to ensure that the drone can complete the inspection task efficiently. (2) Inspection process: The drone flies according to the preset inspection route, using the onboard camera, infrared sensor and other equipment to collect images and data information of the inspection area in real time. (3) Data processing: After the drone completes the inspection, the collected images and data information are transmitted to the ground control station. Analysts analyze this information to identify abnormal conditions, such as equipment damage and leakage.

[0108] However, there are some problems with the current technology and methods, as shown in:

[0109] (1) Insufficient intelligent obstacle avoidance capability: The existing drone inspection system has limited intelligent obstacle avoidance capability when encountering obstacles, which may cause the drone to collide or be unable to bypass obstacles, increasing the inspection risk. (2) Low image recognition accuracy: The image recognition algorithm in the existing technology may not be able to accurately identify abnormal conditions, resulting in missed reports or false reports, reducing inspection efficiency. (3) Abnormal data processing is not structured enough: The abnormal data processing in the existing technology is often scattered, which is not convenient for oilfield managers to obtain and analyze equipment failure information.

[0110] In summary, the current drone inspection systems on the market still have many deficiencies in route planning, anomaly identification, obstacle avoidance, etc., and need further optimization and improvement. Traditional oilfield inspection methods rely on manual inspection of facilities and environments, which are inefficient and have limited coverage, and cannot meet the inventor's expectations. In order to solve this problem, the inventors made the present invention after further research and development. The present invention provides an oilfield drone inspection method and system, which realizes real-time remote monitoring of oilfield facilities and environments through advanced drone technology and image recognition algorithms.

[0111] The objects of the present invention are:

[0112] 1. Improve inspection efficiency: Traditional oilfield equipment inspection methods often require manual on-site inspections, which is time-consuming and inefficient. One of the purposes of the present invention is to achieve automated and efficient inspections by introducing drones and intelligent image recognition technology, thereby improving overall inspection efficiency.

[0113] 2. Reduce inspection costs: Manual inspections require a lot of human resources and require high skills from inspectors. The present invention aims to reduce the number of inspectors and their skill requirements through automation technology, thereby reducing overall inspection costs.

[0114] 3. Improve inspection accuracy: Manual inspection is limited by personnel experience and skill level, which may lead to missed inspections or misjudgments. The present invention improves the accuracy of identifying abnormal conditions of equipment and reduces the risk of missed inspections and misjudgments through intelligent image recognition technology.

[0115] 4. Improve inspection safety: There may be safety hazards during the inspection of oilfield equipment, such as equipment leakage, fire, etc. The present invention uses drone technology to achieve remote inspection and reduce the safety risks of personnel during on-site inspections. Realize real-time monitoring and adjustment: The results of the present invention combine dynamic path planning and intelligent obstacle avoidance functions to enable drones to adjust their routes in real time during the inspection process, improving the adaptability and flexibility of inspections.

[0116] Embodiment 1

[0117] The embodiment of the present invention provides an oilfield unmanned aerial vehicle inspection method, referring to Figure 1 As shown, including:

[0118] S101, using Dijkstra algorithm to plan and obtain the initial inspection path between the oilfield equipment in the inspection area and the target point of the initial inspection path of the UAV;

[0119] In terms of path planning, the Dijkstra algorithm is used to plan the shortest path through all oilfield equipment, and the calculation of the optimal inspection route is realized. The above-mentioned drone uses a DJI Matrice 300RTK drone, equipped with a high-definition camera (4K resolution), an infrared camera (640x512 resolution), a global positioning system (GPS), a high-performance battery (providing 55 minutes of battery life) and a high-precision RTK module to achieve accurate positioning and stable flight performance. The present invention uses drones to inspect oilfield equipment, which has higher efficiency and accuracy than traditional manual inspection methods. At the same time, drone inspections can cover a wider area, especially in harsh environments and terrain conditions, significantly reducing the safety risks of personnel. Moreover, the present invention uses electric drones for inspections, which have lower energy consumption and emissions than traditional fuel-powered drones, which helps to reduce carbon emissions during oilfield inspections and promote green and environmentally friendly development.

[0120] S102, performing inspection based on the initial inspection path, and during the inspection process, using a pre-established abnormality recognition model to recognize the real-time captured oilfield equipment image to determine whether there is any oilfield equipment in an abnormal condition; if so, executing step S103; if not, performing inspection according to the initial inspection path;

[0121] The abnormal recognition model can identify the real-time captured oilfield equipment images. By combining advanced image recognition technology and real-time data analysis, the equipment status of oilfield equipment can be accurately identified and abnormalities can be detected. Compared with traditional manual inspections, the intelligent drone inspection system can discover more potential faults and improve the inspection quality.

[0122] S103, using the A algorithm to plan an inspection sub-path from the current position to the oilfield equipment where the abnormal condition occurs, and using the A* algorithm to re-plan the inspection sub-path from the oilfield equipment where the abnormal condition occurs to the target point to obtain an updated inspection path;

[0123] On the basis of the original path planning based on Dijkstra algorithm, a dynamic path planning function is introduced, and the A algorithm is used for real-time adjustment. When an emergency (such as sudden weather change, equipment failure, etc.) occurs during the drone inspection process, the system can quickly adjust the inspection route in real time to ensure the smooth progress of the inspection task. First, the A algorithm is used to plan the inspection sub-path from the current position to the oilfield equipment with abnormal conditions, and then the A* algorithm is used to re-plan the inspection sub-path from the oilfield equipment with abnormal conditions to the target point, and the updated inspection path is obtained based on the two inspection sub-paths. The present invention adopts a path planning method based on the Dijkstra algorithm, and adjusts the inspection path in real time by introducing the A* algorithm. The method can perform path planning based on real-time information such as emergencies and weather conditions to ensure the smooth progress of drone inspections.

[0124] S104, performing inspection according to the updated inspection path, repeating the above identification process and the process of replanning the inspection path until all oilfield equipment is inspected.

[0125] The oilfield drone inspection method provided by the present invention can significantly improve the inspection efficiency and reduce the inspection time through the combination of drone technology and intelligent image recognition technology. Reduce costs: Reduce manpower input through automation technology, and reduce the overall cost of the inspection process. Utilizing advanced image recognition technology, the present invention can accurately identify abnormal conditions of equipment and reduce the risk of missed detection and misjudgment. Remote inspection by drones reduces the safety risks of personnel during on-site inspections and improves overall inspection safety. Real-time monitoring and adjustment: The present invention has real-time monitoring and dynamic adjustment capabilities, making the inspection process more flexible and adaptable.

[0126] The present invention is suitable for inspection of oilfield equipment, including but not limited to inspection of oil wells, oil pipelines, pumping stations, oil storage tanks and other equipment. At the same time, the technical solution can also be extended to other scenarios, such as inspection of power facilities, inspection of water conservancy facilities, etc. With the rapid development of industrial automation technology, the demand for intelligent inspection in various industries is growing. The results of the present invention have obvious advantages in improving the efficiency of oilfield equipment inspection, reducing costs, and improving recognition accuracy and safety, and have broad market application prospects. With the continuous advancement of drone technology, image recognition technology and big data analysis technology, the performance of the results of the present invention will be further improved, providing more efficient and intelligent solutions for equipment management in various industries.

[0127] In summary, the results of the present invention realize efficient, low-cost, accurate and safe inspection of oilfield equipment, and provide strong support for modern oilfield equipment management.

[0128] In a specific embodiment, the above step S101 uses the Dijkstra algorithm to plan and obtain the initial inspection path between the oilfield equipment in the inspection area and the target point of the initial inspection path of the UAV, including:

[0129] S1011, dividing the inspection area into a plurality of grid units, and determining the weight of each grid unit according to the distance between oilfield equipment and a predetermined terrain level;

[0130] The mapping relationship between terrain data and terrain level, and terrain level and weight is established in advance. According to the terrain data of each grid unit in the inspection area, the terrain level corresponding to each grid unit is determined through the mapping relationship between the terrain data and the terrain level. According to the determined terrain level, the weight corresponding to each grid unit is determined through the mapping relationship between the terrain level and the weight. Different terrain levels are assigned different weights, with complex terrains being assigned higher weights and simple terrains being assigned lower weights.

[0131] S1012, calculating a first distance from the starting point to each oilfield equipment and a second distance between each oilfield equipment according to the weight of each grid unit;

[0132] For the sake of distinction, in this embodiment, the distance from the starting point to the oilfield equipment is called the first distance, and the distance between the oilfield equipment is called the second distance. The starting point is set to the monitoring center, the monitoring center (denoted as point A), and the oilfield equipment is represented by B1, B2...Bi. The set goal is to find a shortest path passing through all oilfield facilities. Calculate the first distance (denoted as di) from the monitoring center A to all oilfield equipment Bi, and calculate the second distance between each oilfield equipment. The first distance and the second distance are represented by Euclidean distance, the first distance is determined according to the weight of each grid unit from the monitoring center A to all oilfield equipment Bi, and the second distance is determined according to the weight of each grid unit between the oilfield equipment. Taking the first distance and the second distance as an example, the distance from A to Bi, that is, the first distance di, as an example: di = sqrt((xA-xBi)^2+(yA-yBi)^2), wherein xA-xBi is obtained by summing the horizontal weights of each grid unit from the monitoring center A to all oilfield equipment Bi, and yA-yBi is obtained by summing the vertical weights of each grid unit from the monitoring center A to all oilfield equipment Bi.

[0133] S1013: Determine the initial inspection path using Dijkstra algorithm according to the first distance from the starting point to each oilfield equipment and the second distance between the oilfield equipment.

[0134] In a specific embodiment, determining the initial inspection path using the Dijkstra algorithm according to the first distance from the starting point to each oilfield device and the second distance between each oilfield device in step S1013 includes:

[0135] S10131. Set the distance of the starting point to 0, create set S and set Q, and add all oilfield devices to set Q;

[0136] Set the distance of starting point A to 0, the distances of other points to infinity, create a set S (visited nodes) and a set Q (unvisited nodes), and add all oilfield devices Bi to set Q. Set S is initially an empty set.

[0137] S10132. Select the oilfield device with the smallest first distance from set Q to obtain oilfield device u, and add oilfield device u to set S;

[0138] S10133. If the oilfield device v in set Q satisfies Equation 1, then update dist[v] based on Equation 2 to generate a sub-path from oilfield device u to oilfield device v;

[0139] dist[u] + dist_between(u, v) < dist[v], Equation 1;

[0140] dist[v] = dist[u] + dist_between(u, v), Equation 2;

[0141] Where dist[u] represents the shortest path length from the starting point to oilfield device u, dist_between(u, v) represents the shortest path length from oilfield device u to oilfield device v, and dist[v] represents the shortest path length from the starting point to oilfield device v; if the oilfield device v in set Q does not satisfy Equation 1, it means that the path from the starting point to oilfield device u and then to oilfield device v is not shorter than the known path. Then do not update dist[v] and keep its original shortest path length, which can ensure that the shortest path already found remains unchanged and avoid invalid updates.

[0142] For the oilfield device v in set Q, determine whether oilfield device v satisfies Equation 1. If it does, it means that the shortest path from the starting point to oilfield device v is A - oilfield device u - oilfield device v. Then the inspection path is selected as A - oilfield device u - oilfield device v, and oilfield device v is added to set S.

[0143] S10134. If the set Q is not empty, use the oilfield equipment v as the new starting point, then obtain the oilfield equipment with the second smallest distance from the new starting point from the set Q, and repeat the above updated process and the process of obtaining oilfield equipment from the set Q until the set Q is empty, to obtain the initial inspection path.

[0144] If the set Q is not empty, use the oilfield equipment v as the new starting point, obtain the oilfield equipment with the smallest distance from the oilfield equipment v, and add the oilfield equipment v to the set S; use the oilfield equipment w with the smallest distance from the oilfield equipment v as the new starting point, and determine whether the oilfield equipment w satisfies dist[v]+dist_between(v,w)<dist[w]. If it is satisfied, update dist[w] to dist[v]+dist_between(v,w), and generate the shortest path from the starting point to the oilfield equipment w: A - oilfield equipment u - oilfield equipment v - oilfield equipment w, and so on, until the set Q is empty, to obtain the inspection path of the drone among each oilfield equipment, that is, the above initial inspection path. In the embodiment of the present invention, every time the oilfield equipment with the closest distance to the current oilfield equipment is obtained, the current oilfield equipment is added to the set S, and the current oilfield equipment is used as the visited node.

[0145] Illustrated by a specific embodiment:

[0146] Suppose there is an oilfield area including 5 oilfield equipment (equipment 1 to equipment 5), and the drone needs to start from equipment 1, inspect all equipment, and return to equipment 1. The coordinates on the map can be represented by points in a two-dimensional plane. The following are the coordinates of each equipment:

[0147] Equipment 1: (0,0)

[0148] Equipment 2: (2,4)

[0149] Equipment 3: (4,2)

[0150] Equipment 4: (3,6)

[0151] Equipment 5: (6,1)

[0152] First, calculate the distances between all oilfield equipment and store them in the adjacency matrix. The distance calculation formula (Euclidean distance) is: di = sqrt((xA - xBi)^2+(yA - yBi)^2).

[0153] The obtained adjacency matrix (distance reserved to one decimal place) is as follows: 1 2 3 4 5

[0155] 1[0.0,4.5,4.5,6.7,6.1]

[0156] 2[4.5,0.0,2.8,2.2,4.2]

[0157] 3[4.5,2.8,0.0,4.1,2.2]

[0158] 4[6.7,2.2,4.1,0.0,5.1]

[0159] 5[6.1,4.2,2.2,5.1,0.0]

[0160] Then use the Dijkstra algorithm to plan the optimal inspection route for the UAV starting from device 1. The Dijkstra algorithm first starts from the starting point, i.e., device 1, and selects the adjacent device with the shortest distance (device 2). Then, from device 2, continue to select the adjacent device with the shortest distance (device 4) until all oilfield devices are visited. Finally, return to the starting device 1.

[0161] Device 1 -> Device 2: 4.5

[0162] Device 2 -> Device 4: 2.2

[0163] Device 4 -> Device 3: 4.1

[0164] Device 3 -> Device 5: 2.2

[0165] Device 5 -> Device 1: 6.1

[0166] The optimal inspection route is: Device 1 -> Device 2 -> Device 4 -> Device 3 -> Device 5 -> Device 1.

[0167] The above optimal inspection route is the initial inspection path, and the total distance is 19.1.

[0168] In a specific embodiment, using the A algorithm to plan the inspection sub-path from the current position to the oilfield device with abnormal conditions in step S103 includes:

[0169] Add the current position and all oilfield devices to the open list, use the current position as the starting point, and use the oilfield device with abnormal conditions as the target node;

[0170] For each oilfield device, respectively determine the actual distance g1(n) from the oilfield device to the starting point and the actual distance g2(n) to the target node, and screen the oilfield devices that meet the following formula 1.0 from the open list to obtain the current node:

[0171] g1(n) + g2(n) < g(n), formula 1.0; where g(n) represents the actual distance from the starting point to the target node;

[0172] Determine whether the current node is a target node;

[0173] If yes, the search is successful. If no, the current node is added to the closed list, and the current node is used as a new starting point to repeat the above process of determining g1(n) and g2(n), the screening process, and the judgment process until no oilfield equipment that satisfies formula 1.0 is found.

[0174] Based on the current nodes obtained by screening, an inspection sub-path from the current position to the oilfield equipment where the abnormal condition occurs is generated.

[0175] In a specific embodiment, the above step S103 of replanning the inspection sub-path from the oilfield equipment with abnormal condition to the target point by using the A* algorithm includes:

[0176] S1031, adding the oilfield equipment with abnormal conditions and all oilfield equipment to an open list (openlist), taking the oilfield equipment with abnormal conditions as the starting point and the target point as the target node;

[0177] S1032, determining the evaluation function value f(n) of each oilfield equipment based on Formula 3, selecting the oilfield equipment with the smallest evaluation function value from the open list, and obtaining the current node;

[0178] f(n)=g(n)+h(n), Formula 3;

[0179] Among them, g(n) represents the actual distance from the oilfield equipment with abnormal conditions to oilfield equipment n, and h(n) represents the heuristic estimated distance from oilfield equipment n to the target node;

[0180] S1033, determine whether the current node is the target node; if so, the search is successful; if not, execute step S1034;

[0181] S1034, taking the current node as a new starting point, traversing all oilfield equipment, calculating a new evaluation function value f(n) of each oilfield equipment by using Formula 3, and updating the oilfield equipment in the open list based on the new evaluation function value f(n), to obtain an updated open list;

[0182] S1035. Add the current node to a closed list, and repeat the above process of screening the oilfield equipment with the smallest evaluation function value from the open list and the above judgment process until the target node is found or the open list is empty, and obtain the inspection sub-path from the current position to the oilfield equipment where the abnormal condition occurs.

[0183] By combining the actual cost, i.e., the above g(n), and the estimated cost, i.e., the above h(n), the A* algorithm can more specifically search for paths close to the target, improve search efficiency, and find a better path. In this embodiment, the A algorithm is first used to plan the path from the current position to the oilfield equipment with abnormal conditions, so that the drone can respond to the abnormality very quickly and reach the location of the oilfield equipment with abnormal conditions in the shortest path. Then, the A* algorithm is used to plan the path from the oilfield equipment with abnormal conditions to the target point. The A* algorithm can comprehensively consider the actual cost and the estimated cost, and choose to expand the node along the path closer to the target, so as to optimize the path selection when planning to the target point. By using the A* algorithm to plan the path from the oilfield equipment with abnormal conditions to the target point, it is possible to more efficiently find the node to be expanded along the path with the lowest estimated cost, reducing unnecessary search and path delays. In this way, the actual cost and heuristic information can be better integrated when planning the path, so that the drone can reach the target point more quickly and accurately, and improve the effect and performance of path planning.

[0184] In a specific embodiment:

[0185] Sample code (using Python as an example):

[0186]

[0187] Suppose there is an oil field area, including 9 oil field equipment, their locations are: A (0,0), B (2,2), C (3,1), D (1,3), E (5,3), F (3,6), G (7,7), H (6,2) and I (8,5). The roads on the map are as follows:

[0188]

[0189]

[0190] It is necessary to go from starting point A to target point I. First, the Dijkstra algorithm is used for path planning. During the planning process, an abnormal message about device E is suddenly received. The drone needs to be guided to device E immediately, so real-time adjustments need to be made based on the current path planning. At this time, the A algorithm is used to adjust the path. The current position of the drone is used as the new starting point, and the position of device E is used as the new target point. The optimal path is calculated by the A algorithm. The calculation result is: A→B→C→E, and the total distance is 7.07.

[0191] After handling the exception of device E, it is necessary to continue the original plan, that is, from device E to device I. At this time, the A* algorithm is used again for path planning, with device E as the new starting point and device I as the new target point. The calculation result is: E→H→I, and the total distance is 4.57.

[0192] Finally, according to the dynamic path planning of the A algorithm and the A* algorithm, the drone first goes to device E to handle the exception, and then goes to the target device I, achieving the goal of real-time adjustment. By introducing the A algorithm and the A* algorithm, a more flexible and efficient path planning can be achieved based on the Dijkstra algorithm.

[0193] In a specific embodiment, deep learning technology can be used to train a convolutional neural network (CNN) image recognition algorithm model for real-time recognition of oilfield equipment images, including facility and environment images, taken by drones. The model is trained with a large number of oilfield equipment images to achieve a high recognition accuracy. Figure 2 As shown, the abnormality recognition model in the above step S102 is trained in the following way:

[0194] S1021, preprocessing the collected image data including normal devices and abnormal devices to obtain a processed data set;

[0195] Image data of normal devices and abnormal devices are collected, and the images are preprocessed, including cropping, scaling, and normalization. The specific method of preprocessing can refer to the description in the prior art.

[0196] S1022, dividing the processed data set into a first training set and a first validation set, and using the first training set to train a pre-established CNN model to obtain a trained model; wherein the CNN model includes a plurality of convolutional layers, a pooling layer, and a fully connected layer;

[0197] In each training cycle, the CNN model extracts image features through the convolution layer, uses the pooling layer to reduce the feature dimension, and finally classifies through the fully connected layer. The optimizer (Adam) is used to update the model parameters to reduce the loss function value.

[0198] S1023. Use the first validation set to evaluate the performance of the trained model, adjust the model parameters of the trained model or change the model structure according to indicators including accuracy, recall rate and F1 score, until the model performance meets preset requirements, and obtain the anomaly recognition model.

[0199] In this embodiment, the accuracy, recall, precision and F1 score are used to adjust and verify the model, specifically:

[0200] (1) Accuracy: Accuracy refers to the proportion of samples correctly classified by the model to the total number of samples. Improving the accuracy means improving the overall model performance. If the accuracy is low, use the following method to adjust it:

[0201] Increase model capacity: Increase the number of convolutional layers or the number of neurons.

[0202] Adjust the learning rate: If the accuracy improves slowly or fluctuates greatly, reduce the learning rate.

[0203] (2) Recall rate: The recall rate indicates the proportion of positive examples that are correctly identified. Improving the recall rate means reducing missed detections. If the recall rate is low, use the following methods:

[0204] Use class-weighted loss function: For imbalanced cases, give higher weights to the less common classes.

[0205] Use data augmentation: Randomly transform the training data to increase the generalization ability of the model.

[0206] (3) Precision: Precision refers to the proportion of correctly identified positive examples to all positive examples. Improving precision means reducing false positives. If the precision is low, use the following methods:

[0207] Use class-weighted loss function: For unbalanced samples, give higher weights to more classes.

[0208] Use data augmentation: Randomly transform the training data to increase the generalization ability of the model.

[0209] (4) F1 score: The F1 score is the harmonic mean of precision and recall, which comprehensively evaluates the performance of the model. If the F1 score is low, use the following method:

[0210] Adjust decision thresholds: According to the output probability of the model, adjust the decision thresholds of positive and negative categories to achieve a better balance between precision and recall.

[0211] Optimize model structure: adjust the number and parameters of convolutional layers, pooling layers, and fully connected layers to achieve better performance.

[0212] Adjust the loss function: Try different loss functions to find one that is appropriate for the problem at hand.

[0213] When adjusting the model, you need to consider multiple indicators such as accuracy, recall, precision, and F1 score. The adjustment process usually requires multiple attempts and verifications to find the model that achieves the best performance. Depending on the actual application scenario and needs, different weights can be given to different indicators as needed. For example, for some applications, you may pay more attention to recall, while for other applications, precision may be more important. In this case, you can adjust the model according to specific needs to achieve the desired performance.

[0214] After the model is trained, the trained CNN model, namely the anomaly recognition model, is applied to the drone inspection process. The drone takes real-time images of oilfield equipment, and the model pre-processes the oilfield equipment images and then inputs them into the anomaly recognition model to identify abnormal conditions.

[0215] In a specific embodiment, the step S1022 described above uses the first training set to train the pre-established CNN model to obtain the trained model, including:

[0216] S10221. Divide the first training set into several batches of sub-training sets according to the batch size, input each batch of sub-training sets into the CNN model in batches to obtain prediction results, determine the loss function value according to the prediction results and the true label, and calculate the gradient of the loss function value with respect to the model parameters;

[0217] The training set is divided into several batches according to the batch size, and then the data of each batch is forward propagated and back propagated to calculate the loss function value and its gradient. The optimizer (Adam) is used to update the model parameters according to the gradient. This process is repeated in each training cycle until the preset number of training cycles is reached. The specific process of forward propagation is: the sub-training set is input into the model, and after calculations such as convolution layer, activation function, pooling layer, etc., the prediction result of the output layer is finally obtained. In this process, the model automatically extracts the features of the input image and generates a prediction value. The difference between the result predicted by the model and the true label is called loss. The loss function is used to measure the gap between the model prediction value and the true label. The smaller the loss function, the more accurate the model prediction. The specific process of back propagation is: the model parameters are updated by calculating the gradient of the loss function value to the model parameters. The gradient is the derivative of the loss function value under the current parameters, which indicates the direction of the change of the loss function. In each iteration, the gradient (g_t) of the loss function of the current batch data with respect to the model parameters is first calculated. By calculating the gradient, you can know how to adjust the parameters to reduce the loss function value. After calculating the gradient, use the optimizer (Adam) to update the model parameters according to the gradient value. The optimizer will adjust the parameters according to the preset learning rate and gradient to reduce the loss function value.

[0218] S10222. Based on the gradient of the loss function with respect to the model parameters, use an optimizer to update the model parameters of the CNN model according to a preset learning rate and smoothing term to obtain a trained model.

[0219] In a specific embodiment, the above step S10222 uses an optimizer to update the model parameters of the CNN model based on the gradient of the loss function with respect to the model parameters according to a preset learning rate and a smoothing term to obtain a trained model, including:

[0220] The two cumulative variables m_t and v_t of the model parameters are updated respectively through equations 4 and 5; among them, m_t is used to estimate the first-order moment of the gradient, i.e., the mean, and v_t is used to estimate the second-order moment of the gradient, i.e., the uncentered variance. When initialized, m_t and v_t are both 0, and m_t and v_t are updated as follows:

[0221] m_t=β_1*m_t-1+(1-β_1)*g_t, formula 4;

[0222] v_t=β_2*v_t-1+(1-β_2)*(g_t)^2, formula 5;

[0223] Among them, β_1 and β_2 are hyperparameters, and the values ​​of β_1 and β_2 are 0.9 and 0.999 respectively, m_t-1 and v_t-1 are the accumulated variables of the previous iteration, and g_t is the gradient of the loss function with respect to the model parameters;

[0224] The deviations of the first-order moment and the second-order moment are corrected by equations 6 and 7 respectively, and the corrected first-order moment and second-order moment are obtained:

[0225] m_t_hat=m_t / (1-(β_1)^t), Formula 6;

[0226] v_t_hat=v_t / (1-(β_2)^t), Formula 7;

[0227] Among them, t is the current iteration number, m_t_hat and v_t_hat are the corrected first-order moment and second-order moment respectively;

[0228] Using the modified first-order moment and second-order moment, combined with the learning rate α and the smoothing term ε, the model parameter θ is updated through equation 8 to obtain the trained model:

[0229] θ=θ-α*m_t_hat / (sqrt(v_t_hat)+ε), equation 8.

[0230] Through the above steps, the Adam optimizer updates the model parameters in each iteration, so that the loss function value gradually decreases. Compared with other optimization algorithms, such as gradient descent and stochastic gradient descent, the Adam optimizer can converge faster and has better stability.

[0231] In a specific embodiment, the specific training, verification and application process of the anomaly recognition model is as follows:

[0232] 1) Data preparation: The collected dataset includes 10,000 normal device images and 3,000 abnormal device images, each with a size of 256x256 pixels. The collected dataset is preprocessed, such as cropping, scaling, and normalization, so that the input data meets the model requirements.

[0233] 2) Model construction: Build a simple CNN model, including 2 convolutional layers, 2 maximum pooling layers, and 2 fully connected layers. The following is a simple CNN model built using Keras:

[0234] import keras

[0235] from keras.models import Sequential

[0236] from keras.layers import Conv2D,MaxPooling2D,Flatten,Dense,Dropout

[0237] model=Sequential()

[0238] model.add(Conv2D(32,(3,3),activation='relu',input_shape=(256,256,3)))

[0239] model.add(MaxPooling2D(pool_size=(2,2)))

[0240] model.add(Conv2D(64,(3,3),activation='relu'))

[0241] model.add(MaxPooling2D(pool_size=(2,2)))

[0242] model.add(Flatten())

[0243] model.add(Dense(128,activation='relu'))

[0244] model.add(Dropout(0.5))

[0245] model.add(Dense(2,activation='softmax'))

[0246] model.compile(optimizer='adam',loss='categorical_crossentropy',metrics=['accuracy'])

[0247] 3) Model training: Divide the dataset into training set and validation set, and use the training set to train the CNN model. Set the batch size to 64 and the training cycle to 20. In each training cycle, the CNN model extracts image features through the convolution layer, uses the pooling layer to reduce the feature dimension, and finally classifies through the fully connected layer. The optimizer (such as Adam) is used to update the model parameters to reduce the loss function value.

[0248] from keras.preprocessing.image import ImageDataGenerator

[0249] train_datagen=ImageDataGenerator(rescale=1. / 255)

[0250] validation_datagen=ImageDataGenerator(rescale=1. / 255)

[0251] train_generator=train_datagen.flow_from_directory(train_dir, target_size=(256,256), batch_size=64, class_mode='categorical')

[0252] validation_generator=validation_datagen.flow_from_directory(validation_dir,target_size=(256,256),batch_size=64,class_mode='categorical')

[0253] history=model.fit_generator(train_generator, steps_per_epoch=150, epochs=20, validation_data=validation_generator, validation_steps=50)

[0254] 4) Model verification: Use the verification set to evaluate the performance of the CNN model. Adjust the model parameters or change the model structure based on indicators such as accuracy and recall. The accuracy and recall obtained on the verification set are 95% and 93%, respectively, indicating good model performance.

[0255] 5) Model application: The trained anomaly recognition model is applied to the drone inspection process. The drone takes real-time images, and the model pre-processes the images before inputting them into the anomaly recognition model to identify abnormal conditions. For example, the identified abnormal conditions include equipment damage, leakage, and debris accumulation.

[0256] Suppose that during the inspection, a drone captures an image of an oil well. After preprocessing (such as scaling and normalization), the image is input into the trained anomaly recognition model. The following is a sample code for image preprocessing and classification:

[0257]

[0258] In this case, the anomaly recognition model successfully identified that there was a leak in the oil well. After receiving the report, the oilfield managers took timely measures to repair the leak.

[0259] In a specific embodiment, based on the existing CNN model, a more complex model ResNet-50 can be used for training to obtain the abnormal recognition model in the present invention to improve the recognition accuracy. The transfer learning technology is introduced to use the features learned by the pre-trained model on a large-scale data set to improve the model training efficiency. Figure 3 As shown, the anomaly recognition model is trained in the following way:

[0260] S201. Use the ImageNet dataset to pre-train the ResNet-50 model, obtain the pre-trained ResNet-50 model, and deploy it to the drone inspection system.

[0261] S202. Use multiple oilfield equipment images to perform transfer learning on the pre-trained ResNet-50 model to obtain the anomaly recognition model.

[0262] In a specific embodiment, the method of pre-training the ResNet-50 model using the ImageNet dataset described in step S201 to obtain the pre-trained ResNet-50 model includes:

[0263] S2011, preprocessing the ImageNet data set to obtain a preprocessed data set;

[0264] Download the ImageNet dataset, which contains about 10 million labeled images covering 1,000 categories. Before training the model, the dataset needs to be preprocessed. Preprocessing includes the following steps:

[0265] Image Resizing: Resize all images to the same size, for example 224x224 pixels.

[0266] Data augmentation: Randomly rotate, flip, and scale images to increase the diversity of training data and improve the generalization ability of the model.

[0267] Normalization: Scaling image pixel values ​​to between 0 and 1 helps speed up training and improve model performance.

[0268] One-hot encoding of labels: Convert category labels into one-hot vectors of length 1000 to facilitate model learning.

[0269] S2012. Use the preprocessed data set to train the ResNet-50 model to obtain a pretrained ResNet-50 model.

[0270] First, build a model. In this embodiment, a predefined model provided in the deep learning framework TensorFlow is used. These models have implemented the architecture of ResNet-50, and the model can be created by calling the corresponding interface. Secondly, compile the model, specifically: specify the loss function, optimizer and evaluation index for the model. The loss function used in the present invention is the cross-entropy loss function (Cross-Entropy Loss), the optimizer is the Adam optimizer (Adam Optimizer), and the evaluation index includes accuracy (Accuracy), recall (Recall), precision (Precision) and F1 score (F1-Score). The loss function is used to measure the performance of the model during the training process, the optimizer is used to adjust the weight of the model to minimize the loss, and the evaluation index is used to evaluate the performance of the model on the validation set. After preparing the data set, preprocessing the data, building and compiling the model, the preprocessed data set ResNet-50 model is used for training. During the training process, the model will learn a large number of common features on the ImageNet data set. During the model training process, regularly evaluating the model's performance on the validation set can help monitor the model's learning progress so that model parameters or training strategies can be adjusted in a timely manner when overfitting or underfitting occurs.

[0271] During the model training process, the specific steps for model evaluation are as follows:

[0272] (a) Preprocessing the validation set: First, the image data in the validation set is preprocessed the same way as the training set, such as cropping, scaling, and normalization.

[0273] (b) Perform model inference: Input the preprocessed validation set data into the model to obtain the prediction results.

[0274] (c) Calculate evaluation indicators: Based on the model's prediction results and true labels on the validation set, calculate various evaluation indicators such as accuracy, recall, precision, F1 score, etc.

[0275] (d) Analyze the evaluation results: Based on the evaluation indicators, the performance of the model can be analyzed to determine whether the model is overfitting or underfitting. For example, if the performance on the training set is very good, but the performance on the validation set is poor, it may be overfitting; on the contrary, if the performance on both the training set and the validation set is poor, it may be underfitting.

[0276] (e) Adjust model parameters or training strategies: Based on the evaluation results, the following measures can be taken to adjust model parameters or training strategies:

[0277] Adjust the learning rate: If the accuracy improves slowly or fluctuates greatly, reduce the learning rate.

[0278] Use the learning rate decay strategy: As training progresses, gradually reduce the learning rate to improve the model convergence speed and accuracy.

[0279] Increase or decrease the number of network layers: If overfitting occurs, reduce the number of network layers; if underfitting occurs, increase the number of network layers.

[0280] Adjust kernel size, stride, or padding: Change the parameters of the convolutional layer to change the representation power of the model.

[0281] Increase or decrease the number of neurons: Adjust the number of neurons in the fully connected layers or convolutional layers to increase model complexity or reduce the risk of overfitting.

[0282] Add data enhancement operations: By performing random transformations on training images (such as flipping, rotating, scaling, etc.), the model generalization ability is increased and the risk of overfitting is reduced.

[0283] Early stopping strategy: If the performance on the validation set does not improve significantly after a certain number of training cycles, terminate the training early to avoid overfitting.

[0284] (f) Repeat the training and evaluation process: Based on the evaluation results and the adjusted model parameters or training strategy, continue training. After each training cycle, perform model evaluation again to monitor changes in model performance. This process will continue until the model performance reaches a satisfactory level or the preset maximum training cycle is reached.

[0285] (g) Model selection: During the training and evaluation process, a variety of different parameters and structures are tried. The best model is selected based on the performance on the validation set.

[0286] (h) Test set evaluation: After the training and validation process is completed, the final selected model is evaluated using the test set. This can provide a more accurate estimate of the model's performance in real-world applications.

[0287] Model evaluation is an iterative process. By continuously training the model on the training set and evaluating the model performance on the validation set, model problems (such as overfitting or underfitting) can be identified, and model parameters or training strategies can be adjusted based on the evaluation results to achieve optimal performance. Throughout the process, model performance can be monitored and adjusted accordingly based on multiple evaluation indicators such as accuracy, recall, precision, F1 score, etc.

[0288] After training is complete, save the pre-trained ResNet-50 model for subsequent fine-tuning on oilfield equipment images. Deploy the pre-trained ResNet-50 model to the drone inspection system. This includes the following steps:

[0289] a) Convert the model to a format suitable for the drone device, such as TensorFlow Lite or ONNX.

[0290] b) Load the converted model onto the UAV device.

[0291] c) Write image processing and recognition code for the drone inspection system to analyze images acquired from the drone camera in real time.

[0292] d) Use the recognition results to guide the behavior of the drone, such as adjusting the inspection route, reporting abnormal situations, etc.

[0293] In a specific embodiment, the pre-trained ResNet-50 model is fine-tuned using oilfield equipment images to obtain a final anomaly recognition model. The above step S202 uses multiple oilfield equipment images to perform transfer learning on the pre-trained ResNet-50 model to obtain the anomaly recognition model, including:

[0294] S2021. Import a pre-trained ResNet-50 model, and remove the last fully connected layer of the pre-trained ResNet-50 model;

[0295] S2022. Adding a new fully connected layer to the pre-trained ResNet-50 model according to the number of categories of oilfield equipment; wherein the number of output units of the new fully connected layer corresponds to the number of categories;

[0296] For example, if there are two classes (normal and abnormal), the number of output units in the fully connected layer is 2.

[0297] S2023. Keep the weight of the convolutional layer of the pre-trained ResNet-50 model unchanged, and update the weight of the newly added fully connected layer to obtain an updated ResNet-50 model;

[0298] S2024. Use multiple oilfield equipment images to train the updated ResNet-50 model to obtain the anomaly recognition model.

[0299] Use oilfield equipment images for training. During the training process, the model will learn features related to oilfield equipment and improve recognition accuracy.

[0300] In a specific embodiment, the above-mentioned S2024 uses a plurality of oilfield equipment images to train the updated ResNet-50 model to obtain the anomaly recognition model, including:

[0301] Dividing the plurality of oilfield equipment images into a training set and a validation set according to a preset ratio;

[0302] According to the preset learning rate, batch size and training cycle, the updated ResNet-50 model is trained using the training set, and the model performance is evaluated using the validation set to obtain the anomaly recognition model.

[0303] The training process of the anomaly recognition model is described through a specific example:

[0304] 1) Prepare the dataset

[0305] A total of 3,000 images of oilfield equipment were collected, including images of abnormal conditions such as leakage, damage, blockage, and normal equipment. These images were randomly divided into a training set (2,400 images) and a validation set (600 images) in a ratio of 8:2.

[0306] 2) Select a pre-trained model

[0307] The ResNet-50 model pre-trained on the ImageNet dataset is selected as the base model. The ImageNet dataset contains 1,000 categories and about 10 million labeled images. This model has learned a wealth of general features and can be used for oilfield equipment anomaly detection tasks.

[0308] 3) Fine-tune the model

[0309] The last fully connected layer of the ResNet-50 model is removed and replaced with a new fully connected layer with the same number of output nodes as the number of categories of the target task (for example, 4 categories: normal, leaking, damaged, and blocked). Next, the newly added fully connected layer is trained using the oilfield equipment image dataset while keeping the weights of other layers unchanged.

[0310] 4) Training process

[0311] The learning rate is set to 0.001, the batch size is 32, and the training is performed for 20 cycles. During the training process, the training set is used for training, and the validation set is used for model performance evaluation. After 20 cycles of training, it is found that the accuracy of the model on the validation set reaches 95%.

[0312] 5) Model application

[0313] The trained anomaly recognition model is deployed to the drone inspection system to perform real-time anomaly detection on the captured oilfield equipment images. Assume that during a certain inspection, the system captures an equipment image, and the model predicts that the equipment is "damaged" with a confidence level of 96%. The drone sends this information back to the monitoring center, and the engineer determines that the equipment is at risk of damage based on the prediction results and confidence level, and immediately dispatches maintenance personnel for inspection and repair.

[0314] By using the ResNet-50 pre-trained model and transfer learning technology, the accuracy of oilfield equipment anomaly detection is improved, and a more intelligent and efficient inspection is achieved. The present invention adopts transfer learning technology and uses the ResNet-50 pre-trained model to identify abnormal conditions of oilfield equipment. Compared with the existing CNN model, this technology can improve the accuracy and robustness of anomaly recognition.

[0315] In the embodiment of the present invention, a 5G network with a bandwidth of 20MHz is used to realize real-time image and data transmission between the drone and the monitoring center. H.265 video encoding technology is used in the data transmission process to reduce bandwidth occupancy and delay. The monitoring center uses high-performance servers and large-screen display devices to receive and analyze the data sent back by the drone, monitor the status of the drone, and display the inspection results in a visual form. The present invention uses wireless communication technology to transmit the images and sensor data collected during the drone inspection process to the control center in real time, so that oilfield managers can understand the inspection situation in real time, detect and handle abnormalities in time, and improve production efficiency and safety.

[0316] In a specific embodiment, the identified abnormal conditions can also be structured so that oilfield managers can quickly obtain equipment failure information. The method also includes:

[0317] S301 : Use a pre-established multi-task model to perform structured processing on the identified abnormal conditions to obtain structured abnormal data.

[0318] First, define different categories for abnormal conditions, such as leakage, damage, blockage, etc. When training the model, use these categories as target variables so that the model can identify these abnormal conditions.

[0319] a) Analyze the historical failure data of oilfield equipment and find out the main failure types.

[0320] b) Define category labels based on fault types. Classify abnormal conditions into leakage, damage, and blockage categories.

[0321] c) Collect a large number of representative images for each category as training data. Make sure these images cover a variety of devices and environmental conditions.

[0322] d) Divide the training data into training set, validation set and test set for model training, validation and evaluation.

[0323] Each class is assigned an integer label (0-leaked, 1-damaged, 2-blocked) for model training.

[0324] When identifying abnormal conditions, detailed fault information is extracted from the image, including the faulty device, fault location, and fault level. This is achieved by adding multiple tasks to the output layer of the ResNet-50 model. The details are as follows:

[0325] a) Use the pre-trained ResNet-50 model as the base model and remove the last fully connected layer to get a new model that can extract image features. Figure 4 As shown, a global average pooling layer is also included to reduce the dimension of the feature vector, reduce the computational complexity and prevent overfitting.

[0326] b) Add additional fully connected layers to the new model. Add a fully connected layer 1 for faulty device prediction, a second fully connected layer 2 for fault location prediction, and a third fully connected layer 3 for fault level prediction. Among them, fully connected layer 1 is connected to the global average pooling layer to predict whether the device is faulty; fully connected layer 2 is connected to the global average pooling layer to predict the location of the fault; fully connected layer 3 is connected to the global average pooling layer to predict the level of the fault.

[0327] c) During the training process, a multi-task learning method is used. For each task (faulty device, fault location), a separate cross entropy loss function is used.

[0328] d) During the training process, the loss functions of all tasks are weighted and summed to get the total loss. By optimizing the total loss, the training model can learn multiple tasks at the same time.

[0329] e) Use the training set and validation set to train the model until the performance of the model on the validation set reaches a satisfactory level.

[0330] f) Use the test set to evaluate the performance of the model to ensure that the model can accurately predict the anomaly category and extract detailed fault information.

[0331] The training process of the multi-task model in S301 is described by a specific embodiment:

[0332] 1) Data preparation:

[0333] A set of oilfield equipment abnormal data was collected, totaling 1,000 records. Each record contains information such as image files, abnormal categories (such as leakage, damage, blockage, etc.), fault equipment type, fault location, and fault level. These data were randomly divided into a training set (800 records) and a validation set (200 records) in a ratio of 8:2.

[0334] 2) Data preprocessing:

[0335] The image data is normalized and scaled to the same size (e.g., 224x224 pixels), and each pixel value is normalized to be between 0 and 1. At the same time, labels such as anomaly category, faulty device type, fault location, and fault level are one-hot encoded to facilitate training of the multi-task model.

[0336] 3) Building a multi-task model:

[0337] Based on the ResNet-50 model, the last fully connected layer is removed and replaced with multiple fully connected layers for predicting different tasks (anomaly category, faulty device type, fault location, and fault level). The number of output nodes of each fully connected layer should be the same as the number of categories of the corresponding task.

[0338] 4) Training process:

[0339] The learning rate is set to 0.001, the batch size is 32, and the training is performed for 30 cycles. During the training process, the training set is used for training, and the validation set is used for model performance evaluation. After 30 cycles of training, it is found that the accuracy of the model on the validation set is: 95% for abnormal category, 90% for faulty equipment type, 88% for fault location, and 92% for fault level.

[0340] 5) Model application:

[0341] The trained multi-task model is deployed in the drone inspection system to perform real-time anomaly detection and structured information extraction on the real-time captured oilfield equipment images. During a certain inspection, the system captured an equipment image, and the model predicted the following results: anomaly category "damage", confidence level 96%, faulty equipment type "pump", confidence level 92%, fault location "inlet valve", confidence level 89%, fault level "serious", confidence level 93%.

[0342] The present invention realizes multi-task recognition of abnormal conditions by adding multiple tasks to the output layer of the image recognition model. This innovative technology enables the model to simultaneously identify information such as fault type, faulty equipment, and fault location, further improving the efficiency of fault handling.

[0343] In a specific embodiment, the method further comprises:

[0344] S302. After the inspection task is completed, an inspection report is generated according to the structured abnormal data; wherein the inspection report includes equipment abnormality classification statistics, abnormal equipment detailed information, inspection route map and drone flight data.

[0345] Combine the anomalies identified by the model and their related information into a structured report. This is done by creating a Python dictionary or class. Once the structured report is created, send it to oilfield managers so they can quickly get information about equipment failures. This is done via email, text messages, or other communication channels. To facilitate analysis and tracking of equipment failures, store the generated structured anomaly reports in a database or other data storage system. Oilfield managers can view historical data as needed to analyze equipment failure modes and trends. Use Python's sqlite3 library to store anomaly reports in a SQLite database.

[0346] The existing drone inspection system mainly conducts inspections through pre-set routes, and the inspection route cannot be adjusted in real time. The abnormal recognition ability is limited, and it is difficult to meet the inspection needs in complex environments. In addition, the intelligent obstacle avoidance function is relatively weak, and safety accidents such as collisions are prone to occur. In order to solve this technical problem, the present invention selects the A algorithm as the obstacle avoidance algorithm. The A* algorithm combines the shortest path advantage of the Dijkstra algorithm and the rapidity of the heuristic search. The core idea is to use the heuristic function to evaluate the nodes and reduce the search space.

[0347] In a specific embodiment, the method provided by the present invention further comprises:

[0348] S401, updating the sensor data on the map according to the sensor data collected in real time to obtain updated map information; wherein the sensor data includes the location and size of the obstacle;

[0349] Integrate data from various sensors on the drone (such as lidar, cameras, etc.). For example, lidar can provide obstacle distance information, and cameras can provide obstacle shape and size information. This data will be used to determine which grid cells contain obstacles.

[0350] S402, if a new obstacle is encountered, using the updated map information and a pre-built obstacle avoidance model to re-plan an obstacle avoidance path to obtain an optimal obstacle avoidance path around the obstacle; wherein the obstacle avoidance model is built based on an A* algorithm;

[0351] The specific process of building an obstacle avoidance model based on the A* algorithm is as follows: first, the flight space of the drone is divided into multiple grid cells, each of which represents a possible flight position. Then, the actual cost g'(n) and the heuristic cost h'(n) are calculated for each grid cell, and the sum is used to obtain the total cost f'(n) = g'(n) + h'(n). Among them, the actual cost g'(n) represents the cost from the starting point to the current node n, and the heuristic cost h'(n) represents the estimated cost from the current node n to the target position.

[0352] Actual cost g'(n): represents the cumulative cost from the starting point to the current node n. It refers to the sum of the distances between nodes on the path that have been traversed. In a grid map, the distance between adjacent nodes is calculated using Euclidean distance. At each step of the A* algorithm, the actual cost from the starting point to the adjacent node can be calculated by adding the distance from the current node to its adjacent node to the g'(n) of the current node.

[0353] When calculating the actual cost g'(n), first set g(n) at the starting point to 0. Then, during the search process, each time a new node is visited, the actual cost from the starting point to the new node needs to be calculated.

[0354] Assume that the current node is current_node and the new node is next_node, then the distance from current_node to next_node is recorded as distance(current_node,next_node), which is calculated using Euclidean distance.

[0355] Each time a new node is accessed, its actual cost g'(n) and h'(n) need to be updated. Specifically, for the new node next_node, its actual cost g'(n) is calculated as follows:

[0356] g'(next_node)=g'(current_node)+distance(current_node,next_node)

[0357] By updating the actual cost g(n) of the node at each step, the actual cost from the starting point to any node can be accumulated and calculated. The heuristic cost h'(n) is calculated using the Euclidean distance.

[0358] During the flight of the drone, obstacles are identified based on sensor data, and the A* algorithm is used to plan the optimal obstacle avoidance path in real time. During the search process, when an obstacle is encountered, the best neighboring node is selected according to the total cost f'(n) for expansion until a safe path around the obstacle is found.

[0359] The details are as follows:

[0360] Step 1 - Collect sensor data: Use the sensors onboard the drone (such as radar, lidar, camera, etc.) to collect information about the surrounding environment in real time, including the location and size of obstacles.

[0361] Step 2 - Update the map: Update the obstacle information to the map based on the collected sensor data. This can be achieved by marking the grid cell where the obstacle is located as "inaccessible".

[0362] Step 3 - Re-planning the path: When the drone encounters a new obstacle, it needs to recalculate the best path from the current position to the target position. Using the updated map information, the A* algorithm is applied to re-plan the obstacle avoidance path. During the search process, the best neighboring node is selected for expansion based on the total cost f'(n). If a node is marked as "inaccessible", it will not be added to the search list. Because the judgment of node accessibility, the calculation of cost values, and the maintenance of search lists and closed lists are the key to realizing real-time obstacle avoidance planning of the A* algorithm.

[0363] Step 4 - Path Tracking: After finding a new path around the obstacle, the drone will fly along the new path. During the flight, the drone needs to continue to collect sensor data, update the map information in real time, and repeat steps 3 and 4 when encountering new obstacles.

[0364] S403: Integrate the updated inspection path with the optimal obstacle avoidance path to generate a new inspection path, and perform inspection according to the new inspection path.

[0365] When the drone completes the obstacle avoidance operation, it needs to update its inspection path. First, record the nodes visited during the obstacle avoidance process, then integrate the original inspection path with the obstacle avoidance path to generate a new inspection path. The specific operation can be achieved by modifying the corresponding nodes in the original inspection path to maintain the integrity of the inspection task.

[0366] The present invention adds an intelligent obstacle avoidance function to the UAV, so that it can automatically identify and avoid obstacles during the inspection process, which not only improves the safety of the UAV but also reduces the inspection risk.

[0367] In a specific embodiment, the method further comprises:

[0368] During the inspection process, the weather conditions are obtained in real time, and the updated inspection path is adjusted in real time according to the weather conditions to obtain the adjusted inspection path, and the inspection is carried out according to the adjusted inspection path. If bad weather suddenly occurs during the flight of the drone, the inspection route can be adjusted according to the weather conditions, and the equipment that is greatly affected by the weather is inspected first. Specifically, the inspection sub-path from the current position to the equipment that is greatly affected by the weather is planned by the A algorithm, and the inspection sub-path from the equipment that is greatly affected by the weather to the target point is planned by the A* algorithm. The specific process can refer to the above description of first using the A algorithm to plan the inspection sub-path from the current position to the oilfield equipment with abnormal conditions, and then using the A* algorithm to re-plan the inspection sub-path from the oilfield equipment with abnormal conditions to the target point, which will not be repeated here. The present invention combines meteorological information and topographic data, so that the drone inspection system can adapt to complex environmental changes, and improves the reliability and stability of the drone inspection system under different climate and terrain conditions.

[0369] Embodiment 2

[0370] This example describes the specific process of performing an inspection task in an oil field:

[0371] During an inspection mission at an oil field, there are 10 oil field equipment that need to be inspected. The following are the equipment numbers and their corresponding location coordinates (unit: meter):

[0372] Device 1: (100,200);

[0373] Device 2: (300,600);

[0374] Device 3: (150,800);

[0375] Device 4: (600,900);

[0376] Device 5: (1000,1200);

[0377] Device 6: (1200,500);

[0378] Device 7: (800, 200);

[0379] Device 8: (1300,1300);

[0380] Device 9: (400,400);

[0381] Device 10: (1100,100);

[0382] Coordinates of the drone’s take-off point: (0,0);

[0383] Drone target location coordinates: (1400,1400).

[0384] Implementation steps:

[0385] 1) Generate equipment inspection sequence

[0386] The Dijkstra algorithm is used to calculate the shortest inspection path. First, the Euclidean distance between oilfield equipment is used as the edge weight to construct a complete graph. Then, the Dijkstra algorithm is run with the takeoff point as the source point to find the shortest path between each device. Finally, the inspection sequence is generated based on the shortest path.

[0387] Assume that the inspection order calculated by the Dijkstra algorithm is:

[0388] Take-off point->device 1->device 9->device 2->device 3->device 4->device 5->device 8->device 6->device 10->device 7->target location.

[0389] 2) Equipment abnormality identification

[0390] Use a transfer learning model based on ResNet-50, i.e., anomaly recognition model, to identify equipment anomalies. Assume that the model has been fine-tuned using oilfield equipment images. During the inspection process, drones take pictures of the equipment and use the model to identify the equipment status of the oilfield equipment in real time.

[0391] During the inspection process, the model identified the following anomalies:

[0392] Equipment 2: Leakage, serious;

[0393] Equipment 6: Damage, minor;

[0394] Device 10: Blocked, medium.

[0395] 3) Structuring of abnormal data

[0396] The identified abnormal conditions are structured so that oilfield managers can quickly obtain equipment failure information. For example, abnormal conditions are classified into different categories (such as leakage, damage, blockage, etc.), and detailed failure information (such as failed equipment, failure location, failure level, etc.) is provided for each category.

[0397] The following are examples of structured exception data:

[0398] {

[0399] "leakage":[

[0400] {

[0401] "Device":"Device 2",

[0402] "Position":"(300,600)",

[0403] "Level": "Severe"

[0404] }

[0405] ],

[0406] "damage":[

[0407] {

[0408] "Device":"Device6",

[0409] "Position":"(1200,500)",

[0410] "Level": "Mild"

[0411] }

[0412] ],

[0413] "Block":[

[0414] {

[0415] "Device":"Device10",

[0416] "Position":"(1100,100)",

[0417] "Level":"Medium"

[0418] } ]

[0420] }

[0421] 4) Drone intelligent obstacle avoidance

[0422] During the inspection process, the drone is equipped with intelligent obstacle avoidance function. The A* algorithm is used as the obstacle avoidance algorithm. The drone is equipped with laser radar and camera as sensors to transmit data in real time. During the flight of the drone, obstacles are identified through sensor data and the obstacle avoidance path is updated in real time.

[0423] For example, when the drone was flying from device 3 to device 4, it found a 100-meter-high tower blocking the path. The A* algorithm automatically calculated a safe detour route to avoid the tower. After completing the obstacle avoidance, the original inspection path was integrated with the obstacle avoidance path to generate a new inspection path.

[0424] 5) Adjust inspection routes in real time

[0425] During the inspection process, the inspection route can be adjusted in real time by real-time monitoring of weather conditions, equipment status and other information. For example, if bad weather suddenly occurs during the flight of the drone, the inspection route can be adjusted according to the weather conditions, giving priority to inspecting equipment that is most affected by the weather.

[0426] 6) Generate inspection report

[0427] After the inspection task is completed, an inspection report is generated based on the structured abnormal data. The report should include the following content: equipment abnormality classification statistics, abnormal equipment details, inspection route map, drone flight data, etc. Oilfield managers can quickly understand the equipment status based on the report and formulate maintenance plans.

[0428] In this implementation case, the drone successfully completed the oilfield equipment inspection task. By applying technologies such as the Dijkstra algorithm, ResNet-50 model, and A algorithm, the inspection efficiency, recognition accuracy, and flight safety were improved. Structured abnormal data and inspection reports provide oilfield managers with intuitive and comprehensive equipment status information.

[0429] The present invention is based on Dijkstra algorithm, A algorithm and A* algorithm to realize dynamic path planning and automatic inspection route planning, and improve inspection efficiency; the abnormal conditions in the equipment and environment are identified in real time by image recognition technology, and the abnormal condition identification ability of the abnormal recognition model in the equipment and environment is improved by using convolutional neural network (CNN) and transfer learning technology. By performing structured processing on the identified abnormal conditions, structured processing abnormal information is generated, which is convenient for oilfield managers to quickly obtain equipment fault information. The present invention can also add intelligent obstacle avoidance function to the drone to reduce the inspection risk; and provide real-time fault reports, which is convenient for oilfield managers to quickly locate and handle equipment faults and improve fault handling efficiency. The results of the present invention are aimed at the specific needs of oilfield equipment inspection, integrate multiple advanced technologies, and form a complete set of intelligent inspection solutions. This innovative achievement provides efficient and safe technical support for oilfield equipment management and has strong practical value. The results of the present invention have good scalability and can adapt to various oilfield equipment inspection scenarios. At the same time, it can also be extended to the fields of automated inspection and fault diagnosis of other industrial equipment.

[0430] Embodiment 3

[0431] The embodiment of the present invention provides an oilfield unmanned aerial vehicle inspection system, referring to Figure 5 As shown, including:

[0432] Static path planning module 501, used to plan the initial inspection path between the oilfield equipment in the inspection area and the target point of the initial inspection path by the UAV using Dijkstra algorithm;

[0433] The identification module 502 is used to perform inspection based on the initial inspection path, and during the inspection process, use a pre-established abnormality identification model to identify the real-time captured oilfield equipment image to determine whether there is any oilfield equipment in an abnormal condition;

[0434] The dynamic path planning module 503 is used to use the A algorithm to plan the inspection sub-path from the current position to the oilfield equipment where the abnormal condition occurs, and use the A* algorithm to re-plan the inspection sub-path from the oilfield equipment where the abnormal condition occurs to the target point to obtain an updated inspection path; and, perform inspections according to the updated inspection path, repeat the above-mentioned identification process and the process of re-planning the inspection path until all oilfield equipment are inspected.

[0435] In a specific embodiment, the oilfield UAV inspection system further includes:

[0436] The intelligent obstacle avoidance module 504 is used to update the sensor data on the map according to the sensor data collected in real time to obtain updated map information; wherein the sensor data includes the location and size of the obstacle; if a new obstacle is encountered, the obstacle avoidance path is replanned using the updated map information and a pre-built obstacle avoidance model to obtain an optimal obstacle avoidance path that bypasses the obstacle; wherein the obstacle avoidance model is built based on the A* algorithm; the updated inspection path is integrated with the optimal obstacle avoidance path to generate a new inspection path, and inspection is performed according to the new inspection path.

[0437] In a specific embodiment, the oilfield UAV inspection system further includes:

[0438] The adjustment module 505 is used to obtain weather conditions in real time during the inspection process, adjust the updated inspection path in real time according to the weather conditions to obtain an adjusted inspection path, and perform inspection according to the adjusted inspection path.

[0439] In a specific embodiment, the oilfield UAV inspection system further includes:

[0440] The structuring module 506 is used to perform structural processing on the identified abnormal conditions using a pre-established multi-task model to obtain structured abnormal data.

[0441] In a specific embodiment, the oilfield UAV inspection system further includes:

[0442] The inspection report generation module 507 is used to generate an inspection report based on the structured abnormal data after the inspection task is completed; wherein the inspection report includes equipment abnormality classification statistics, abnormal equipment detailed information, inspection route map and drone flight data.

[0443] The oilfield UAV inspection system provided by the embodiment of the present invention has similar implementation principles and technical effects to any of the aforementioned method embodiments, which will not be repeated here.

[0444] The present invention adopts a modular design concept to make each functional module independent of each other, which is convenient for maintenance and upgrading. This innovative design method provides convenience for the continuous optimization and improvement of the UAV inspection system.

[0445] Embodiment 4

[0446] An embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the oilfield drone inspection method described in any one of the aforementioned method embodiments is implemented.

[0447] The computer-readable storage medium may be included in the device / apparatus described in the above embodiment; or it may exist independently without being assembled into the device / apparatus. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0448] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, apparatus, or device.

[0449] Embodiment 5

[0450] An embodiment of the present invention provides an electronic device, referring to Figure 6 As shown, it includes a processor 111, a communication interface 112, a memory 113 and a communication bus 114, wherein the processor 111, the communication interface 112 and the memory 113 communicate with each other through the communication bus 114.

[0451] Memory 113, used for storing computer programs;

[0452] The processor 111 is used to implement the oilfield UAV inspection method described in any of the above method embodiments when executing the program stored in the memory 113.

[0453] The implementation principle and technical effect of the electronic device provided by the embodiment of the present invention are similar to those of any of the aforementioned method embodiments, and will not be repeated here.

[0454] The above-mentioned memory 113 can be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk or a ROM. The memory 113 has a storage space for program codes for executing any method steps in the above-mentioned method. For example, the storage space for program codes may include various program codes for implementing various steps in the above method respectively. These program codes can be read from or written into one or more computer program products. These computer program products include program code carriers such as hard disks, compact disks (CDs), memory cards or floppy disks. Such computer program products are usually portable or fixed storage units. The storage unit may have a storage segment or storage space arranged similarly to the memory 113 in the above-mentioned electronic device. The program code can be compressed, for example, in an appropriate form. Generally, the storage unit includes a program for executing the method steps according to an embodiment of the present invention, that is, a code that can be read by, for example, a processor 111, and when these codes are run by an electronic device, the electronic device performs the various steps in the method described above.

[0455] In this document, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. The orientation or position relationship indicated by the terms "upper", "lower", etc. is based on the orientation or position relationship shown in the accompanying drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0456] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. It should be noted that the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict. The present invention is not limited to any single aspect, any single embodiment, or any combination and / or permutation of these aspects and / or embodiments. Each aspect and / or embodiment of the present invention can be used alone, or in combination with one or more other aspects and / or other embodiments.

[0457] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. An oilfield drone inspection method, characterized in that: Including: Using the Dijkstra algorithm to plan the initial inspection path between each oilfield device in the inspection area for the UAV and the target points of the initial inspection path; Conducting inspections based on the initial inspection path, and during the inspection process, using a pre-established anomaly recognition model to identify the images of oilfield devices captured in real time to determine whether there are oilfield devices with abnormal conditions; If so, using the A algorithm to plan the inspection sub-path from the current position to the oilfield device with abnormal conditions, and using the A* algorithm to re-plan the inspection sub-path from the oilfield device with abnormal conditions to the target point to obtain an updated inspection path; Conducting inspections according to the updated inspection path, repeating the above recognition process and the process of re-planning the inspection path until all oilfield devices have been inspected.

2. The oilfield drone inspection method according to claim 1, characterized in that: The step of using the Dijkstra algorithm to plan the initial inspection path between each oilfield device in the inspection area for the UAV and the target points of the initial inspection path includes: Dividing the inspection area into multiple grid cells, and determining the weight of each grid cell according to the distance between oilfield devices and a pre-determined terrain level; Calculating the first distance from the starting point to each oilfield device and the second distance between each oilfield device according to the weight of each grid cell; Using the Dijkstra algorithm to determine the initial inspection path according to the first distance from the starting point to each oilfield device and the second distance between each oilfield device.

3. The oilfield drone inspection method according to claim 2, characterized in that: The step of using the Dijkstra algorithm to determine the initial inspection path according to the first distance from the starting point to each oilfield device and the second distance between each oilfield device includes: Setting the distance of the starting point to 0, creating set S and set Q, and adding all oilfield devices to set Q; Selecting the oilfield device with the smallest first distance from set Q to obtain oilfield device u, and adding oilfield device u to set S; If the oilfield device v in set Q satisfies Equation 1, then update dist[v] based on Equation 2 to generate the sub-path from oilfield device u to oilfield device v; dist[u]+dist_between(u,v)<dist[v], Equation 1; dist[v] = dist[u]+dist_between(u,v), Equation 2; Where, dist[u] represents the shortest path length from the starting point to oilfield device u, dist_between(u,v) represents the shortest path length from oilfield device u to oilfield device v, and dist[v] represents the shortest path length from the starting point to oilfield device v; If set Q is not empty, then using oilfield device v as the new starting point, obtaining the oilfield device with the smallest second distance from set Q to the new starting point, adding oilfield device v to set S, repeating the above update process and the process of obtaining oilfield devices from set Q until set Q is empty to obtain the initial inspection path.

4. The oilfield drone inspection method according to claim 1, characterized in that: The step of using the A* algorithm to re-plan the inspection sub-path from the oilfield device with abnormal conditions to the target point includes: Add the oilfield equipment with abnormal conditions and all oilfield equipment to the open list, take the oilfield equipment with abnormal conditions as the starting point, and take the target point as the target node; Based on Formula 3, the evaluation function value f(n) of each oilfield equipment is determined, and the oilfield equipment with the smallest evaluation function value is selected from the open list to obtain the current node; f(n)=g(n)+h(n), Formula 3; Among them, g(n) represents the actual distance from the starting point to the oilfield equipment n, and h(n) represents the heuristic estimated distance from the oilfield equipment n to the target node; Determine whether the current node is a target node; If not, taking the current node as a new starting point, traverse all oilfield equipment, calculate the new evaluation function value f(n) of each oilfield equipment by formula 3, and update the oilfield equipment in the open list based on the new evaluation function value f(n) to obtain an updated open list; The current node is added to the closed list, and the process of screening the oilfield equipment with the smallest evaluation function value from the open list and the process of judging are repeated until the target node is found or the open list is empty, thereby obtaining the inspection sub-path from the current position to the oilfield equipment where the abnormal condition occurs.

5. The oilfield drone inspection method according to claim 1, characterized in that: The anomaly recognition model is trained in the following way: Preprocessing the collected image data including normal devices and abnormal devices to obtain a processed data set; Dividing the processed data set into a first training set and a first validation set, and using the first training set to train a pre-established CNN model to obtain a trained model; wherein the CNN model includes a plurality of convolutional layers, a pooling layer, and a fully connected layer; The performance of the trained model is evaluated using the first validation set, and the model parameters of the trained model are adjusted or the model structure is changed according to indicators including accuracy, recall rate and F1 score until the model performance meets the preset requirements, thereby obtaining the anomaly recognition model.

6. The oilfield drone inspection method according to claim 5, characterized in that: The method of using the first training set to train the pre-established CNN model to obtain a trained model includes: Dividing the first training set into several batches of sub-training sets according to the batch size, inputting each batch of sub-training sets into the CNN model in batches to obtain prediction results, determining a loss function value according to the prediction results and the true labels, and calculating the gradient of the loss function value with respect to the model parameters; Based on the gradient of the loss function with respect to the model parameters, an optimizer is used to update the model parameters of the CNN model according to a preset learning rate and a smoothing term to obtain a trained model.

7. The oilfield drone inspection method according to claim 6, characterized in that: The method of updating the model parameters of the CNN model based on the gradient of the loss function with respect to the model parameters using an optimizer according to a preset learning rate and a smoothing term to obtain a trained model comprises: The two cumulative variables m_t and v_t of the model parameters are updated respectively by equations 4 and 5; among them, m_t is used to estimate the first-order moment of the gradient, and v_t is used to estimate the second-order moment of the gradient. m_t and v_t are updated as follows: m_t=β_1*m_t-1+(1-β_1)*g_t, formula 4; v_t=β_2*v_t-1+(1-β_2)*(g_t)^2, formula 5; Among them, β_1 and β_2 are hyperparameters, m_t-1 and v_t-1 are the accumulated variables of the previous iteration, and g_t is the gradient of the loss function with respect to the model parameters; The deviations of the first-order moment and the second-order moment are corrected by equations 6 and 7 respectively, and the corrected first-order moment and second-order moment are obtained: m_t_hat=m_t / (1-(β_1)^t), Formula 6; v_t_hat=v_t / (1-(β_2)^t), Formula 7; Among them, t is the current iteration number, m_t_hat and v_t_hat are the corrected first-order moment and second-order moment respectively; Using the modified first-order moment and second-order moment, combined with the learning rate α and the smoothing term ε, the model parameter θ is updated through equation 8 to obtain the trained model: θ=θ-α*m_t_hat / (sqrt(v_t_hat)+ε), equation 8.

8. The oilfield drone inspection method according to claim 1, characterized in that: The anomaly recognition model is trained in the following way: Use the ImageNet dataset to pre-train the ResNet-50 model, obtain the pre-trained ResNet-50 model, and deploy it to the drone inspection system; The anomaly recognition model is obtained by performing transfer learning on the pre-trained ResNet-50 model using multiple oilfield equipment images.

9. The oilfield drone inspection method according to claim 6, characterized in that: The ResNet-50 model is pre-trained using the ImageNet dataset to obtain a pre-trained ResNet-50 model, including: Preprocessing the ImageNet dataset to obtain a preprocessed dataset; The ResNet-50 model is trained using the preprocessed data set to obtain a pretrained ResNet-50 model.

10. The oilfield drone inspection method according to claim 9, characterized in that: The use of multiple oilfield equipment images to perform transfer learning on the pre-trained ResNet-50 model to obtain the anomaly recognition model includes: Import the pre-trained ResNet-50 model and remove the last fully connected layer of the pre-trained ResNet-50 model; According to the number of categories of oilfield equipment, adding a new fully connected layer to the pre-trained ResNet-50 model; wherein the number of output units of the new fully connected layer corresponds to the number of categories; The weights of the convolutional layer of the pre-trained ResNet-50 model remain unchanged, and the weights of the newly added fully connected layer are updated to obtain an updated ResNet-50 model; The updated ResNet-50 model is trained using multiple oilfield equipment images to obtain the anomaly recognition model.

11. The oilfield drone inspection method according to claim 10, characterized in that: The updated ResNet-50 model is trained using a plurality of oilfield equipment images to obtain the anomaly recognition model, including: Dividing the plurality of oilfield equipment images into a training set and a validation set according to a preset ratio; According to the preset learning rate, batch size and training cycle, the updated ResNet-50 model is trained using the training set, and the model performance is evaluated using the validation set to obtain the anomaly recognition model.

12. The oilfield drone inspection method according to claim 1, characterized in that: The method further comprises: According to the sensor data collected in real time, the sensor data is updated on the map to obtain updated map information; wherein the sensor data includes the location and size of the obstacle; If a new obstacle is encountered, the updated map information is used to replan the obstacle avoidance path using a pre-built obstacle avoidance model to obtain an optimal obstacle avoidance path around the obstacle; wherein the obstacle avoidance model is built based on the A* algorithm; The updated inspection path is integrated with the optimal obstacle avoidance path to generate a new inspection path, and inspection is performed according to the new inspection path.

13. The oilfield drone inspection method according to claim 12, characterized in that: If a new obstacle is encountered, the updated map information is used to replan the obstacle avoidance path using the pre-built obstacle avoidance model to obtain the optimal obstacle avoidance path around the obstacle, including: If a new obstacle is encountered, the current node where the new obstacle is located in the updated map information is obtained, multiple neighboring nodes of the current node are accessed, and the total cost f'(next_node) of each neighboring node is determined by Formula 9: f'(next_node)=g'(next_node)+h'(next_node), equation 9; Among them, g'(next_node) represents the cumulative cost from the starting point to the neighboring node n, and h'(n) represents the estimated cost from the neighboring node n to the target position; The neighboring nodes with the smallest total cost are screened to obtain the optimal neighboring node, and the optimal neighboring node is used as a new starting point to visit the new node, and the above-mentioned process of determining the total cost and the screening process are repeated until the obstacle is bypassed and the optimal obstacle avoidance path is obtained.

14. The oilfield drone inspection method according to claim 1, characterized in that: The method further comprises: During the inspection process, weather conditions are acquired in real time, the updated inspection path is adjusted in real time according to the weather conditions to obtain an adjusted inspection path, and the inspection is performed according to the adjusted inspection path.

15. The oilfield drone inspection method according to claim 1, characterized in that: The method further comprises: The identified abnormal conditions are structured using a pre-established multi-task model to obtain structured abnormal data.

16. The oilfield drone inspection method according to claim 15, characterized in that: The method further includes: after the inspection task is completed, generating an inspection report based on the structured abnormal data; wherein the inspection report includes equipment abnormality classification statistics, abnormal equipment detailed information, an inspection route map and drone flight data.

17. An oilfield drone inspection system, characterized in that: include: A static path planning module is used to plan the initial inspection path between the oilfield equipment in the inspection area and the target point of the initial inspection path by using the Dijkstra algorithm; An identification module is used to perform inspection based on the initial inspection path, and during the inspection process, use a pre-established abnormality identification model to identify the real-time captured oilfield equipment image to determine whether there is any oilfield equipment in an abnormal condition; A dynamic path planning module is used to use the A algorithm to plan the inspection sub-path from the current position to the oilfield equipment with abnormal conditions, and use the A* algorithm to re-plan the inspection sub-path from the oilfield equipment with abnormal conditions to the target point to obtain an updated inspection path; And, inspection is carried out according to the updated inspection path, and the above identification process and inspection path re-planning process are repeated until all oilfield equipment are inspected.

18. The oilfield UAV inspection system according to claim 17, characterized in that: Also includes: The intelligent obstacle avoidance module is used to update the sensor data on the map according to the sensor data collected in real time to obtain updated map information; wherein the sensor data includes the location and size of the obstacle; if a new obstacle is encountered, the obstacle avoidance path is replanned using the updated map information and a pre-built obstacle avoidance model to obtain an optimal obstacle avoidance path that bypasses the obstacle; wherein the obstacle avoidance model is built based on the A* algorithm; the updated inspection path is integrated with the optimal obstacle avoidance path to generate a new inspection path, and inspection is performed according to the new inspection path.

19. The oilfield UAV inspection system according to claim 17, characterized in that: Also includes: The adjustment module is used to obtain weather conditions in real time during the inspection process, adjust the updated inspection path in real time according to the weather conditions to obtain an adjusted inspection path, and perform inspections according to the adjusted inspection path.

20. The oilfield UAV inspection system according to claim 17, characterized in that: Also includes: The structuring module is used to perform structured processing on the identified abnormal conditions using a pre-established multi-task model to obtain structured abnormal data.

21. The oilfield unmanned aerial vehicle inspection system according to claim 20, characterized in that: Also includes: The inspection report generation module is used to generate an inspection report based on the structured abnormal data after the inspection task is completed; wherein the inspection report includes equipment abnormality classification statistics, abnormal equipment detailed information, inspection route map and drone flight data.

22. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the oilfield UAV inspection method as described in any one of claims 1 to 16 is implemented.

23. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory, used to store computer programs; The processor is used to implement the oilfield UAV inspection method as described in any one of claims 1 to 16 when executing the program stored in the memory.

Images