Unmanned aerial vehicle self-adaptive inspection method and system for industrial three-proofing field

Abstract

The invention provides an unmanned aerial vehicle self-adaptive inspection method and system for the industrial three-proofing field. The method comprises the steps of obtaining a to-be-recognized image returned by an unmanned aerial vehicle; generating an inspection object atlas according to the image objects of the image areas in the to-be-identified image and the causal relation between the different image objects; the inspection object atlas comprises a plurality of object nodes and relation connecting lines, one object node is used for representing an image object of one image area, and one relation connecting line is used for representing a causal relation between the image objects represented by the connected object nodes; when image recognition is carried out on the to-be-recognized image according to the patrolling object atlas, screening is carried out from the patrolling object atlas to obtain contrast object nodes capable of being recognized at the same time; and carrying out simultaneous identification processing on the image objects represented by the contrast object nodes to obtain abnormal identification evaluation information of the to-be-identified image. Based on the method, the image recognition speed can be increased, the rapid generation of the inspection instruction is improved, and the inspection efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to an unmanned aerial vehicle adaptive inspection method and system for the industrial three-defense field. Background Art

[0002] In the field of industrial automation and intelligent manufacturing, especially in the industrial three-proof (dustproof, waterproof, and shockproof) environment, drone inspection plays an increasingly important role as an efficient and flexible monitoring method. The industrial three-proof environment places extremely high demands on the stability and durability of equipment. Drone inspection can achieve remote and real-time monitoring of industrial facilities in these environments to ensure production safety and efficiency. However, traditional inspection methods face many challenges when dealing with complex industrial facilities in industrial three-proof environments. These facilities, for example, contain a large number of interrelated and environmentally susceptible equipment, such as storage tanks, pipelines, and valves in chemical plants. These devices are not only numerous, but their operating status is directly related to production safety and efficiency. Traditional inspection methods often rely on manual interpretation of images taken by drones, which is not only time-consuming and labor-intensive, but also easily affected by human factors, resulting in limited inspection efficiency and accuracy. In order to improve inspection efficiency and accuracy, researchers have begun to explore drone automatic inspection technologies based on computer vision and machine learning. These technologies can achieve rapid detection and abnormal identification of industrial facilities by automatically processing and analyzing images taken by drones. However, most of the existing automatic inspection technologies only focus on the recognition of single image objects, ignoring the causal relationship between image objects and the potential impact of the industrial three-proof environment on the equipment status. This leads to false alarms and missed alarms in complex industrial scenarios, especially in industrial three-proof environments. In addition, when performing image recognition, the existing automatic inspection technology, for example, adopts a serial processing method, that is, processing image objects one by one. This processing method is inefficient when processing large-scale image data, and it is difficult to meet the needs of real-time inspection in industrial three-proof environments. Summary of the invention

[0003] In view of this, the present application provides a method and system for self-adaptive inspection of unmanned aerial vehicles in the field of industrial defense. The technical solution of the present application is implemented as follows: On the one hand, the present application provides an adaptive inspection method for unmanned aerial vehicles (UAVs) in the field of industrial three-defense, comprising: obtaining an image to be identified of a target three-defense area photographed by a UAV; wherein the image to be identified includes multiple image areas, and an image object in one image area represents a facility to be inspected; generating an inspection object map based on the image objects in each image area in the image to be identified and the causal relationship between different image objects; the inspection object map includes multiple object nodes and relationship connection lines, one object node is used to represent an image object in an image area, and one relationship connection line is used to characterize the causal relationship between the image objects represented by the connected object nodes; when performing image recognition on the image to be identified based on the inspection object map, screening out control object nodes that can be simultaneously identified from the inspection object map; performing simultaneous recognition processing on the image objects represented by the control object nodes to obtain abnormal recognition evaluation information of the image to be identified; generating a route confirmation instruction based on the abnormal recognition evaluation information, and sending the route confirmation instruction to the UAV so that the UAV responds to the route confirmation instruction for inspection.

[0004] On the other hand, the present application also provides a computer system, including a memory and a processor, wherein the memory stores a computer program that can be executed on the processor, and the processor implements the steps in the above method when executing the program.

[0005] Beneficial effects of the present application: In the embodiment of the present application, an image to be identified returned by a drone is obtained, and the image to be identified includes multiple image areas, and an image object in an image area represents a facility to be detected; based on the image objects in each image area in the image to be identified and the causal relationship between different image objects, a patrol object map can be generated. Among them, an object node in the patrol object map is used to represent an image object, and a connecting line represents the causal relationship between image objects represented by different object nodes. Based on this, the present application can convert the image to be identified into a patrol object map, and the causal relationship between each image object in the image to be identified can be clearly presented based on the patrol object map, which helps to perform concurrent image recognition based on the patrol object map. In other words, when performing image recognition on the image to be identified based on the patrol object map, the control object nodes that can be simultaneously identified can be screened in the patrol object map, and the image objects represented by the control object nodes that can be simultaneously identified can be simultaneously identified. In this way, the control object nodes that can be simultaneously identified can be directly and quickly determined based on the generated patrol object map. The control object nodes that can be identified simultaneously are identified simultaneously, so that the identification of the corresponding image objects in each image area in the image to be identified is completed concurrently, so as to increase the recognition speed of the image to be identified, thereby improving the rapid generation of inspection instructions and increasing inspection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1 A schematic diagram of the implementation process of a drone adaptive inspection method for the industrial three-defense field provided in an embodiment of the present application.

[0007] Figure 2 A hardware entity diagram of a computer system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0008] The embodiment of the present application provides a method for self-adaptive inspection of drones in the field of industrial defense, which can be executed by a processor of a computer system. The computer system can refer to a device with data processing capabilities such as a server, a laptop, a tablet computer, a desktop computer, a mobile device, etc. The computer system is communicatively connected to at least one drone.

[0009] Figure 1 A schematic diagram of the implementation process of a UAV adaptive inspection method for industrial three-defense fields provided in the embodiment of the present application is as follows: Figure 1 As shown, the method includes: Step S100: Acquire an image to be identified of a target three-defense area photographed by a drone; wherein the image to be identified includes multiple image areas, and an image object in an image area represents a facility to be inspected.

[0010] In step S100, the drone can fly over complex terrain and enter areas that are difficult to reach manually, thereby achieving comprehensive coverage of the target three-defense area. The so-called three-defense area refers to an industrial area with special requirements such as fire prevention, waterproofing, and explosion prevention, such as a chemical plant, oil depot, and nuclear power plant. These areas often have harsh environments and potential safety hazards, so regular inspections are particularly important. When the drone performs an inspection task, the camera it carries will capture images of the target three-defense area in real time. These images not only record the environmental conditions of the area, but also contain a large amount of key information related to the inspection. The computer system receives the image data transmitted back by the drone through wireless communication technology, such as 4G, 5G or satellite communication. After decoding and preprocessing, these image data become images to be identified, providing basic data support for subsequent steps. The image to be identified is the core object processed by the computer system, which contains multiple image areas. Each image area can be regarded as an independent part of the image, corresponding to a specific location or facility in the target three-defense area. For example, in a chemical plant inspection image, multiple image areas such as a storage tank area, a pipeline area, and a control room are included. The image objects in each image area represent specific facilities in the area, such as storage tanks, pipelines, valves, etc.

[0011] Consider a chemical plant inspection scenario where a drone flies over the plant and captures an image of multiple tanks. In this image, each tank corresponds to an image region, and the tank itself and its ancillary facilities constitute the image objects in the image region. The drone returns the image to the computer system, which accurately identifies these image objects and determines whether their status is normal. After acquiring the image to be identified, the computer system performs a series of preprocessing operations on it. These operations include image denoising, contrast enhancement, color balance adjustment, etc., which are designed to improve image quality and reduce the difficulty of subsequent processing. After the preprocessing is completed, the computer system divides the image into multiple regions, each of which corresponds to a facility to be inspected.

[0012] When dividing image regions, the computer system can use a variety of methods, such as region segmentation algorithms based on edge detection, target detection algorithms based on machine learning, etc. Taking the target detection algorithm based on machine learning as an example, the computer system can pre-train a convolutional neural network (CNN) model to detect specific targets in the image (such as storage tanks, pipes, etc.). During the detection process, the CNN model extracts the feature vector of the image and matches it with the pre-learned feature library to determine whether the target object exists in the image and its location.

[0013] The feature vector is used to describe the content of the image. It contains a variety of information such as the texture, color, shape, etc. of the image. In the object detection task, the feature vector can help the computer system distinguish different image objects. For example, although storage tanks and pipelines are both industrial facilities, their shapes, sizes, colors and other features are significantly different. By extracting these features and constructing feature vectors, the computer system can accurately identify storage tanks and pipelines in the image.

[0014] In addition to target detection, the state of the image objects in the image area is also judged. This involves, for example, further analysis of the image objects, such as measuring the liquid level of a tank, detecting whether there is a leak in a pipeline, etc. These tasks can be completed by specific image processing algorithms or machine learning models. For example, for the liquid level measurement task, the computer system can use a water level line extraction algorithm based on edge detection; for the leak detection task, an anomaly detection algorithm based on image differences can be used.

[0015] Step S200: Generate an inspection object map based on the image objects in each image area in the image to be identified and the causal relationship between different image objects; the inspection object map includes multiple object nodes and relationship connection lines, an object node is used to represent an image object in an image area, and a relationship connection line is used to represent the causal relationship between the image objects represented by the connected object nodes.

[0016] In step S200, the computer system analyzes each image area in the image to be identified. These image areas are independent parts of the image, corresponding to specific facilities or locations in the target three-defense area. For example, in a chemical plant inspection image, it may contain multiple image areas such as a storage tank area, a pipeline area, and a control room. Each image area contains at least one image object, which represents specific facilities in the area, such as a storage tank, a pipeline, a valve, etc.

[0017] In order to accurately identify these image objects, the computer system can use a machine learning-based target detection algorithm. Taking the convolutional neural network (CNN) as an example, the computer system can pre-train a CNN model to detect specific targets in the image. During the detection process, the CNN model extracts the feature vectors of the image. These feature vectors contain a variety of information such as the texture, color, shape, etc. of the image, which can help the model distinguish different image objects. By matching the extracted feature vectors with the pre-learned feature library, the CNN model can accurately identify target objects such as storage tanks and pipes in the image.

[0018] In addition to identifying image objects, computer systems also need to understand the causal relationships between these objects. These relationships, for example, describe the interactions and impacts between different facilities based on industry expertise. For example, in a chemical plant, changes in the liquid level of a storage tank can affect the pressure of a pipeline, and abnormal pressure in the pipeline may trigger the activation of a safety valve. These causal relationships are crucial for inspection tasks because they can help computer systems predict potential safety hazards and take timely countermeasures.

[0019] In order to represent these complex causal relationships, the computer system uses the tool of inspection object graph. Inspection object graph is a graphical data structure that contains multiple object nodes and relationship connection lines. Each object node represents an image object in an image area, and each relationship connection line represents the causal relationship between the connected object nodes. For example, in the inspection graph of a chemical plant, a tank object node may be connected to a pipeline object node through a relationship connection line, indicating that the change of the tank's liquid level will affect the pressure of the pipeline.

[0020] When generating the inspection object map, the computer system follows certain rules and methods. First, the computer system creates corresponding object nodes in the map based on the results of the target detection algorithm. Each object node contains some attribute information, such as object name, type, location, etc., for subsequent processing and analysis. Secondly, the computer system determines the causal relationship between different object nodes based on domain knowledge and expert experience, and draws corresponding relationship connection lines in the map. These connection lines may be directional, indicating the transmission direction of the causal relationship.

[0021] Taking the inspection of a chemical plant as an example, suppose the image to be identified contains two tanks (tank A and tank B) and a pipe connecting them (pipeline C). Based on domain knowledge, we know that the change in the liquid level of tank A will affect the pressure of pipe C, and the abnormal pressure of pipe C may trigger the activation of the safety valve of tank B. In order to represent these causal relationships, the computer system will create three object nodes in the graph: tank A node, pipe C node, and tank B node. Then, it will draw two directed connection lines: one from the tank A node to the pipe C node, indicating that the change in the liquid level of tank A will affect the pressure of pipe C; the other from the pipe C node to the tank B node, indicating that the abnormal pressure of pipe C may trigger the activation of the safety valve of tank B.

[0022] In the process of generating the inspection object graph, the computer system can also use some complex computing methods and models to optimize the structure and accuracy of the graph. For example, it can use graph neural networks (GNNs) to learn the representation of object nodes and relationship connection lines in the graph, and capture the potential structure and patterns in the graph through node embedding and edge embedding. In addition, the computer system can also use causal reasoning models to analyze the causal relationship in the graph and predict potential safety hazards.

[0023] The generated inspection object map is not static. It can be continuously updated and improved as the inspection task proceeds. For example, during the inspection process, the computer system can discover new image objects or causal relationships, and then the map needs to be modified and expanded accordingly. At the same time, as domain knowledge and expert experience accumulate, the computer system can also refine and optimize the relationship connection lines in the map to improve the accuracy and efficiency of the inspection task.

[0024] Step S300: when performing image recognition on the image to be recognized according to the inspection object map, a control object node that can be simultaneously recognized is obtained by screening from the inspection object map.

[0025] In step S300, the computer system makes full use of the image objects presented by the inspection object map and the causal relationship between them, and realizes parallel processing of the image recognition task through intelligent analysis and decision-making.

[0026] In step S300, the computer system first performs image recognition on the image to be identified based on the inspection object map. The inspection object map is a graphical data structure containing multiple object nodes and relationship connection lines. Each object node represents an image object in an image area, and the relationship connection lines represent the causal relationship between different object nodes. By traversing the inspection object map, the computer system can clearly understand the location, attributes and interactions of each image object in the image to be identified.

[0027] In the image recognition process, the computer system processes a large amount of image data, which contains, for example, rich texture, color, shape and other information. In order to efficiently complete the image recognition task, the computer system uses, for example, image recognition algorithms based on machine learning, such as convolutional neural networks (CNNs) and recurrent neural networks (RNNs). These algorithms extract the feature vectors of the image and match them with the pre-learned feature library to achieve accurate recognition of image objects. However, in complex inspection scenes, there are often complex causal relationships between image objects, which means that the recognition results of certain image objects can be affected by the status of other image objects. Therefore, when performing image recognition, the computer system fully considers these causal relationships to ensure the accuracy and reliability of the recognition results.

[0028] In order to optimize the efficiency of image recognition, the computer system adopts a parallel processing method, that is, multiple image objects are recognized simultaneously. However, not all image objects can be recognized simultaneously, because the causal relationship between certain objects can lead to mutual dependence of recognition results. Therefore, in step S300, the computer system selects control object nodes that can be recognized simultaneously from the inspection object map.

[0029] The process of screening control object nodes involves multiple considerations. First, the computer system determines the importance weight of each object node. This weight can be calculated based on multiple factors such as the position of the object node in the inspection object map, the number of connected nodes, the complexity of the causal relationship, etc. For example, an object node connected to multiple critical facilities may have a higher importance weight because its state changes can have a significant impact on the entire inspection area. Secondly, the computer system considers the feasibility of parallel identification between object nodes. This involves, for example, the analysis of the causal relationship between object nodes. If there is no direct causal relationship between two object nodes, or the causal relationship between them can be decoupled by other means (such as pre-set rules or models), then the two object nodes can be regarded as control object nodes and identified simultaneously.

[0030] When screening the control object nodes, the computer system can adopt a variety of strategies and methods. For example, it can first sort the object nodes according to their importance weights, and then check the causal relationship between each node and other nodes in turn to determine which nodes can be identified simultaneously. In addition, the computer system can also use complex models such as graph neural networks (GNNs) to learn the structural features and causal relationship patterns in the inspection object map, so as to more accurately screen the control object nodes.

[0031] Taking the inspection of a chemical plant as an example, suppose that the inspection object map contains multiple object nodes such as tanks, pipelines, and valves. When the computer system screens the reference object nodes, it can find that there is a direct causal relationship between some tanks and pipelines (such as the change in the liquid level of the tank will affect the pressure of the pipeline), while there is no direct causal relationship between other tanks. In this case, the computer system can treat the tanks without a direct causal relationship as reference object nodes and identify them simultaneously. At the same time, it can also consider certain pipelines and valves as reference object nodes, as long as they have no direct causal dependency with the currently identified tanks.

[0032] Step S400: Perform simultaneous recognition processing on the image objects represented by the control object nodes to obtain abnormality recognition evaluation information of the image to be recognized.

[0033] In step S400, the computer system determines the image objects that need to be identified simultaneously based on the reference object nodes screened out in step S300. These reference object nodes have no direct causal dependency in the inspection object map, or the causal relationship between them can be decoupled by other means (such as pre-set rules or models), so they can be treated as independent recognition tasks for processing.

[0034] In order to simultaneously identify these image objects, the computer system uses, for example, image recognition algorithms based on machine learning. These algorithms are pre-trained and optimized to accurately extract key features from images, such as shape, texture, color, etc., and match these features with predefined target categories. In the scene of chemical plant inspection, these target categories may include industrial facilities such as storage tanks, pipelines, valves, and their possible abnormal conditions, such as leakage, blockage, damage, etc.

[0035] When performing image recognition, the computer system inputs the image object represented by each reference object node into the image recognition algorithm and simultaneously starts multiple computing threads or processes to handle these recognition tasks. Since there is no direct dependency between these tasks, they can be executed in parallel, greatly improving processing efficiency.

[0036] During the recognition process, the computer system will evaluate the status of each image object based on the output of the image recognition algorithm. These evaluation results may include basic information such as the location, size, shape, color, and judgment of whether there is an abnormal state. For example, when identifying a storage tank, the computer system can detect whether the liquid level of the storage tank is too high or too low, whether there are signs of leakage, etc. In order to obtain abnormal recognition evaluation information for the image to be identified, the computer system conducts a comprehensive analysis of the recognition results of all control object nodes. This process may involve reasoning and judgment of the causal relationship between multiple image objects. For example, if the computer system detects that the pressure of a pipeline has increased abnormally, and at the same time finds that the liquid level of the storage tank connected to the pipeline has also dropped abnormally, then it can infer that there may be a leakage problem in the pipeline.

[0037] In the comprehensive analysis process, the computer system can use some complex calculation methods and models to improve the accuracy of the evaluation. For example, it can use an inference algorithm based on a probabilistic graphical model to simulate the causal relationship between image objects and calculate the probability of each object being in an abnormal state. In addition, the computer system can also use technologies such as deep learning to build anomaly detection models, automatically extract the characteristics and patterns of abnormal states through learning and analyzing a large amount of historical data, and use them to perform real-time detection and evaluation of new image data.

[0038] Taking the inspection of a chemical plant as an example, suppose the computer system selects two control object nodes: one represents tank A, and the other represents pipe B connected to tank A. During the simultaneous recognition process, the computer system can detect that the liquid level of tank A has dropped abnormally, while the pressure of pipe B has increased abnormally. By comprehensively analyzing the recognition results of the two image objects and the causal relationship between them, the computer system can conclude that there is a leak in pipe B, and feed back this abnormal recognition assessment information to the inspector or drone control system so that further countermeasures can be taken.

[0039] Step S500: Generate a route confirmation instruction based on the abnormality identification and evaluation information, and send the route confirmation instruction to the drone, so that the drone responds to the route confirmation instruction to perform inspection.

[0040] In step S500, the computer system analyzes and evaluates the overall condition of the inspection area based on the abnormality recognition and evaluation information obtained in step S400. The abnormality recognition and evaluation information includes the abnormal state detection results of each image area (i.e., the inspection object) in the image to be identified, and these results may indicate potential safety hazards or equipment failures. For example, in a chemical plant inspection scenario, the abnormality recognition and evaluation information may include abnormal liquid level of a storage tank, abnormal pressure of a pipeline, and incorrect opening and closing status of a valve.

[0041] To generate a route confirmation instruction, the computer system considers multiple factors, including but not limited to the severity of the anomaly identification assessment information, the location of the anomaly, the current location and status of the drone, the objectives and requirements of the inspection mission, etc. These factors together determine what actions the drone should take, as well as the priority and order of actions.

[0042] In the decision-making process, the computer system can use some complex calculation methods and models to assist in judgment. For example, it can use a rule-based decision-making system to judge the severity and urgency of abnormal conditions according to preset rules and thresholds; or use a prediction model based on machine learning to predict the development trend and potential impact of abnormal conditions through analysis and learning of historical data. The application of these calculation methods and models helps the computer system make more accurate and reasonable decisions. Once the decision is made, a route confirmation instruction will be generated based on the decision results. The route confirmation instruction is a set of instructions containing information such as the flight path of the drone, the location of the inspection point, and the inspection action requirements. These instructions are designed to guide the drone to conduct inspections along a predetermined route and sequence to ensure that the inspection area can be fully and efficiently covered, and the abnormal state can be further inspected and processed.

[0043] When generating route confirmation instructions, the computer system fully considers the flight capabilities and limitations of the drone, such as flight speed, flight altitude, endurance, obstacle avoidance, etc. At the same time, it is also necessary to consider the urgency and importance of the inspection task, as well as possible risks and uncertainties. For example, when serious safety hazards are detected, the computer system can give priority to arranging drones to go to the area for emergency inspections; in the case of limited resources or time constraints, it may be necessary to optimize inspection routes and actions to improve inspection efficiency. It can be understood that the specific implementation process of the inspection route formulation and other processes is not limited, and can be formulated with reference to existing technologies, and is not limited here.

[0044] After generating the route confirmation instruction, the computer system will send the instruction to the drone through the wireless communication network. After receiving the instruction, the drone will parse the instruction content and adjust its own flight status and inspection strategy to respond to the route confirmation instruction for inspection. During the inspection process, the drone can dynamically adjust and optimize the route confirmation instruction based on the real-time image data and sensor information to adapt to the actual conditions and changes in the inspection area.

[0045] Taking the inspection of a chemical plant as an example, suppose the computer system detects an abnormal drop in the liquid level of tank A through abnormal identification and evaluation information. When generating a route confirmation instruction, the computer system can give priority to arranging the drone to inspect the area where tank A is located, and require the drone to monitor and record the liquid level of tank A in real time. At the same time, based on the causal relationship and potential impact between tank A and other equipment (such as pipelines, valves, etc.), the computer system can also arrange for the drone to inspect adjacent areas to eliminate other potential safety hazards. During the inspection process, if the drone finds a new abnormal situation or needs to adjust the route, it will send feedback information to the computer system through the wireless communication network and wait for new route confirmation instructions.

[0046] Through the decision-making and instruction generation process of step S500, the computer system can ensure that the drone performs the inspection task according to the optimal strategy, thereby improving the inspection efficiency and accuracy. At the same time, through real-time feedback and dynamic adjustment mechanisms, the computer system can also flexibly adjust and optimize the inspection task according to the actual situation and changes in the inspection area to adapt to the complex and changing inspection environment.

[0047] As an implementation method, the generated inspection object map is an optimized inspection object map, and the relationship connection lines in the optimized inspection object map are connection lines with directions. Based on this, step S200 generates an inspection object map based on the image objects in each image area of ​​the image to be identified and the causal relationship between different image objects, including: Step S210: Generate a basic inspection object map based on the image objects in each image region of the image to be identified and the causal relationship between different image objects; the basic inspection object map includes object nodes for representing image objects, and the connection lines between the object nodes are from the correspondingly represented effect image object to the cause image object, wherein the state of the effect image object is affected by the change in the state of the cause image object; Step S220: Based on the object nodes and directions of connection lines contained in the basic inspection object graph, the basic inspection object graph is optimized to obtain an optimized inspection object graph.

[0048] Step S210 is the preliminary stage of generating an inspection object map, the goal of which is to construct a basic inspection object map based on the image objects in each image region of the image to be identified and the causal relationship between them. In this map, each object node represents an image object, and the connecting lines between the object nodes reflect the causal relationship between the image objects.

[0049] When constructing the basic inspection object map, the computer system first uses a target detection algorithm (such as a convolutional neural network (CNN)) to identify each image object in the image to be identified. These image objects may include industrial facilities such as storage tanks, pipes, and valves. For each identified image object, the computer system creates a corresponding object node in the map and assigns a unique identifier and related attribute information (such as object name, type, location, etc.) to the node.

[0050] Next, the computer system determines the causal relationship between these image objects. This is achieved, for example, by analyzing the physical connection, functional dependency or logical association between the image objects based on domain knowledge and expert experience. For example, in a chemical plant inspection, the computer system can identify the connection between the storage tank and the pipeline, and the impact of the pipeline pressure change on the tank liquid level.

[0051] Once the cause-effect relationship between the image objects is determined, the computer system will draw the corresponding connection lines in the basic inspection object map. These connection lines are directional, and they point from the cause image object to the effect image object, indicating that the state change of the cause image object will affect the state of the effect image object. For example, if the change of the liquid level in the tank affects the pressure in the pipeline, then a directed connection line will be drawn in the map from the tank node to the pipeline node.

[0052] Through step S210, the computer system constructs a basic inspection object map including object nodes and directed connection lines. This map preliminarily reflects the causal relationship between image objects in the image to be identified, and provides a basis for subsequent optimization steps.

[0053] Step S220 is an optimization step based on the basic inspection object map, and its goal is to improve the accuracy and practicality of the map. The optimization process is mainly based on the directionality of object nodes and connecting lines in the map, and is achieved by deleting invalid nodes and simplifying connecting lines.

[0054] During the optimization process, the computer system first searches the database for abnormal identification and evaluation information that has been saved. This information may come from previous inspection tasks or historical data, which records the abnormal performance of certain image objects in specific states. By searching this information, the computer system can identify invalid nodes that may exist in the basic inspection object map, that is, those nodes that are unlikely to be abnormal in actual inspections or have no direct causal relationship with other nodes.

[0055] For example, in a chemical plant inspection, if a valve has never been abnormal in previous inspections and has no direct causal relationship with other key facilities (such as storage tanks and pipelines), the node corresponding to the valve can be considered an invalid node. During the optimization process, the computer system will delete these invalid nodes from the graph to reduce unnecessary complexity and computation.

[0056] In addition to deleting invalid nodes, the computer system also simplifies the connections in the basic inspection object graph. This involves, for example, merging connections with the same causal relationship and deleting indirect causal relationships that are unlikely to occur in actual inspections. For example, if the causal relationship between two nodes can be reflected through other more direct paths, then the indirect connection between them can be deleted to simplify the graph structure.

[0057] During the optimization process, the computer system also uses complex models such as graph neural networks (GNNs) to learn structural features and causal patterns in the graph. These models can extract useful information by analyzing the nodes and connections in the graph, and help the computer system more accurately identify invalid nodes and simplify connections. For example, the GNN model can represent the similarities and differences between nodes by learning node embedding and edge embedding in the graph, thereby assisting the computer system in making optimization decisions. Through the optimization process of step S220, a more accurate and practical inspection object graph is obtained. This graph not only reflects the direct causal relationship between image objects in the image to be identified, but also reduces complexity and computational complexity by deleting invalid nodes and simplifying connections. This enables subsequent tasks such as image recognition, anomaly detection, and route planning to be performed more efficiently and accurately.

[0058] Taking the inspection of a chemical plant as an example, assume that the image to be identified contains multiple image objects such as storage tanks, pipes and valves. Through step S210, the computer system constructs a basic inspection object map containing these object nodes and directed connecting lines. Then, in step S220, the computer system searches the abnormality identification evaluation information in the database, identifies a valve node that rarely has abnormalities and has no direct causal relationship with other nodes, and deletes it from the map. At the same time, the computer system also merges connecting lines with the same causal relationship and deletes those indirect causal relationships that are unlikely to occur, thereby simplifying the map structure. The optimized inspection object map finally obtained is clearer and more accurate, providing strong support for subsequent inspection tasks.

[0059] As an implementation mode, step S220, based on the object nodes and the directions of the connection lines included in the basic inspection object map, optimizes the basic inspection object map to obtain an optimized inspection object map, including: Step S221: searching the database for abnormal recognition evaluation information of an image object represented by any object node in the basic inspection object map; Step S222: if the abnormality recognition evaluation information of the image object represented by any object node is searched, based on the position of any object node in the basic inspection object map and the direction of the connection line contained in the basic inspection object map, determine the remaining object nodes having a direction relationship with any object node; Step S223: clear invalid nodes from any object node and other object nodes to obtain an optimized inspection object graph.

[0060] In step S221, the goal is to search the database for abnormal recognition evaluation information of the image object represented by any object node in the basic inspection object map. This information may come from previous inspection tasks, historical data or domain knowledge, which records the abnormal performance of image objects in specific states. By searching this information, the computer system can understand which image objects are more likely to be abnormal and what factors are related to these abnormalities.

[0061] During the search process, the computer system first obtains the identifier or related attribute information of any object node in the basic inspection object map. Then, it uses this information to perform a query operation in the database to find the abnormality recognition evaluation information of the image object corresponding to the object node. This evaluation information may include the type of abnormality, the frequency of occurrence, the scope of impact, and the association with other image objects.

[0062] For example, in a chemical plant inspection scenario, assuming that the basic inspection object map contains an object node representing a storage tank, in step S221, the computer system searches the database for abnormal identification and evaluation information related to the storage tank, such as liquid level abnormalities, leakage events, etc. If such information exists in the database, the computer system will record it for subsequent analysis.

[0063] Step S222 is performed on the basis of step S221, and its goal is to determine the remaining object nodes that have a direction relationship with any object node based on the position of the object node in the basic inspection object map and the direction of the connection line. These nodes, for example, have a direct causal relationship with the current node, that is, the state change of the current node can affect the state of these nodes, or the state change of these nodes can affect the state of the current node.

[0064] When determining the remaining object nodes with directional relationships, the computer system first checks the connection lines of the current object node in the basic inspection object map. These connection lines are directional, and they point from the cause image object to the effect image object, indicating the transmission direction of the causal relationship. The computer system then searches along the direction of these connection lines to find other nodes that are directly connected to the current node. During the search process, the computer system also considers the weight and type of the connection line. The weight can represent the strength and importance of the causal relationship, while the type can represent the nature of the causal relationship (such as positive influence, negative influence, etc.). By comprehensively considering these factors, the computer system can more accurately determine which nodes have a significant directional relationship with the current node.

[0065] For example, in a chemical plant inspection scenario, assume that the current object node represents a storage tank, and the storage tank is connected to a pipeline. In step S222, the computer system searches along the connection line from the storage tank to the pipeline to determine the remaining object nodes that have a direction relationship with the storage tank. If the state change of the pipeline (such as abnormal pressure) affects the liquid level of the storage tank, then the pipeline node will be regarded as a node that has a direction relationship with the storage tank.

[0066] Step S223 is the last step of the optimization process, and its goal is to clean up invalid nodes for any object node determined in step S222 and the remaining object nodes to obtain an optimized inspection object graph. Invalid nodes refer to nodes that are unlikely to be abnormal in actual inspections, have no direct causal relationship with other nodes, or contribute little to the inspection task. By deleting these nodes, the graph structure can be simplified and the inspection efficiency can be improved.

[0067] When cleaning invalid nodes, the computer system will take multiple factors into consideration. First, it will check the location and connection of the node in the basic inspection object map. If the node has few connections with other nodes or no direct causal relationship, then it may be regarded as an invalid node. Secondly, the computer will refer to the abnormality recognition evaluation information searched in step S221. If the image object corresponding to the node rarely or never has abnormalities in previous inspection tasks, then it may also be regarded as an invalid node.

[0068] In addition, the computer system can also use machine learning models to assist in determining the effectiveness of nodes. For example, it can train a classification model to predict the importance of a node in an inspection task or the probability of an abnormality. By inputting the node's attribute information (such as type, location, connection status, etc.) and related historical data, the model can output a score or label representing the effectiveness of the node. Based on this score or label, the computer system can decide whether to delete the node.

[0069] After determining the invalid nodes, the computer system will delete these nodes and the connecting lines between them from the basic inspection object map. In this way, an optimized inspection object map is obtained, which is more concise, accurate and practical.

[0070] Taking the chemical plant inspection as an example, assume that in step S222 it is determined that the tank node and the pipeline node connected to it have a directional relationship. However, in step S223, the computer system finds that the pipeline node rarely has abnormalities in previous inspection tasks, and it has few connections with other key nodes. Based on this information, the computer system can regard the pipeline node as an invalid node and delete it from the map. In this way, the optimized inspection object map focuses more on key nodes such as storage tanks and their direct causal relationships, thereby improving the pertinence and efficiency of the inspection.

[0071] As an implementation method, the database stores object features of image objects for which abnormality recognition evaluation information has been determined. Based on this, step S221 searches the database for abnormality recognition evaluation information of image objects represented by any object node in the basic inspection object map, including: Step S2211: obtaining the reference object feature of the image object represented by any object node in the basic inspection object map; Step S2212: Search the database for object features that meet feature similarity conditions with the control object features, and when object features that meet feature similarity conditions with the control object features are matched from the database, determine that the abnormality recognition evaluation information of the image object represented by any object node is matched; otherwise, determine that the abnormality recognition evaluation information of the image object represented by any object node is not matched.

[0072] Step S2211 is the first step of step S221, and its goal is to obtain the reference object features of the image object represented by any object node in the basic inspection object map. These reference object features are a set of attributes or parameters closely related to the image object, which can uniquely identify the image object and are used to search for matching abnormality recognition evaluation information in the database.

[0073] When obtaining the reference object features, the computer system first locates any object node in the basic inspection object map. This node represents the image object to be searched for abnormal identification assessment information. The computer then extracts relevant attribute information from the node, which constitutes the reference object features. These features may include the type, shape, size, color, texture, position, etc. of the image object, depending on the characteristics of the image object and the requirements of the inspection task.

[0074] For example, in a chemical plant inspection scenario, assume that the basic inspection object map contains an object node representing a storage tank. In step S2211, the computer system can extract the reference object features of the storage tank, such as the type of storage tank (such as vertical storage tank, horizontal storage tank), diameter, height, color, material, etc. These features together constitute the unique identification of the storage tank, so that the computer system can accurately search for the abnormal identification and evaluation information related to it in the database.

[0075] Step S2212 is the second step of step S221, and its goal is to search the database for object features that meet the feature similarity condition with the reference object features, and determine whether the abnormality recognition evaluation information of the image object represented by any object node is matched. The key to this step is how to define and calculate the feature similarity condition, and how to efficiently perform the search operation.

[0076] During the search process, the computer system first constructs a search query in the database using the extracted features of the control object. This query specifies the target of the search (i.e., the anomaly identification assessment information) and the search conditions (i.e., the object features that are similar to the control object features). In order to measure the similarity between features, the computer system can use various distance metrics, such as Euclidean distance, Manhattan distance, cosine similarity, etc. These methods can quantify the degree of difference between different feature vectors, thereby helping the computer system determine which object features are similar to the control object features.

[0077] When constructing a search query, the computer system also needs to consider the structure and indexing mechanism of the database. In order to improve the search efficiency, the database will, for example, create indexes for certain key fields or features. These indexes can speed up the query process, allowing the computer system to find object features similar to the reference object features more quickly. Therefore, in step S2212, the computer will use these indexes to optimize the search operation and reduce unnecessary calculations.

[0078] Once the search query is constructed, the computer system will perform a search operation in the database. This operation will traverse the records in the database and compare the similarity between the object features in each record and the reference object features. If object features that meet the feature similarity conditions with the reference object features are found, the computer system will further check the anomaly recognition evaluation information corresponding to these object features. If these anomaly recognition evaluation information are related to the image object currently being searched, the computer system will determine that the anomaly recognition evaluation information is matched and return it as the search result.

[0079] If no object feature that satisfies the feature similarity condition with the control object feature is found after the search operation is completed, or the abnormality recognition evaluation information corresponding to the found object feature is not related to the image object currently being searched, then the computer system will determine that no abnormality recognition evaluation information is matched. In this case, the computer system can continue to search for abnormality recognition evaluation information of image objects represented by other object nodes, or perform other optimization steps to improve the inspection object map.

[0080] Taking the inspection of a chemical plant as an example, assume that in step S2211, the computer system extracts the reference object features of the storage tank, such as type, diameter, height, etc. In step S2212, the computer will use these features to construct a search query in the database and search for similar object features. If there are records of other storage tanks with similar features to the storage tank in the database, and these records contain abnormality identification and evaluation information (such as liquid level abnormalities, leakage events, etc.), then the computer system will return this information as search results. If no similar object features or related abnormality identification and evaluation information are found in the database, then the computer system will continue to search or perform other operations.

[0081] As an implementation mode, step S300, screening out control object nodes capable of simultaneous identification from the inspection object map, includes: Step S310: determining the importance weight of each object node based on the inspection object map; the importance weight represents the processing order of the image objects represented by the corresponding object nodes during image recognition, and the image objects represented by the object nodes corresponding to the same importance weight have the same processing order during image recognition; Step S320: The object nodes corresponding to the same importance weight are used as comparison object nodes that can be simultaneously identified.

[0082] Step S310 is the basis of step S300, and its core goal is to assign an importance weight to each object node according to the characteristics of each object node in the inspection object map and their mutual relationship. This weight represents the processing order of the corresponding image object during image recognition, that is, the object node with a higher weight will be processed first. At the same time, the image objects represented by the object nodes with the same importance weight can be regarded as control object nodes during image recognition and be recognized and processed simultaneously.

[0083] When determining the importance weight of an object node, the computer system considers multiple factors. First, the position of the object node in the inspection object map is an important indicator. For example, object nodes located at the center of the map or closely connected to multiple other nodes may have higher importance because they may represent key facilities or core links in the inspection area. Secondly, the type, size, color and other characteristics of the image object represented by the object node may also affect the weight assignment. For example, in a chemical plant inspection, a large storage tank or a high-pressure pipeline may have a higher importance weight than other small equipment. In addition, the causal relationship between object nodes is also an important factor in determining the weight. If the state change of an object node directly affects the state of multiple other object nodes, then the object node may have a higher importance weight. On the contrary, if the causal relationship between an object node and other nodes is weak or there is no direct connection, then its importance weight may be relatively low.

[0084] In order to more accurately evaluate the importance weights of object nodes, computer systems can use a variety of methods and models. For example, graph neural networks (GNNs) can be used to learn the structural features and node relationships in the inspection object graph, thereby automatically assigning weights to each node. The GNN model can update the representation of nodes by iteratively aggregating the information of neighboring nodes, and ultimately obtain the importance score of each node. Another method is to manually set weight rules based on domain knowledge and expert experience. Although this method requires human intervention, it can ensure the rationality and accuracy of the weights.

[0085] When determining the weights, the computer system can also consider the specific requirements and goals of the inspection task. For example, if the inspection task focuses on detecting specific types of anomalies (such as leaks, blockages, etc.), then object nodes that are closely related to these anomalies can be given higher weights. Similarly, if the inspection task is time-sensitive, the computer system can prioritize object nodes that can complete identification quickly to improve overall inspection efficiency.

[0086] Step S320 is the core of step S300, and its goal is to screen out the control object nodes that can be simultaneously identified from the inspection object map according to the importance weights determined in step S310. The image objects represented by these nodes can be processed in parallel during image recognition, thereby significantly improving the recognition efficiency. When screening the control object nodes, the computer system will first traverse all the object nodes in the inspection object map and classify them according to their importance weights. Object nodes with the same weights will be grouped together, and these nodes are potential control object nodes. The computer will then check the causal relationship and mutual dependence between these nodes to ensure that there is no direct conflict or dependency between them, so that simultaneous identification can be performed safely.

[0087] For example, in a chemical plant inspection, suppose there are two tanks (tank A and tank B) with the same importance weight and no direct causal relationship between them (i.e., the state change of tank A will not affect the state of tank B). In this case, the object nodes corresponding to tanks A and B can be regarded as control object nodes, and their image objects can be recognized and processed at the same time.

[0088] However, in actual applications, not all object nodes with the same weight can be used as control object nodes. For example, if two object nodes have the same weight but there is an indirect causal relationship between them (such as being connected through other nodes), then the impact of this relationship on the recognition results may need to be considered when performing simultaneous recognition. In addition, if the inspection task has specific requirements for the recognition order (such as identifying upstream devices before identifying downstream devices), then even if two object nodes have the same weight, they may not be used as control object nodes for simultaneous recognition.

[0089] Therefore, when screening the control object nodes, the computer system comprehensively considers multiple factors, including the importance weight, causal relationship, interdependence and specific requirements of the inspection task of the object nodes. Through comprehensive analysis and judgment, the computer system can accurately screen out the control object nodes that can be recognized simultaneously, providing strong support for the subsequent image recognition process.

[0090] Taking the inspection of a chemical plant as an example, assume that the inspection object map contains multiple object nodes such as tanks, pipelines and valves. Through the weight determination process of step S310, the computer system assigns an importance weight to each node. Then, in step S320, the computer will screen out the control object nodes that can be identified simultaneously based on these weights. For example, if the weights of tank A and tank B are the same and there is no direct causal relationship between them, then their image objects can be identified and processed at the same time. Similarly, if a section of pipeline is connected to multiple valves and the importance weights of these valves are also the same, then the image objects of these valves can also be used as control object nodes for simultaneous identification. In this way, the computer system can significantly improve the efficiency of image recognition during the inspection process and provide strong support for subsequent tasks such as anomaly detection and route planning.

[0091] As an implementation mode, step S310, determining the importance weight of each object node based on the inspection object map, includes: Step S311: obtaining the hierarchical structure of any object node in the inspection object graph; Step S312: Determine the importance weight of any object node based on the hierarchical structure of any object node.

[0092] Step S311 is the basis of step S310, and its goal is to obtain the hierarchical structure of any object node in the inspection object map. The hierarchical structure refers to the hierarchical relationship and position of the object node in the map, which reflects the relative importance and relationship between the object node and other nodes. By obtaining the hierarchical structure of the object node, the computer system can better understand the role and status of the object node in the inspection task, and provide a basis for the subsequent determination of the importance weight.

[0093] When obtaining the hierarchical structure of object nodes, the computer system will first traverse the inspection object graph and locate the target object node. Then, the computer will analyze the connection relationship between the node and other nodes in the graph, including direct connection and indirect connection. Direct connection means that object nodes are directly connected through relationship connection lines, while indirect connection means that object nodes are indirectly connected through other nodes. By analyzing these connection relationships, the computer system can construct a local or global network structure of object nodes.

[0094] On the basis of constructing the network structure, the computer system can use a variety of methods to classify the object nodes. One feasible method is to use the centrality measurement indicators in graph theory, such as degree centrality, closeness centrality, betweenness centrality, etc. Degree centrality refers to the number of nodes directly connected to the object node, which reflects the local importance of the object node in the graph. Closeness centrality refers to the sum of the reciprocals of the shortest path lengths from the object node to all other nodes in the graph, which reflects the global accessibility of the object node in the graph. Betweenness centrality refers to the number of times the object node appears on all the shortest paths in the graph, which reflects the bridging role of the object node in the graph. By calculating these centrality measurement indicators, the computer system can sort and classify the object nodes to obtain their hierarchical structure. In addition to graph theory methods, the computer system can also use machine learning models to assist in determining the hierarchical structure of object nodes. For example, clustering algorithms (such as K-means, DBSCAN, etc.) can be used to divide object nodes into different categories or levels. Clustering algorithms can group object nodes according to the similarity or distance between them, thereby revealing the potential structure and hierarchical relationship of object nodes in the graph. In addition, deep learning models such as graph neural networks (GNNs) can be used to learn complex relationships between object nodes and automatically extract hierarchical structure features.

[0095] Step S312 is the core of step S310, and its goal is to determine the importance weight of the object node according to its hierarchical structure. The importance weight represents the relative importance and processing priority of the object node in the inspection task, and is a key parameter for optimizing the subsequent image recognition process.

[0096] When determining the importance weight of an object node, the computer system will comprehensively consider multiple factors such as the hierarchical structure of the object node, its connection relationship with other nodes, its role and status in the inspection task, etc. Specifically, the following methods can be used: 1. Weight allocation based on hierarchical structure: Different weight values ​​can be assigned to object nodes according to their hierarchical results. For example, object nodes with higher hierarchical levels (i.e. nodes in more central or critical positions in the network structure) will be assigned higher weights. For example, linear or nonlinear functions can be used to map hierarchical results to weight values ​​to ensure consistency between hierarchical structure and weight values.

[0097] 2. Weight adjustment based on connection relationship: In addition to the hierarchical structure, the connection relationship between the object node and other nodes is also an important factor in determining the weight. If the object node is directly connected to multiple important nodes (such as nodes with higher hierarchies), then its weight can be increased accordingly. Conversely, if the object node is connected to fewer or less important nodes, then its weight can be reduced. By adjusting the connection weights between object nodes, the weight distribution results can be further refined.

[0098] 3. Weight optimization based on task requirements: When determining weights, the specific requirements and goals of the inspection task also need to be considered. For example, if the inspection task focuses on detecting specific types of anomalies (such as leaks, blockages, etc.), then object nodes that are closely related to these anomalies can be given higher weights. Similarly, if the inspection task is time-sensitive or resources are limited, the computer system can prioritize object nodes that can be quickly identified and have a greater impact on the overall inspection results.

[0099] 4. Weight prediction based on machine learning models: In order to more accurately determine the importance weight of object nodes, the computer system can also use machine learning models for prediction. For example, a regression model can be trained to predict the weight value of an object node. The model can input information such as the hierarchical structure, connection relationship, attribute characteristics, etc. of the object node and output the corresponding weight prediction result. By continuously optimizing the model parameters and structure, the accuracy and robustness of weight prediction can be improved.

[0100] Taking the inspection of a chemical plant as an example, assume that the inspection object map contains multiple object nodes such as storage tanks, pipelines and valves. In step S311, the computer system divides these object nodes into different levels by analyzing the connection relationship and centrality measurement indicators in the map. For example, a storage tank located at the center of the map and connected to multiple key pipelines may be classified as the highest level, while a valve located at the edge and with fewer connections may be classified as a lower level. Then, in step S312, the computer system assigns an importance weight to each object node based on these classification results and connection relationships. For example, the highest-level storage tank may be assigned the highest weight value, while a lower-level valve may be assigned a lower weight value. In this way, the computer system can more accurately determine the importance of the object node in the inspection task, providing strong support for the subsequent optimization of the image recognition process.

[0101] As an implementation mode, if the inspection object map has an object node that is consistent with the hierarchical structure of any object node and is different from the parent object node of any object node, the method further includes: Step S313: acquiring the number of first object nodes that correspond to the same upper-level object node as any object node, and the number of second object nodes that correspond to different upper-level object nodes as any object node and have the same hierarchical structure, wherein the first object node and the second object node are both object nodes corresponding to image objects that have not been identified; If the number of the first object nodes is greater than the number of the second object nodes, the importance weight of any object node is greater than the importance weight of an object node that corresponds to a different parent object node and has the same hierarchical structure as any object node; If the number of the first object nodes is less than the number of the second object nodes, the importance weight of any object node is less than the importance weight of an object node that corresponds to a different parent object node and has the same hierarchical structure as any object node; If the number of the first object nodes is equal to the number of the second object nodes, the importance weight of any object node is equal to the importance weight of an object node that corresponds to a different upper-level object node and has the same hierarchical structure as the object node.

[0102] The implementation of step S313 is based on a premise: in the inspection object map, there are at least two object nodes with the same hierarchical structure, but different parent object nodes. This situation is more common in complex inspection scenarios. For example, multiple tanks in a chemical plant may have similar physical structures and functions, but their positions in the inspection map (i.e., parent object nodes) may vary depending on the process flow or layout.

[0103] In order to refine the importance weight distribution of these object nodes, step S313 first requires the computer system to perform the following operations: Obtaining the number of first object nodes: The computer system counts the number of other object nodes that correspond to the same parent object node as any object node (referred to as a "target node"). These nodes have the same "parent node" as the target node in the inspection graph, that is, they are in a similar hierarchical structure and are affected or controlled by the same parent object.

[0104] Obtaining the number of second object nodes: At the same time, the computer system also needs to count the number of object nodes that have the same hierarchical structure as the target node but have different parent object nodes. These nodes may be physically or functionally similar to the target node, but due to their different positions in the inspection map (i.e., different "parent nodes"), they may be affected or controlled by different factors.

[0105] After obtaining the number of these two types of object nodes, the computer system will determine the importance weight of the target node according to the following rules: Quantity comparison and weight assignment: If the number of first object nodes (i.e., the number of nodes with the same parent object node as the target node) is greater than the number of second object nodes, the computer system will consider the target node to be more important in the inspection task and will assign it a higher importance weight. This is because the target node is more closely connected to its parent object node, and its state changes may have a greater impact on the overall inspection results.

[0106] On the contrary, if the number of the first object nodes is less than the number of the second object nodes, the computer system will consider the target node to be relatively less important and will assign it a lower importance weight. This is because although the target node is similar to other nodes in the hierarchical structure, it may not be a key node in the inspection task due to the difference of its parent object node.

[0107] If the number of nodes in the two categories is equal, the computer system will assume that the target node has no significant difference in importance from other similar nodes, and will therefore assign them the same importance weight.

[0108] The logic behind this rule is that the object nodes in the inspection task are not only affected by their own attributes and hierarchical structure, but also by their superior object nodes (i.e., "parent nodes"). The importance of a node is not only reflected in its own attributes and functions, but also in its relationship with other nodes (especially superior nodes) in the entire inspection network. By comparing the relationship between a node and its superior object node, the computer system can more comprehensively evaluate the importance of the node and optimize the weight distribution accordingly.

[0109] Taking the inspection of a chemical plant as an example, suppose that the inspection object map contains multiple tank nodes, where tank A and tank B have the same hierarchical structure (i.e., they have similar physical structures and functions), but the parent object node of tank A is reactor X, while the parent object node of tank B is reactor Y. In this case, the computer system will count the number of other tanks that have the same parent object node as tank A and tank B respectively. If the number of tanks that have the same parent object node (reactor X) as tank A is greater than the number of tanks that have the same parent object node (reactor Y) as tank B, then the computer system will consider tank A to be more important in the inspection task and assign it a higher importance weight. This is because tank A is more closely connected to reactor X, and its state changes may have a greater impact on the entire process related to reactor X.

[0110] By implementing step S313, the computer system can more accurately determine the importance weight of each object node in the inspection object map, thereby optimizing the subsequent image recognition process. This not only improves the efficiency of the inspection task, but also enhances the accuracy and reliability of the inspection results.

[0111] As an implementation manner, the abnormality recognition evaluation information of the image to be recognized includes abnormality recognition evaluation information of each image region in the image to be recognized. Based on this, the method further includes: Step S10: Based on the recognition status of image recognition for each image area in the image to be recognized, a process indicator is implanted into the object node corresponding to the corresponding image area in the inspection object map; the process indicator includes any one of the following process indicators: a recognition completed process indicator, a pending recognition process indicator, and a recognition in progress process indicator; Step S20: based on the process indicator implanted in each object node in the inspection object map, obtaining the object features of the image area in the object node implanted with the identification completion process indicator and the corresponding abnormality identification evaluation information; Step S30: associate the acquired object features with the corresponding abnormality recognition evaluation information and save them in a database.

[0112] Step S10 implants a process indicator for the object node corresponding to the corresponding image area in the inspection object map based on the progress of image recognition for each image area in the image to be recognized. The process indicator is a mark used to indicate the current recognition status of the image area represented by the object node. It has three possible values: recognition completed process indicator, recognition pending process indicator and recognition in progress indicator.

[0113] When the image recognition task of an image area has been completed and the result has been verified and confirmed, the computer system will implant this indicator on the corresponding object node. This means that the recognition work of the image area has been completed and its result can be used for subsequent abnormality assessment and inspection decisions.

[0114] If the image recognition task has not yet started for an image area, or the recognition task has been temporarily put on hold (for example, due to resource limitations or priority adjustments), the computer system will implant a pending recognition process indicator on the object node. This means that the image area is part of the inspection plan but has not yet been processed.

[0115] When an image recognition task is in progress but not yet completed, the computer will place an in-progress indicator on the object node. This means that the image area is being processed but the results are not yet available.

[0116] Taking the inspection of a chemical plant as an example, assume that the inspection object map contains multiple tank and pipeline object nodes. In step S10, the computer system will implant corresponding process indicators for these nodes in the map according to the progress of image recognition. For example, if the image recognition of tank A has been completed, a recognition completion process indicator will be implanted in its object node; if the image recognition of pipeline B has not yet started, a pending recognition process indicator will be implanted in its object node; if the image recognition of tank C is in progress, a recognition process indicator will be implanted in its object node.

[0117] Step S20 is performed on the basis of step S10, and its goal is to obtain the object features and corresponding abnormality recognition evaluation information of the image area represented by the object nodes that have completed the image recognition task based on the process indicators implanted in each object node in the inspection object map. This information is crucial for subsequent inspection decisions and abnormality handling.

[0118] In step S20, the computer system will traverse all object nodes in the inspection object map and check their implanted process indicators. For those nodes implanted with identification completion process indicators, the computer will further extract the object features (such as shape, size, color, texture, etc.) of the image areas represented by these nodes and abnormality identification evaluation information (such as whether there is an abnormality, the type of abnormality, the severity, etc.). This information will be used for subsequent abnormality analysis, inspection route optimization, database update and other tasks.

[0119] Continuing with the chemical plant inspection as an example, in step S20, the computer system will check each object node in the inspection object map and find those nodes that have been implanted with the identification completion process indicator. Assuming that this indicator is implanted in the nodes of tank A and pipeline C, the computer will extract the image features of tank A and pipeline C (such as the shape, size, liquid level height of the tank, the color, material, curvature of the pipeline, etc.) and their abnormality identification evaluation information (such as whether the liquid level of tank A is abnormal, whether there is a sign of leakage in pipeline C, etc.).

[0120] Step S30 is the last step of this derivative implementation, and its goal is to associate the object features obtained in step S20 with the corresponding abnormality identification and evaluation information and save them in the database. This step is crucial for establishing a long-term storage and retrieval mechanism for inspection information, which enables inspection data to be reused and analyzed in future inspection tasks.

[0121] In step S30, the computer system combines the object features, abnormal identification evaluation information and related metadata (such as the location of the image area, identification time, identification personnel, etc.) of each object node into a data record. Then, the computer saves this data record to a pre-established database. The database should have efficient data retrieval and query functions so that these inspection data can be quickly accessed and analyzed when needed.

[0122] When saving data, the computer system also ensures that the association between object features and anomaly identification and assessment information is maintained. This means that when anomaly identification and assessment information of a certain object node needs to be queried, the computer system can quickly locate the image features and metadata related to the node. This association is crucial for subsequent anomaly analysis, pattern recognition, and predictive modeling tasks.

[0123] Continuing with the chemical plant inspection as an example, in step S30, the computer will combine the object features, abnormal identification and evaluation information, and related metadata of the storage tank A and pipeline C into data records, and save these data records to the database. When future inspection tasks need to analyze abnormal patterns in these areas or optimize inspection routes, the computer system can quickly access the relevant data in the database and make more accurate and efficient decisions based on this information.

[0124] As an implementation manner, the abnormality recognition evaluation information of the image to be recognized includes: abnormality recognition evaluation information recognized simultaneously and abnormality recognition evaluation information recognized sequentially, wherein the abnormality recognition evaluation information is obtained by performing image recognition by multiple processing units included in a distributed computing environment; then, the manner of performing image recognition by using a distributed computing environment includes: Step S1: obtaining the idle computing power of each processing unit in the distributed computing environment, and screening the target object node for current image recognition in the inspection object map according to the importance weight of each object node in the inspection object map; Step S2: based on the idle computing power of each processing unit and the data size of the image object represented by the target object node, the target processing unit is obtained by screening each processing unit; Step S3: Coordinate the target processing unit to perform image recognition on the image object represented by the target object node, and when the corresponding abnormal recognition evaluation information is obtained, refresh the idle computing power of the target processing unit to obtain refreshed idle computing power.

[0125] Step S1 is the first step of the distributed computing environment image recognition process. Its goal is to obtain the idle computing power of each processing unit in the distributed computing environment, and screen out the target object nodes for current image recognition based on the importance weights of the object nodes in the inspection object map.

[0126] When obtaining idle computing power, the computer system traverses all processing units in the distributed computing environment and collects the current status information of each unit, including the occupied computing resources (such as CPU, memory, GPU, etc.) and the remaining computing power (i.e., idle computing power). This information is crucial for subsequent task scheduling because it determines which processing units can take on new image recognition tasks.

[0127] At the same time, the computer system will also refer to the importance weights of each object node in the inspection object map. These weights are determined in the previous steps, and they reflect the relative importance and processing priority of the object nodes in the inspection task. When screening the target object nodes, the computer will comprehensively consider the idle computing power of the processing unit and the importance weights of the object nodes, and select those object nodes that have both sufficient idle computing power and high importance as the current target for image recognition.

[0128] For example, in a chemical plant inspection scenario, assume that the distributed computing environment contains three processing units A, B, and C, and their idle computing power is 50%, 70%, and 60%, respectively. At the same time, the inspection object map contains multiple tank and pipeline object nodes, among which tank 1 has the highest importance weight, tank 2 is second, and pipeline 1 has the lowest weight. In step S1, the computer will first compare the idle computing power of the three processing units and find that processing unit B has the highest idle computing power. Then, based on the importance weight of the object node, the computer will select tank 1 as the target object node for the current image recognition, because it has the highest importance weight and is likely to complete the recognition task within the idle computing power range of processing unit B.

[0129] Step S2 is performed on the basis of step S1, and its goal is to select the target processing unit that is most suitable for executing the current image recognition task based on the data scale of the image object represented by the target object node (such as image size, resolution, color depth, etc.) and the idle computing power of each processing unit.

[0130] During the processing unit screening process, the computer system will first analyze the data size of the image object represented by the target object node. This data size information is crucial for evaluating the computing power requirements of the processing unit. For example, a high-resolution, large-size image may require more computing resources to process, while a low-resolution, small-size image may only require fewer resources.

[0131] The computer then compares the idle computing power of each processing unit with the data size of the image object. This comparison process may involve some complex calculations, such as evaluating the processing speed, memory usage, and energy consumption of the processing unit under a given data size. Through comparison, the computer system can determine which processing units have enough idle computing power to process the current image object and can complete the recognition task in a reasonable time.

[0132] After screening out potential target processing units, the computer system can further consider other factors, such as communication delays between processing units, load balancing, and fault recovery capabilities. These factors are critical to ensuring the stability and reliability of the distributed computing environment. Ultimately, the computer will select an optimal target processing unit to perform the current image recognition task.

[0133] Continuing with the chemical plant inspection as an example, assume that the image object data size of tank 1 is large and requires higher computing power to process. In step S2, the computer will compare the idle computing power of processing units A, B, and C with the data size of the image object of tank 1. If the idle computing power of processing unit B is sufficient to process this image object, and its other performance indicators (such as communication delay, load balancing, etc.) are also better than other units, then the computer will select processing unit B as the target processing unit to perform the image recognition task of tank 1.

[0134] Step S3 is the last step of the distributed computing environment image recognition process, and its goal is to coordinate the target processing unit to perform image recognition on the image object represented by the target object node, and refresh the idle computing power of the processing unit after obtaining the abnormal recognition evaluation information.

[0135] During the coordination process, the computer system sends the image recognition task and related data (such as image data of the image object, pre-trained model parameters, etc.) to the target processing unit. The computer then monitors the execution progress of the processing unit to ensure that the task can proceed smoothly. Once the processing unit completes the image recognition task and returns the abnormality recognition evaluation information, the computer immediately receives this information and uses it for subsequent inspection decisions and abnormality handling.

[0136] After obtaining the abnormal identification and evaluation information, the computer will refresh the idle computing power of the target processing unit. This refresh process includes updating the occupied computing resources and remaining computing power information of the processing unit. By refreshing the idle computing power, the computer system can grasp the status of each processing unit in the distributed computing environment in real time, and provide accurate data support for subsequent task scheduling and optimization.

[0137] Continuing with the chemical plant inspection as an example, in step S3, the computer will coordinate processing unit B to perform image recognition on the image object of tank 1. Once processing unit B completes the recognition task and returns abnormal recognition evaluation information (such as whether the liquid level of tank 1 is abnormal, whether there are signs of leakage, etc.), the computer will immediately receive this information and adjust the inspection plan or take other necessary measures based on this information. At the same time, the computer will refresh the idle computing power information of processing unit B to reflect its current status and computing power. In this way, when the next image recognition task arrives, the computer system can make more reasonable scheduling decisions based on the latest idle computing power information.

[0138] As an implementation mode, step S2, based on the idle computing power of each processing unit and the data size of the image object represented by the target object node, screens each processing unit to obtain a target processing unit, including: Step S21: obtaining the data size of the image object represented by the target object node; Step S22: Based on the idle computing power of each processing unit, determine in the distributed computing environment a comparison processing unit whose idle computing power can perform image recognition on image objects of corresponding data scale; Step S23: Obtain the determination probability of the control processing unit, and determine the target processing unit according to the determination probability of each processing unit.

[0139] Step S21 is the first step of screening the target processing unit, and its goal is to obtain the data scale of the image object represented by the target object node. Data scale refers to the amount of resources occupied by the image object during storage and processing, which includes parameters such as image size, resolution, color depth, etc. This data scale information is crucial for subsequent processing unit screening because it determines the computing power and storage space required for the processing unit.

[0140] When obtaining the data scale, the computer system first locates the target object node in the inspection object map. Then, the computer extracts the data scale information related to the image object from the node. This information may be stored in the map in the form of metadata, or through the file system associated with the image object. For example, in a chemical plant inspection scenario, if the target object node represents a tank, the computer will extract the size (such as width and height), resolution (such as dots per inch DPI) and color depth (such as bit depth) of the tank image as the data scale of the image object.

[0141] Step S22 is performed on the basis of step S21, and its goal is to determine the control processing unit that can process the image object of the corresponding data scale in the distributed computing environment based on the idle computing power of each processing unit. This process involves the evaluation of the idle computing power of the processing unit and the matching of the image object data scale.

[0142] When evaluating the idle computing power of a processing unit, the computer system will traverse all processing units in the distributed computing environment and collect the current status information of each unit. This information includes the usage of the computing resources (such as CPU, GPU, memory, etc.) of the processing unit, the remaining computing power, and the potential load pressure. With this information, the computer system can calculate the idle computing power of each processing unit, that is, the ability of the unit to handle additional tasks in the current state.

[0143] The computer then matches the data size of the target object node with the idle computing power of each processing unit. This matching process involves converting the data size of the image object (such as image size, resolution, etc.) into the amount of computing resources required by the processing unit (such as computing time, memory usage, etc.), and comparing it with the idle computing power of the processing unit. If the idle computing power of a processing unit is sufficient to process the image object, then the unit is regarded as a control processing unit.

[0144] For example, in a chemical plant inspection, assume that the target object node represents a high-resolution tank image with a large data size. In step S22, the computer converts the data size of the image into the amount of computing resources required by the processing unit and compares it with the idle computing power of each processing unit in the distributed computing environment. If the idle computing power of processing unit A is sufficient to process the image, while the idle computing power of processing unit B is insufficient, then processing unit A will be determined as a control processing unit.

[0145] Step S23 is performed on the basis of step S22, and its goal is to screen out a target processing unit from a plurality of control processing units. When there are a plurality of control processing units, the computer system selects one of them as the target processing unit based on a certain strategy. This process involves calculating and comparing the determination probability of each control processing unit.

[0146] The determination probability refers to the possibility or probability that a processing unit is selected as the target processing unit. In a distributed computing environment, the determination probability is comprehensively evaluated based on multiple factors such as the idle computing power, load pressure, and communication delay of the processing unit. In order to simplify the calculation, an indication method based on the scheduling selection probability can be adopted, that is, the greater the scheduling selection probability, the greater the determination probability.

[0147] When calculating the certainty probability, the computer system can use the idle computing power of the processing unit as the numerator and the total idle computing power of the distributed computing environment as the denominator to obtain a ratio as the certainty probability of the processing unit. This ratio reflects the relative idleness of the processing unit in the current environment, that is, its ability to take on additional tasks.

[0148] The computer then compares the determination probabilities of the control processing units and selects the processing unit with the highest determination probability as the target processing unit. If the determination probabilities of multiple processing units are similar, the computer system can also consider other factors (such as communication delays between processing units, load balancing, etc.) to make the final selection.

[0149] For example, in a chemical plant inspection, assume that there are two control processing units A and B, and their idle computing power is 50% and 60% respectively. In step S23, the computer calculates the determination probability of the two processing units based on the total idle computing power of the distributed computing environment (assuming it is 100%). The determination probability of processing unit A is 50% / 100%=0.5, and the determination probability of processing unit B is 60% / 100%=0.6. Since the determination probability of processing unit B is higher, the computer will select processing unit B as the target processing unit to perform the image recognition task.

[0150] Through the implementation of steps S21 to S23, the computer system can efficiently select the target processing unit that is most suitable for performing the image recognition task. This process not only improves the resource utilization of the distributed computing environment, but also ensures the accuracy and efficiency of the image recognition task.

[0151] As an implementation mode, step S1, obtaining the idle computing power of each processing unit in the distributed computing environment, includes: Step S11: for any processing unit in the distributed computing environment, obtaining the actual load information and consumption ratio of any processing unit; wherein the actual load information represents the number of object nodes for image recognition by any processing unit, and the consumption ratio represents the computing power consumption ratio of any processing unit in the process of image recognition of the corresponding number of object nodes; Step S12: determining the average consumption ratio to be consumed by any processing unit in the process of performing image recognition on an object node according to the consumption ratio and the actual load information; Step S13: Obtain the idle computing power ratio of any processing unit, and obtain the idle computing power of any processing unit based on the consumption ratio mean and the idle computing power ratio.

[0152] Step S11 is the basis of step S1, and its goal is to collect the actual load information and consumption ratio of any processing unit in the distributed computing environment. The actual load information represents the amount of image recognition tasks currently being processed by the processing unit, that is, the number of object nodes for image recognition. The consumption ratio reflects the use of computing power by the processing unit when performing these tasks, that is, the proportion of computing power consumed by image recognition under the corresponding number of object nodes to the total computing power.

[0153] In acquiring the actual load information of the processing unit, the computer system will traverse all the processing units in the distributed computing environment and record the image recognition tasks currently being processed by each unit. These tasks are represented, for example, in the form of object nodes, each node corresponding to an image object to be recognized. By counting the number of object nodes in each processing unit, the computer system can obtain the actual load information of each unit.

[0154] Obtaining the consumption ratio involves monitoring the computing power usage of the processing unit when performing image recognition tasks. This depends, for example, on the performance monitoring tools inside the processing unit or the external performance monitoring system. These tools or systems can record in real time the computing power consumption of the processing unit when processing image recognition tasks, including key indicators such as CPU usage, memory occupancy, and GPU load. By calculating the average or peak value of these indicators during the processing task, the computer system can derive the consumption ratio of each processing unit.

[0155] Taking chemical plant inspection as an example, assume that the distributed computing environment includes three processing units A, B, and C. In step S11, the computer system can find that processing unit A is currently processing 5 image recognition tasks (i.e., 5 object nodes), processing unit B is processing 3 tasks, and processing unit C is in an idle state. At the same time, through the monitoring tool, the computer system can conclude that the computing power consumption ratio of processing unit A when executing these tasks is 70%, processing unit B is 60%, and processing unit C is in an idle state, so its consumption ratio is 0%.

[0156] Step S12 is performed on the basis of step S11, and its goal is to calculate the average computing power consumption ratio of any processing unit when processing a single object node image recognition task, that is, the mean consumption ratio. This calculation is crucial for the subsequent evaluation of the idle computing power of the processing unit, because it can provide a standardized indicator to measure the computing power requirements of the processing unit when processing a single task.

[0157] When determining the mean consumption ratio, the computer system first uses the actual load information and consumption ratio data obtained in step S11. For each processing unit, the computer divides its consumption ratio by the actual load information (i.e., the number of object nodes) to obtain a mean value representing the computing power consumption ratio when processing a single object node. This mean value reflects the average computing power requirement of the processing unit when processing a single image recognition task.

[0158] Continuing with the chemical plant inspection as an example, in step S12, the computer system can calculate that the average consumption ratio of processing unit A is 70% / 5=14%, and the average consumption ratio of processing unit B is 60% / 3=20%. Since processing unit C is in an idle state, its average consumption ratio is 0%. These average data provide an important basis for the subsequent evaluation of the idle computing power of the processing units.

[0159] Step S13 is the last step of step S1, and its goal is to calculate the idle computing power of each processing unit by combining the average consumption ratio of the processing unit and the idle computing power ratio. Idle computing power refers to the amount of computing power resources that the processing unit can use to process additional image recognition tasks in the current state.

[0160] When calculating idle computing power, the computer system first obtains the idle computing power ratio of each processing unit. The idle computing power ratio refers to the ratio of the computing power resources remaining after the processing unit executes the current task to the total computing power. This ratio can be obtained in real time through monitoring tools or systems, for example, and it reflects the amount of remaining computing power resources of the processing unit in the current state.

[0161] The computer then combines the idle computing power ratio of the processing unit with the average consumption ratio to calculate the idle computing power. Specifically, the computer multiplies the idle computing power ratio by (1-average consumption ratio) to obtain the idle computing power ratio of the processing unit that can be used to process additional tasks in the current state. This ratio is multiplied by the total computing power of the processing unit to obtain the specific value of the idle computing power.

[0162] Continuing with the chemical plant inspection as an example, in step S13, it is assumed that the total computing power of processing unit A is 100 units, the idle computing power ratio is 30% (i.e., 30 units of computing power remaining), and the average consumption ratio is 14%. Then, the idle computing power of processing unit A is calculated as 30%*(1-14%)=25.8 units. Similarly, if the total computing power of processing unit B is also 100 units, the idle computing power ratio is 40%, and the average consumption ratio is 20%, then its idle computing power is calculated as 40%*(1-20%)=32 units. For processing unit C, since it is in an idle state and the average consumption ratio is 0%, its idle computing power is the total computing power of 100 units multiplied by the idle computing power ratio (assuming it is 100%, i.e., completely idle), which is also 100 units.

[0163] As an implementation mode, in step S1, according to the importance weight of each object node in the inspection object map, the target object node currently to be image recognized is screened in the inspection object map, including: Step S14: screening in the inspection object map to obtain at least one first object node that has been identified and an upper-level object node of any first object node; Step S15: obtaining the importance weight of each upper-level object node in the inspection object map, and obtaining the set of start-up object nodes to be image recognized; Step S16: determining the importance weight of each second object node in the startup object node set, and adding each upper object node to the startup object node set in order according to the importance weight of each upper object node and the importance weight of each second object node, so that in the startup object node set of each upper object node, each object node is arranged in descending order according to the corresponding importance weight; Step S17: determining the first object node in the startup object node set, and determining the first object node as the target object node for current image recognition.

[0164] In the UAV adaptive inspection method, the core task of step S1 is to select the target object nodes for image recognition in the inspection object map. This process relies on the evaluation of the importance weights of each object node in the map, as well as intelligent decision-making based on these weights and the map structure.

[0165] Step S14 is the starting point of the screening process, and its goal is to find those object nodes (referred to as "first object nodes") that have not completed image recognition in the inspection object map, and further determine the parent object nodes of these nodes. The parent object nodes are, for example, at a higher level in the map, and there is a direct or indirect causal relationship between them and the first object nodes, that is, the state change of the first object node may be affected by the state of the parent object node.

[0166] During the screening process, the computer system first traverses the inspection object map and marks all object nodes that have completed image recognition. Then, it screens out those object nodes that have not been marked, namely the first object nodes. These nodes represent the image objects that have not been processed in the current inspection task and are the focus of subsequent image recognition work.

[0167] Next, the computer system determines the parent object nodes of each first object node based on the connection lines in the graph (representing the causal relationship between object nodes). These connection lines are, for example, directional, pointing from the cause node to the effect node, indicating the transmission direction of the causal relationship. By tracing back the source of the connection lines, the computer system can find all parent object nodes of each first object node.

[0168] Taking the inspection of a chemical plant as an example, assume that the inspection object map contains multiple tank and pipeline object nodes. In step S14, the computer system can select tank A and pipeline B as the first object nodes because they have not yet completed image recognition. At the same time, by tracing the connection lines, the computer will find that the parent object node of tank A is reactor X, and the parent object nodes of pipeline B are tank A and pump Y. In this way, the computer system determines all the first object nodes and their parent object nodes that have not been identified.

[0169] Step S15 is performed on the basis of step S14, and its goal is to obtain the importance weight of each parent object node in the inspection object map, and prepare a startup object node set for storing the object nodes to be identified. The importance weight is an indicator for evaluating the relative importance of the object node in the inspection task, which reflects the degree of influence of the image object represented by the node on the overall inspection result.

[0170] When obtaining the importance weights, the computer system will refer to the weight values ​​determined in the previous step. These weight values ​​may be calculated based on a variety of factors such as the location of the object node, the connection relationship, historical anomaly records, etc. For each parent object node, the computer will query its corresponding weight value and save it in memory for subsequent use.

[0171] At the same time, the computer system will initialize an empty startup object node set. This set will be used to store the object nodes to be identified and processed in a certain order. The initial state of the startup object node set may only contain some basic system nodes or preset starting nodes. As the screening process proceeds, it will gradually contain more object nodes to be identified.

[0172] Step S16 is a key step in the screening process, and its goal is to add the upper-level object nodes to the startup object node set in order according to the importance weight of the upper-level object node and the importance weight of the existing object nodes in the startup object node set. The purpose of this step is to ensure that important object nodes can be processed first, thereby improving inspection efficiency and accuracy.

[0173] During the joining process, the computer system first compares the importance weights of the object nodes already in the starting object node set with the importance weight of the parent object node currently under consideration. If the weight of the parent object node is higher than the weights of all the existing nodes in the set, it is added directly to the beginning of the set (i.e., the highest priority position). If the weight of the parent object node is lower than the weights of some nodes in the set but higher than other nodes, it is inserted at the corresponding position to keep the object nodes in the set arranged in descending order of weight.

[0174] This process is repeated until all parent object nodes have been considered. After each comparison and insertion operation, the starting object node set is updated to reflect the current object node sorting state.

[0175] Taking the inspection of a chemical plant as an example, suppose that in step S15, the computer system obtains the importance weights of reactor X and pump Y and initializes the startup object node set. In step S16, the computer compares the weights of reactor X and pump Y with the weights of the existing nodes in the set. If reactor X has the highest weight, it will be added to the beginning of the set. If pump Y has a lower weight than reactor X but higher than other nodes in the set, it will be inserted into the appropriate position. In this way, the startup object node set is sorted according to the importance weights of the object nodes.

[0176] Step S17 is the last step of the screening process, and its goal is to determine the target object node currently to be image recognized from the sorted set of startup object nodes. This node will be the next object to be processed, and its image recognition result will be used for subsequent inspection decisions and abnormality analysis.

[0177] When determining the target object node, the computer system will check the beginning position of the start object node set (i.e. the highest priority position). The object node at this position is the target object node for the current image recognition. The computer will record the identifier or related attribute information of this node and pass it to the subsequent image recognition module for processing.

[0178] Once the target object node is determined and processed, the computer removes it from the startup object node set and repeats the process of steps S14 to S17 to filter out the next target object node. This process will continue until all the object nodes to be identified have been processed.

[0179] Taking the chemical plant inspection as an example, in step S17, the computer system can determine that reactor X is the target object node for the current image recognition. Once the image recognition task of reactor X is completed, it will be removed from the set of start object nodes. Then, the computer will repeat the screening process, determine the next target object node (which may be pump Y or other unrecognized object nodes), and continue processing.

[0180] Through the implementation of steps S14 to S17, the computer system can intelligently screen out the target object nodes for image recognition based on the importance weights and graph structure of the object nodes in the inspection object graph. This process not only improves the efficiency and accuracy of the inspection task, but also ensures that important object nodes can be processed first, thereby enhancing the reliability and practicality of the inspection results.

[0181] As an implementation method of a processing unit, each processing unit includes a processing component and a dynamic monitoring component; wherein the processing component is configured to perform image recognition, and the dynamic monitoring component is configured to determine the actual load information and consumption ratio of the corresponding processing unit in the distributed computing environment; the distributed computing environment is configured to implement distributed computing, and the distributed computing is coordinated with the coordination task to perform image recognition at the same time; the coordination task includes at least one of a graph generation component, a process coordination component and a data storage component; wherein the graph generation component is configured to generate an inspection object graph of the image to be identified; the process coordination component is configured to coordinate the distributed computing environment based on the inspection object graph, and perform image recognition through the distributed computing environment; the data storage component is configured to save the abnormal recognition evaluation information of the image to be identified and the abnormal recognition evaluation information of each image area in the image to be identified.

[0182] In the UAV adaptive inspection method, the processing unit, as the core component in the distributed computing environment, undertakes multiple responsibilities such as image recognition, resource monitoring, task coordination and data preservation. Each processing unit is mainly composed of two core components: the processing component and the dynamic monitoring component. The processing component is the core of the processing unit and is responsible for performing image recognition tasks. It uses advanced image processing algorithms and machine learning models (such as convolutional neural networks CNN, deep learning, etc.) to efficiently and accurately recognize the input image data. These algorithms and models can automatically extract key features in the image, such as shape, texture, color, etc., and match them with predefined target categories, thereby achieving accurate recognition of image objects.

[0183] The dynamic monitoring component is responsible for monitoring the real-time status of the processing unit, including its actual load information and consumption ratio. The actual load information reflects the amount of image recognition tasks currently being processed by the processing unit, that is, the number of object nodes that are undergoing image recognition. The consumption ratio refers to the use of computing power by the processing unit when performing these tasks, that is, the proportion of computing power consumed by processing image recognition tasks to the total computing power. By continuously monitoring this information, the dynamic monitoring component can evaluate the load status of the processing unit in real time, providing an important basis for subsequent task scheduling and resource allocation.

[0184] A distributed computing environment is a network of multiple processing units that are interconnected through a high-speed network and jointly undertake complex computing tasks. In this environment, each processing unit is an independent computing node that can process image recognition tasks in parallel, thereby significantly improving the overall computing efficiency. In order to coordinate the work of these processing units, the distributed computing environment also contains a series of coordination task components, including a graph generation component, a process coordination component, and a data storage component. The graph generation component is responsible for generating an inspection object graph of the image to be identified. This graph is a graphical data structure that contains the various object nodes in the image and the causal relationships between them. Through the graph, the computer system can clearly understand the layout and relationships of the image objects, providing strong support for subsequent image recognition tasks.

[0185] The process coordination component is responsible for coordinating the distributed computing environment based on the inspection object map. It determines which processing units should handle which image recognition tasks based on the object nodes and connection line information in the map, and allocates corresponding computing resources. At the same time, the process coordination component is also responsible for monitoring the real-time status of the processing units to ensure that tasks can be executed in the predetermined order and priority.

[0186] The data storage component is responsible for saving the abnormal recognition evaluation information of the image to be identified and the abnormal recognition evaluation information of each image area. This information is an important output of the inspection task and is of great significance for subsequent abnormal analysis, fault prediction and decision support. The data storage component will store this information in a reliable database and ensure the integrity and accessibility of the data.

[0187] Taking the inspection of a chemical plant as an example, assume that the distributed computing environment contains three processing units A, B, and C. Each processing unit is equipped with advanced processing components and dynamic monitoring components. When the drone captures the image to be identified of the chemical plant, the map generation component will first generate an inspection object map based on the image content. This map contains the object nodes such as tanks, pipelines, valves, etc. in the chemical plant and the connection relationships between them.

[0188] Then, the process coordination component determines which processing units should handle which image recognition tasks based on the inspection object map and the real-time status of each processing unit (such as actual load information and consumption ratio). For example, if processing unit A has a high idle computing power and is not currently processing other tasks, it can be assigned to handle the image recognition task of a key tank. At the same time, the dynamic monitoring component will continue to monitor the status of processing unit A to ensure that it can successfully complete the task.

[0189] During the processing, the processing component uses advanced image recognition algorithms to process the input image data. For example, for the image recognition task of a storage tank, the processing component can extract key features such as the shape, size, and liquid level of the tank and match them with predefined tank categories. Once the recognition is completed, the processing component sends the results to the data storage component for storage.

[0190] The data storage component stores the abnormality identification and evaluation information and the abnormality identification and evaluation information of each image area in the database. For example, if the liquid level of a tank rises or falls abnormally, the data storage component will record this abnormal information and its related image area information. This information will help in subsequent abnormality analysis, fault prediction and decision support.

[0191] In this way, the processing unit and its components can work together to complete the UAV adaptive inspection task. They not only improve the efficiency and accuracy of image recognition, but also achieve optimal resource utilization and dynamic scheduling of tasks through technical means such as distributed computing and real-time monitoring. This implementation method is not only applicable to chemical plant inspection scenarios, but can also be extended to other fields that require efficient image processing and distributed computing.

[0192] Figure 2 A hardware entity diagram of a computer system provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the hardware entity of the computer system 1000 includes: a processor 1001 and a memory 1002, wherein the memory 1002 stores a computer program that can be run on the processor 1001, and the processor 1001 implements the steps in the method of any of the above embodiments when executing the program.

[0193] The above is only an implementation method of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. An adaptive inspection method for unmanned aerial vehicles in the field of industrial defense, characterized in that: include: Acquire an image to be identified of a target three-defense area taken by a drone; wherein the image to be identified includes multiple image areas, and an image object in one image area represents a facility to be detected; Generate an inspection object map based on the image objects in each image area in the image to be identified and the causal relationship between different image objects; the inspection object map includes multiple object nodes and relationship connection lines, one object node is used to represent an image object in an image area, and one relationship connection line is used to represent the causal relationship between the image objects represented by the connected object nodes; When performing image recognition on the image to be recognized according to the inspection object map, a control object node capable of simultaneous recognition is obtained by screening from the inspection object map; Performing simultaneous recognition processing on the image objects represented by the reference object nodes to obtain abnormality recognition evaluation information of the image to be recognized; A route confirmation instruction is generated based on the abnormality identification and evaluation information, and the route confirmation instruction is sent to the drone so that the drone responds to the route confirmation instruction to perform an inspection.

2. The method according to claim 1, characterized in that The generated inspection object map is an optimized inspection object map, and the relationship connection lines in the optimized inspection object map are connection lines with directions; the generation of the inspection object map based on the image objects in each image area in the image to be identified and the causal relationship between different image objects includes: Generate a basic inspection object map based on the image objects in each image area of ​​the image to be identified and the causal relationship between different image objects; the object nodes included in the basic inspection object map are used to represent image objects, and the connection lines between the object nodes are from the correspondingly represented effect image object to the cause image object, wherein the state of the effect image object is affected by the change of the state of the cause image object; Based on the object nodes and directions of connection lines included in the basic inspection object map, the basic inspection object map is optimized to obtain an optimized inspection object map.

3. The method according to claim 2, characterized in that The step of optimizing the basic inspection object graph based on the object nodes and directions of the connection lines included in the basic inspection object graph to obtain an optimized inspection object graph includes: Searching the database for abnormal recognition evaluation information of the image object represented by any object node in the basic inspection object map; If the abnormality recognition evaluation information of the image object represented by any one of the object nodes is searched, then based on the position of any one of the object nodes in the basic inspection object map and the direction of the connection line contained in the basic inspection object map, determine the remaining object nodes having a direction relationship with the any one of the object nodes; Invalid nodes are cleared for any one of the object nodes and the remaining object nodes to obtain an optimized inspection object graph.

4. The method according to claim 3, characterized in that The database stores object features of image objects for which abnormality recognition assessment information has been determined; The step of searching the database for abnormality recognition evaluation information of an image object represented by any object node in the basic inspection object graph includes: Obtaining a reference object feature of an image object represented by any object node in the basic inspection object map; The database is searched for object features that satisfy feature similarity conditions with the reference object features, and when object features that satisfy feature similarity conditions with the reference object features are matched from the database, it is determined that the abnormality recognition evaluation information of the image object represented by any one of the object nodes is matched; otherwise, it is determined that the abnormality recognition evaluation information of the image object represented by any one of the object nodes is not matched.

5. The method according to claim 1, characterized in that The step of screening the inspection object graph to obtain the control object nodes capable of simultaneous identification includes: Determine the importance weight of each object node based on the inspection object map; the importance weight represents the processing order of the image objects represented by the corresponding object nodes when performing image recognition, and the image objects represented by the object nodes corresponding to the same importance weight have the same processing order when performing image recognition; Object nodes corresponding to the same importance weight are used as comparison object nodes that can be simultaneously identified.

6. The method according to claim 5, characterized in that The determining the importance weight of each object node based on the inspection object graph includes: Obtaining a hierarchical structure of any object node in the inspection object graph; An importance weight of the any object node is determined based on the hierarchical structure of the any object node.

7. The method according to claim 6, characterized in that If the inspection object map has an object node that is consistent with the hierarchical structure of any object node and is different from the parent object node of any object node, the method further includes: Acquire the number of first object nodes that correspond to the same upper-level object node as any one of the object nodes, and the number of second object nodes that correspond to different upper-level object nodes as any one of the object nodes and have the same hierarchical structure, wherein the first object node and the second object node are both object nodes corresponding to image objects that have not been identified; If the number of the first object nodes is greater than the number of the second object nodes, the importance weight of any one of the object nodes is greater than the importance weight of an object node that corresponds to a different upper-level object node and has the same hierarchical structure as the any one of the object nodes; If the number of the first object nodes is less than the number of the second object nodes, the importance weight of any one of the object nodes is less than the importance weight of an object node that corresponds to a different upper-level object node and has the same hierarchical structure as the any one of the object nodes; If the number of the first object nodes is equal to the number of the second object nodes, the importance weight of any one of the object nodes is equal to the importance weight of an object node that corresponds to a different upper-level object node and has the same hierarchical structure as the any one of the object nodes.

8. The method according to claim 1, characterized in that The abnormality recognition evaluation information of the image to be recognized includes abnormality recognition evaluation information of each image area in the image to be recognized; the method further includes: Based on the recognition status of image recognition of each image area in the image to be recognized, a process indicator is implanted for the object node corresponding to the corresponding image area in the inspection object map; the process indicator includes any one of the following process indicators: a recognition completion process indicator, a to-be-recognized process indicator, and a recognition process indicator; Based on the process indicator implanted in each object node in the inspection object map, obtaining the object features of the image area in the object node implanted with the identification completion process indicator and the corresponding abnormality identification evaluation information; The acquired object features and corresponding anomaly identification and evaluation information are associated and saved in a database; The abnormality recognition evaluation information of the image to be recognized includes: abnormality recognition evaluation information recognized simultaneously and abnormality recognition evaluation information recognized in sequence, wherein the abnormality recognition evaluation information is obtained by performing image recognition by multiple processing units included in a distributed computing environment; the manner of performing image recognition by the distributed computing environment includes: Obtaining the idle computing power of each processing unit in the distributed computing environment, and screening the target object node for current image recognition in the inspection object map according to the importance weight of each object node in the inspection object map; Based on the idle computing power of each processing unit and the data size of the image object represented by the target object node, a target processing unit is obtained by screening the processing units; The target processing unit is coordinated to perform image recognition on the image object represented by the target object node, and when corresponding abnormality recognition evaluation information is obtained, the idle computing power of the target processing unit is refreshed to obtain refreshed idle computing power.

9. The method according to claim 8, characterized in that The step of selecting a target processing unit from the processing units based on the idle computing power of the processing units and the data size of the image object represented by the target object node includes: Acquire the data size of the image object represented by the target object node; Based on the idle computing power of each processing unit, determining in the distributed computing environment a comparison processing unit whose idle computing power can perform image recognition on image objects of a corresponding data scale; Obtaining the determination probability of the control processing unit, and determining the target processing unit according to the determination probability of each processing unit; The obtaining of the idle computing power of each processing unit in the distributed computing environment includes: For any processing unit in the distributed computing environment, the actual load information and consumption ratio of the any processing unit are obtained; wherein the actual load information represents the number of object nodes for image recognition by the any processing unit, and the consumption ratio represents the computing power consumption ratio of the any processing unit in the process of image recognition of the corresponding number of object nodes; Determine, according to the consumption ratio and the actual load information, a mean value of the consumption ratio to be consumed by any one of the processing units in the process of performing image recognition on an object node; Obtaining the idle computing power ratio of any one of the processing units, and obtaining the idle computing power of any one of the processing units based on the average of the consumption ratios and the idle computing power ratio; The step of screening the target object node to be currently subjected to image recognition in the inspection object map according to the importance weight of each object node in the inspection object map comprises: Screening in the inspection object graph to obtain at least one first object node that has been identified and an upper-level object node of any first object node; Obtaining the importance weight of each upper-level object node in the inspection object graph, and obtaining a set of starting object nodes for image recognition; Determine the importance weight of each second object node in the startup object node set, and according to the importance weight of each upper object node and the importance weight of each second object node, add each upper object node to the startup object node set in order, so that in the startup object node set of each upper object node, each object node is arranged in descending order according to the corresponding importance weight; Determine the first object node in the startup object node set, and determine the first object node as the target object node for current image recognition; Each processing unit includes a processing component and a dynamic monitoring component; wherein the processing component is configured to perform image recognition, and the dynamic monitoring component is configured to determine the actual load information and consumption ratio of the corresponding processing unit in the distributed computing environment; the distributed computing environment is configured to implement distributed computing, and the distributed computing is coordinated with the coordination task and image recognition is performed simultaneously; the coordination task includes at least one of a map generation component, a process coordination component and a data storage component; wherein the map generation component is configured to generate an inspection object map of the image to be identified; the process coordination component is configured to coordinate the distributed computing environment based on the inspection object map, and perform image recognition through the distributed computing environment; the data storage component is configured to save the abnormal recognition evaluation information of the image to be identified and the abnormal recognition evaluation information of each image area in the image to be identified.

10. A computer system comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, wherein: When the processor executes the program, the steps in the method according to any one of claims 1 to 9 are implemented.

Images