An automatic topology method and system based on machine vision

Abstract

The application relates to an automatic topology method and system based on machine vision, which comprises the following steps: attaching a label printed with label data on a surface to a standard position of equipment or wiring in a corresponding place when a business change is implemented; shooting a live video image and a label image; identifying the type of equipment and the position relationship thereof in the live image according to an image processing method, and identifying label data of the label image; querying and obtaining business coding information from a system database according to the label data; generating a new version of equipment topology according to the type of equipment, the business coding information and the position relationship; visualizing a topology graph, and giving a pre-warning prompt for changes in the topology structure.

Description

Technical Field

[0001] This invention relates to the field of automatic topology discovery technology for industrial system equipment, and in particular to an automatic topology discovery method and system for equipment in automated control systems such as power grids, communications, and manufacturing. Background Technology

[0002] In digital systems such as automation control and production management in industries like power grids, communications, and manufacturing, equipment topology is a structured representation of device connections using network topology diagrams. On one hand, equipment topology technology can improve the user-friendliness and intuitiveness of human-computer interaction; on the other hand, as business knowledge, equipment topology technology can be combined with artificial intelligence to enhance the system's intelligence.

[0003] Especially in scenarios with frequent changes in power distribution networks and communication access, the maintenance of equipment topology in various substations and cabinets presents challenges such as high workload, repetitive tasks, and cumbersome operations. Due to additions, repairs, modifications, and decommissioning, equipment and wiring frequently change, requiring corresponding changes in production and operation systems, and necessitating maintenance of the network topology of multiple systems. Automatic topology discovery technology utilizes IoT, intelligent sensing, and discovery algorithms to detect physical changes in equipment and wiring, automatically synchronizing equipment topology information, significantly reducing reliance on manual intervention and improving the consistency of basic equipment topology information. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides an automated topology method and system based on machine vision. It uses unified IoT coding tags to uniquely identify physical entities of devices and wiring. Machine vision is employed to identify the physical location relationships of devices and wiring in real time. Combined with a domain-specific device knowledge graph, a device topology model is simultaneously constructed and verified. When devices or wiring change, changes are detected in real time, and the device topology information is synchronized to the system database. The system can automatically compare the designed topology with the actual topology, detect errors, and issue warnings. The entire process is automated by the automated topology system, significantly improving work efficiency and data consistency.

[0005] The technical solution adopted by the present invention to achieve the above objectives is as follows:

[0006] An automatic topology method and system based on machine vision, comprising:

[0007] The device tag module is used to generate and maintain tag data with business semantic information and industrial internet identifiers;

[0008] Labels, with label data printed on their surface, are used to specify the location of equipment or wiring in the corresponding location when business changes are implemented;

[0009] The on-site camera module captures on-site video images and tag images;

[0010] The machine vision recognition module identifies the types of equipment and their location relationships in the field image based on image processing methods, as well as the label data of the label image.

[0011] The device topology discovery module is used to query the system database to obtain service code information based on tag data; and to generate a new version of the device topology based on device type, service code information and location relationship.

[0012] The domain equipment knowledge graph module is used to describe the equipment information and connection relationships of existing industry equipment in the form of a graph;

[0013] The verification and early warning module is used to provide early warnings for changes in the topology.

[0014] The system database is used to store tag information, business semantic information, device topology information, and corresponding image information.

[0015] The device tag module includes:

[0016] The tag generation unit takes the actual information of the devices and connections involved in the business changes as input conditions, and generates corresponding tag data and industrial internet identifiers based on the business coding information. The business coding information includes multiple attribute codes to express the business codes required for device topology construction, including business system, device type, device model, interface type, location information, business type, business number, and entity number. The industrial internet identifier is a code that can be parsed by the industrial internet identifier resolution system.

[0017] The tag maintenance unit is used to update business codes and industrial internet identifiers in real time.

[0018] The label data printed on the label surface is in the form of a barcode or QR code.

[0019] The label is attached by pasting, hanging, or fixing.

[0020] The on-site camera module consists of one or more fixed or autonomously movable network camera devices and an on-site network.

[0021] The machine vision recognition module includes:

[0022] The target classification and detection unit uses the YOLO-V5 target recognition algorithm to identify the device body or label in the on-site image;

[0023] A label recognition unit is used to recognize label data in a label image;

[0024] The target relative position measurement and estimation unit calculates the actual relative position of the device and the tag based on the pixel position of the target selection location in the image plane coordinates, and outputs a list of device type, tag data and positional relationship; the positional relationship is the actual relative position of the device in the current field environment.

[0025] The domain equipment knowledge graph module uses a graph database approach to describe industry equipment, including equipment information, historical operation event information, and connection relationships.

[0026] The equipment information includes equipment model, function, size, and appearance; the connection relationship includes interface location and interface constraints; the historical operation event information includes equipment creation, dismantling, commissioning, decommissioning, relocation, maintenance, and fault event information, which are maintained and updated through manual addition, modification, and import.

[0027] The device topology discovery module obtains device type, tag data, location relationships, and corresponding on-site image files from the machine vision recognition module, and obtains device information and connection constraints from the domain device knowledge graph; including:

[0028] The business information decoding unit uses tag data as a condition to query the system database to decode and obtain business encoding information;

[0029] The topology building unit generates a new version of the device topology based on service coding information and location relationships;

[0030] The topology verification unit checks the correctness of the device topology based on the device connection constraints.

[0031] The topology comparison unit compares the new device topology with the original device topology and synchronizes the changes to the system database.

[0032] An automated method based on machine vision includes the following steps:

[0033] S1: Tag Assignment: Create tag files and attach them to the specified locations on the equipment or wiring in the corresponding locations when business changes are implemented.

[0034] 2) Image Acquisition: On-site image acquisition obtains images of on-site equipment and tag images:

[0035] 3) Identification: Based on image processing methods, identify the equipment types and their location relationships in the field images, and identify the tag data of the tag images:

[0036] 4) Automatic Topology Construction: Using tag data as a condition, the system database is queried to obtain service code information; a new device topology is generated based on device type, service code information, and location relationships.

[0037] 5) Topology verification and update: Check the correctness of the device topology based on the device connection constraints;

[0038] 6) Topology Comparison and Early Warning: Compare the new device topology with the original device topology and provide early warnings for changes in the topology structure;

[0039] 7) After the above process is completed, one automatic topology discovery process is finished. Repeat steps 1)-7) to perform periodic automatic topology discovery.

[0040] The present invention has the following beneficial effects and advantages:

[0041] 1) The integration of tag semantics and machine vision has enabled the automation of consistent maintenance of device information and topology information, reduced the technical implementation cost, and made large-scale automatic device topology application a reality.

[0042] 2) The field camera module consists of network cameras and a field network. Multiple fixed or autonomously moving network cameras can be flexibly configured according to the field conditions. It can adapt to environments such as open spaces, complex structural spaces, and double-panel cabinets, greatly improving the automation rate.

[0043] 3) The device topology is automatically constructed using the knowledge graph of the integrated domain devices, which has good scalability and can be extended to support new models or types of devices and wiring rules;

[0044] 4) The entire working process includes coding, identification, generation, verification, and early warning of equipment topology-related information, forming a complete closed-loop control with a high degree of automation. Attached Figure Description

[0045] Figure 1 This is a schematic diagram of the structure of an automated topology system based on machine vision.

[0046] Figure 2 This is a flowchart of an automatic topology method based on machine vision. Detailed Implementation

[0047] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0048] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0049] like Figure 1 The diagram illustrates an automatic topology system based on machine vision, comprising a device tag module, tags, a field camera module, a machine vision recognition module, a domain device knowledge graph module, a device topology module, a system database, and a verification and early warning module. The device tag module is used to uniformly generate and maintain tag codes with business semantic information. When business changes are implemented, the tags with these codes are attached to the corresponding standardized locations on the equipment or wiring. The field camera module reads on-site video from the equipment and transmits it to the machine vision recognition module. The machine vision recognition module identifies the device type, location relationship, and tag data. The device topology discovery module, using the tag data as a condition, retrieves business semantic information from the system database, calls the domain device knowledge graph module, generates the device topology, and uploads it to the system database. The verification and early warning module reads the designed topology from the system database and compares it with the actual topology. If inconsistencies are found, an early warning message is generated.

[0050] The device tag module is the foundation of IoT sensing, consisting of business coding rules, tag generation, and tag maintenance. The business coding rules are composed of multiple segments, expressing the business codes required for device topology construction, including but not limited to business system, device type, device model, interface type, location information, business type, business number, and entity number. Using the actual information of the devices and connections involved in business changes as input conditions, corresponding tag data is generated according to the business coding rules. Business system refers to PMIS, OMS, ERP, SCADA, distribution network maintenance, etc.; device type refers to the category of the device (e.g., the current device type is a transformer); interface type refers to the connection interface provided by the device (e.g., the device provides network port or serial port connection); location information refers to the actual placement location of the device, including but not limited to geographical coordinates and the plant / station to which it belongs; business type refers to actual power grid business such as dispatching, marketing, operation and maintenance, and metering; business number is the internal coding of various businesses by the unit using this invention or the unified coding of various businesses by this invention; and entity number refers to the unique ID number of the device. The industrial internet identifier is a code that can be parsed by the industrial internet identifier resolution system. The industrial internet identifier resolution system is an existing resolution system.

[0051] The label is the carrier of label data, which is printed on the label surface in the form of barcodes, QR codes, etc. During on-site business implementation, depending on the type of equipment and connection, it is attached to the agreed location by means of pasting, hanging or fixing. The equipment site or cabinet also needs corresponding labels. All labels are visible and accessible to the on-site camera module.

[0052] The on-site camera module is a means of sensing changes on-site, consisting of one or more fixed or autonomously movable network camera devices and an on-site network; the network camera device is responsible for acquiring on-site images, and the on-site network is responsible for transmitting on-site images.

[0053] The machine vision recognition module consists of target classification and detection, label recognition, and target relative position measurement and estimation. Target classification and detection uses common target recognition algorithms such as YOLOv5 to identify the equipment and labels in the scene image. Label recognition uses common barcode and QR code recognition algorithms to identify the labels. Target relative position measurement and estimation calculates the relative positions of the equipment and labels based on the pixel positions of the target selection location in the image plane, resulting in a list of equipment types, labels, and positional relationships. The target selection location is either a selection mark automatically made by the system for automatically identified equipment or a selection box manually made in the video image.

[0054] The domain equipment knowledge graph module uses a graph database approach to describe industry equipment, including but not limited to equipment model, function, size, appearance, interface location, interface constraints, and other knowledge. It can be maintained and updated through manual addition, modification, and import.

[0055] The device topology discovery module is the core module, consisting of business information decoding, topology construction, topology verification, and topology comparison. The device topology discovery module obtains device type, tags, location relationships, and corresponding field image files from the machine vision recognition module. It also obtains device information and connection constraints from the domain device knowledge graph. Business information decoding is performed by querying the system database using tag data as a condition, obtaining information including but not limited to business systems, device types, device models, interface types, location information, business types, business numbers, and entity numbers. Topology construction generates a new version of the device topology based on the decoded business information and location relationships. Topology verification checks the correctness of the device topology based on device connection constraints. Topology comparison compares the new version of the device topology with the original version, synchronizing the changes to the system database. The topology map generated by this patent includes the actual physical location information (geographic coordinates) of the equipment in the current field area, as well as the direction of business parameters (e.g., power transmission and generation directions, and connection relationships between devices).

[0056] The system database is responsible for uniformly storing tag information, business semantic information, device topology information, corresponding image information, etc.

[0057] The inconsistency warning module generates a warning message when the device topology in the new version differs from the original version, and the device topology relationships do not conform to the constraints described in the domain device knowledge graph. The warning message can take the form of a dialog box or a short message.

[0058] like Figure 2 The diagram illustrates the workflow of an automatic topology method based on machine vision according to the present invention. The entire process includes encoding, identification, generation, verification, and early warning of device topology-related information, forming a complete closed-loop control with a high degree of automation. The fusion of tag semantics and machine vision automates the maintenance of consistency between device and topology information, reduces technical implementation costs, and makes large-scale automatic topology applications a reality. The specific steps of the method are as follows:

[0059] 1) Equipment coding and label changes: Create label files and attach them to the specified locations of equipment or wiring in the corresponding areas when business changes are implemented.

[0060] 2) On-site image acquisition: Acquire images of on-site equipment and tags;

[0061] 3) Equipment and connection identification: Identify the equipment types and their location relationships in the field images based on image processing methods, and identify the tag data of the tag images;

[0062] 4) Automatic Topology Construction: Based on tag data, retrieve business code information from the system database; generate a new device topology according to device type, business code information, and location relationships;

[0063] 5) Topology verification and update: Check the correctness of the device topology based on the device connection constraints;

[0064] 6) Topology Comparison and Early Warning: Compare the new device topology with the original device topology and provide early warnings for changes in the topology structure;

[0065] 7) After the above process is completed, one automatic topology discovery process is finished. Repeat steps 1)-7) to perform periodic automatic topology discovery.

[0066] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. An automated topology system based on machine vision, characterized in that, include: The device tag module is used to generate and maintain tag data with business semantic information and industrial internet identifiers; Labels, with label data printed on their surface, are used to specify the location of equipment or wiring in the corresponding location when business changes are implemented; The on-site camera module captures on-site video images and tag images; The machine vision recognition module identifies the types of equipment and their location relationships in the field image based on image processing methods, as well as the label data of the label image. The device topology discovery module is used to query the system database to obtain service code information based on tag data; and to generate a new version of the device topology based on device type, service code information and location relationship. The domain equipment knowledge graph module is used to describe the equipment information and connection relationships of existing industry equipment in the form of a graph; The verification and early warning module is used to provide early warnings about changes in the topology. Warning messages are delivered via dialog boxes and text messages. The system database is used to store tag information, business semantic information, device topology information, and corresponding image information; The machine vision recognition module includes: The target classification and detection unit uses the YOLO-V5 target recognition algorithm to identify the device body or label in the on-site image; A label recognition unit is used to recognize label data in a label image; The target relative position measurement and estimation unit calculates the actual relative position of the device and the tag based on the pixel position of the target selection location in the image plane coordinates, and outputs a list of device type, tag data and positional relationship; the positional relationship is the actual relative position of the device in the current field environment; The device topology discovery module obtains device type, tag data, location relationships, and corresponding on-site image files from the machine vision recognition module, and obtains device information and connection constraints from the domain device knowledge graph; including: The business information decoding unit uses tag data as a condition to query the system database to decode and obtain business encoding information; The topology building unit generates a new version of the device topology based on service coding information and location relationships. The topology verification unit checks the correctness of the device topology based on the device connection constraints. The topology comparison unit compares the new device topology with the original device topology and synchronizes the changes to the system database. The domain equipment knowledge graph module uses a graph database approach to describe industry equipment, including equipment information, historical operation event information, and connection relationships. The equipment information includes equipment model, function, size, and appearance. Connection relationships include interface location and interface constraints. Historical operation event information includes equipment creation, dismantling, commissioning, decommissioning, relocation, maintenance, and fault event information, which is maintained and updated through manual addition, modification, and import.

2. The automatic topology system for machine vision according to claim 1, characterized in that, The device tag module includes: The tag generation unit takes the actual information of the devices and connections involved in the business changes as input conditions, and generates corresponding tag data and industrial internet identifiers based on the business coding information. The business coding information includes multiple attribute codes to express the business codes required for device topology construction, including business system, device type, device model, interface type, location information, business type, business number, and entity number. The industrial internet identifier is a code that can be parsed by the industrial internet identifier resolution system. The tag maintenance unit is used to update business codes and industrial internet identifiers in real time.

3. The automatic topology system for machine vision according to claim 1, characterized in that, The label data printed on the label surface is in the form of a barcode or QR code.

4. The automatic topology system for machine vision according to claim 1, characterized in that, The label is attached by pasting or hanging.

5. The automatic topology system for machine vision according to claim 1, characterized in that, The on-site camera module consists of one or more fixed or autonomously movable network camera devices and an on-site network.

6. An automatic topology method based on machine vision, characterized in that, The system is based on any one of claims 1-5, and the method includes the following steps: 1) Tag assignment: Create tag files and attach them to the specified locations of equipment or wiring in the corresponding locations when business changes are implemented; 2) Image Acquisition: Acquire images of on-site equipment and tags; 3) Identification: Identify the equipment types and their location relationships in the field images based on image processing methods, and identify the tag data of the tag images; 4) Automatic Topology Construction: Based on tag data, retrieve business code information from the system database; generate a new device topology according to device type, business code information, and location relationships; 5) Topology verification and update: Check the correctness of the device topology based on the device connection constraints; 6) Topology Comparison and Early Warning: The new device topology is compared with the original device topology, and early warnings are issued for changes in the topology structure; the early warning information is delivered via dialog boxes and text messages. 7) After steps 1)-6) above are completed, the entire automatic topology discovery process is finished; Repeat steps 1) through 7) to perform periodic automatic topology discovery.

Images