An image modeling and recognition method based on image templates

Abstract

The application constructs a flexible and easy-to-expand intelligent image recognition algorithm management and cooperation framework through an image template-based image modeling and recognition method; unified management of artificial intelligence algorithms based on image processing and deep learning is completed through image template modeling; image recognition services perform unified scheduling of recognition algorithms and image recognition according to modeling information, concurrent recognition of the states and readings of different types of equipment in the recognition object, and recognition of abnormal scenes are realized. The method can effectively solve the problem of different types of equipment data sample quantity in different substations at different implementation stages of actual projects, provide adaptive recognition algorithm strategies, and can complete iterative maintenance of recognition algorithms simply and quickly according to gradually enriched data samples, and has the advantages of scientific and reasonable method, strong applicability, simple operation, high recognition accuracy, good implementation effect and the like.

Description

Technical Field

[0001] This invention relates to the fields of intelligent image recognition inspection of rail-powered substations, digital power plants, and intelligent image recognition inspection of power grid substations. Background Technology

[0002] Currently, although railway traction substations, digital substations, power plants, and rail transit control rooms have largely installed integrated automation systems, meeting the daily work needs of dispatchers to a certain extent, they do not significantly improve the automation level of operation and maintenance personnel's work. Routine inspections closely related to these tasks, such as checking for leaks in equipment rooms, the integrity of equipment exteriors, the presence of foreign objects in the operating environment, abnormal alarms from equipment, and verification of readings or statuses of various types of equipment, remain unresolved. In fact, this may even lead to safety hazards. Furthermore, for maintenance personnel, this not only results in low work efficiency but also exposes them to risks such as human error. Additionally, outdoor inspections of substations are affected by weather conditions. Therefore, automated and intelligent inspection and maintenance methods are urgently needed to effectively improve substation operation and maintenance levels, reduce security risks caused by leaks in equipment and equipment rooms, and ultimately promote unmanned operation of substations.

[0003] With the continuous improvement of computer hardware performance and the development of technologies such as artificial intelligence and image information processing, these technologies can be innovatively applied in a variety of traditional and emerging industrial fields. Visual image detection and recognition of various types of equipment in railway power supply substations and power grid substations can replace manual operations such as meter reading, equipment condition checks, and personnel intrusion detection in high-voltage areas. This avoids human-introduced data errors, reduces manpower burden, ensures the safety of staff and the stable operation of the power system, and achieves the goals of low-cost, convenient, and fully unmanned detection. Furthermore, it promotes the intelligent upgrading of substations, enabling substation-related work to move towards unmanned operation, reconfigurability, information digitization, functional integration, compact structure, and status visualization.

[0004] However, based on practical engineering experience, there are many types of substations in various industrial sectors. For example, in urban rail transit power supply systems, there are generally several types of substations, such as main substations, traction step-down hybrid substations, step-down substations, and follow-type step-down substations. Railway power supply and urban power grids also include different types of substations. And substations contain a wide variety of equipment.

[0005] From a technical perspective, image recognition can be broadly categorized into two approaches: image processing-based methods and deep learning-based methods. Traditional image recognition algorithms have relatively low requirements for image data and produce somewhat mechanical processing results. They are effective for objects with simple backgrounds and low target complexity. However, because they can only extract shallow features such as shape, orientation, and color, their generalization ability is weak, making it difficult to uncover the rich, intrinsic information of the image. Therefore, they perform poorly in recognizing targets in complex scenes. Data-driven deep learning methods effectively overcome the subjectivity, ambiguity, and uncertainty of manually designed features in image processing methods. They can fully mine the deep semantic features of image data and have a wider range of applications than image processing methods. However, they heavily rely on factors such as the size and diversity of the data, and the balance of sample sizes.

[0006] Currently, a common problem with image recognition in rail-connected substations or digital power plant substations is the inability to accurately and conveniently identify multiple equipment types simultaneously within a single image. For example, an image of an equipment cabinet typically includes various types of equipment such as circuit breaker indicator lights, a number of circuit breakers and their indicator lights, voltage and ammeters, a number of pressure plates or connectors (sometimes further divided into those in use and those on standby), gear knobs, and LED digital displays. To address this issue, on-site engineers typically set multiple preset positions for each equipment cabinet, capturing images of a specific type of equipment at a biased angle, establishing a correspondence between the preset positions and the corresponding equipment types, and then sequentially performing image recognition on multiple preset positions. This approach significantly increases the number of camera preset positions and the workload for engineers, leading to low project implementation efficiency. Furthermore, some manufacturers' software products cannot even perform batch recognition of the same type of equipment. Such recognition software is more suitable for identifying the status or analog quantities of fixed types of equipment in equipment rooms of subway station substations.

[0007] Furthermore, while employing deep learning-based neural network techniques allows for the use of advanced algorithms, a significant challenge lies in the initial stages of project implementation. This is compounded by a lack of training datasets and the diverse range of devices available, making it difficult to collect mutually exclusive state data or negative sample data for specific devices. Therefore, this approach, particularly in the early stages, is inherently constrained by the limited dataset, hindering its ability to achieve general applicability. However, this technique does have its applicable scenarios. It can serve as an upgrade and optimization method for image recognition when sufficient datasets become available later. For example, it is well-suited for detecting whether personnel in a designated area are wearing safety helmets (judging whether workers are wearing safety helmets and other safety behaviors), where a large test dataset is readily available.

[0008] In summary, to better adapt to the current state of practical engineering, image processing-based methods and deep learning-based methods can be combined for application. However, the main problem this method aims to solve is how to uniformly manage and schedule advanced intelligent recognition algorithms or algorithms suitable for specific devices, allowing for human intervention in adjustments, and applying them to appropriate recognition scenarios. Summary of the Invention

[0009] To address the aforementioned problems, the purpose of this invention is to establish a general and easily expandable intelligent image recognition algorithm management and collaboration framework through an image template-based image modeling and recognition method, thereby achieving unified management and scheduling of image processing-based and deep learning-based artificial intelligence algorithms.

[0010] To achieve the above objectives, the technical solution adopted by this invention is: an image modeling and recognition method based on image templates, comprising the following steps:

[0011] Step 1: Collect image data of various types of equipment in the power supply substation using the data acquisition function.

[0012] The data acquisition function is completed by the camera PTZ management system and the image recognition request client working together. The camera PTZ management system is used to set the preset position of the PTZ camera to be inspected, and the image recognition request client is responsible for calling the camera PTZ management system interface to perform inspections and capture images of the equipment.

[0013] Step 2: Use the equipment image recognition template modeling tool to complete the image recognition template modeling for different types of equipment in the substation.

[0014] The equipment image recognition template modeling tool first annotates the preset position images acquired by the data acquisition function with image recognition auxiliary information, then binds different image recognition algorithms to different identified devices by binding the image recognition algorithm function, and finally exports the substation equipment object image recognition template modeling annotation information file and its corresponding template image.

[0015] The device image recognition template modeling tool has the functions of labeling auxiliary information for the image to be recognized, binding image recognition algorithm, image pre-recognition, saving labeling information and template images, and importing image recognition template modeling labeling information and corresponding template images.

[0016] The device image recognition template modeling tool can perform information annotation, recognition algorithm binding, and parameter setting for multiple identical or different types of devices (such as energy storage status gauges, disconnectors, circuit breakers, air switches, connecting plates, temperature patches, knobs, pressure plates, level gauges, indicator lights, single pointer gauges, dual pointer gauges, and character-type indicator gauges) on the image being recognized.

[0017] The identification auxiliary annotation information includes public annotation information and private annotation information.

[0018] The public annotation information includes annotation coordinates, preset position numbers, data point numbers agreed upon with external systems, and data type information.

[0019] The private labeling information is designed according to the characteristics of different types of equipment. For example, pointer table type data may include: starting range coordinates, center coordinates, starting range, and whether it is mirrored; liquid level equipment includes: maximum scale value and minimum scale value.

[0020] The bound image recognition algorithm function contains a core tuple, denoted as .<deviceclass,algorithm,ways,map> Its meaning includes: device category, algorithm number (the algorithm number corresponds one-to-one with the device subcategory), identification methods included for the same algorithm number, and different value options for device status.

[0021] The value of `deviceclass` serves as the substation equipment category, named according to the actual category, and is also used as the keyword for the entire equipment category configuration item. For example, the indicator light category can be named "light", the circuit breaker category "kongkai", the pointer meter type "meter", the switch knob category "xuanniu", the transformer night position category "yewei", the energy storage equipment category "chuneng", the disconnector equipment category "daozha", and the circuit breaker equipment category "duanluqi".

[0022] The `algorithm` object, as a subcategory of the main device category, assigns a specific algorithm number to each subcategory. It is a configuration object with configuration items in the form of "key:algorithm-number". The key is named according to the actual category, and the `algorithm-number` specifies a unique number. For example, if the disconnect switch device category contains two subcategories, an `algorithm` object corresponding to that category can be defined, in the form of `{"algorithm":{"daozha":2,"gekai":45}`.

[0023] `ways`, representing different identification methods within the same algorithm ID, is itself an array structure. Each element of the array is an object with a value in the form of: "{identification method(key): identification algorithm ID(identify-way-number), name: identification method display name}". `identify-way-number` specifies a unique ID. Assuming an indicator light type device has 4 different identification methods, there are 4 sets of values, in the following format:

[0024] [{"light_DL":0,"name":"Indicator Light (Deep Learning)"},{"light_little1":1,"name":"Light One"},{"light_little2":2,"name":"Light Two"},{"light_color":3,"name":"Color Judgment"},{"light":4,"name":"Light"}]. Selecting the first group during modeling indicates using a deep learning algorithm to identify the on / off state of the indicator light. Other methods use image processing for recognition. Selecting the second-to-last group indicates using image processing to identify the indicator light's color.

[0025] The `algorithm` and `ways` parameters are primarily used to control the granularity and flexibility of the bound recognition algorithm. Generally, different device subcategories correspond to one recognition algorithm. When the recognition requirements are met, `ways` does not need to be further specified. When multiple deep learning or image processing recognition methods can be used for the same device subcategory, the set of optional image recognition algorithms can be further expanded by configuring `ways` to meet the needs of device image recognition at different implementation stages of the project, and to facilitate subsequent algorithm updates and replacements.

[0026] The `map` property primarily transforms the recognition results into understandable annotations on the recognized image. It is an array structure, where each element is an object with a value in the format: "recognition result in numeric form (key): recognition result mapping name (map-name)". Here, `key` represents the image recognition result in numerical form, and `map-name` corresponds to a mapping name with different meanings. Multiple groups can be configured according to actual needs, and one group is selected during modeling as required. Taking the knob category as an example, there are 9 groups of values, in the following format:

[0027] [{"0":"Input","1":"Exit"},{"0":"Distant","1":"Local"},{"0":"Local","1":"Distant"},{"0":"Open","1":"Close"},{"0":"Manual","1":"Local","2":"Distant"},{"0":"Distant","1":"Prohibited","2":"Local"},{"0":"Open (Green on, Red off)","1":"Close (Green off, Red on)","-1":"Abnormal"},{"-1":"Automatic","0":"0","1":"1","2":"2","3":"3","4":"4","5":"5","6":"6"},{"0":"Close (Green off, Red on)","1":"Open (Green on, Red off)","-1":"Abnormal"}].

[0028] When the last group is selected during the modeling process, it means that when the image recognition result is 0, the knob at the corresponding position on the recognized image will be selected and "Close (green off, red on)" will be displayed.

[0029] When binding the image recognition algorithm, the process involves first setting the device category, then setting the algorithm category (device subcategory), and then, if the device to which the recognition algorithm is to be bound has different recognition methods configured, setting one of the recognition methods according to the configuration options. Finally, the recognition result mapping information is set, which completes the mapping setting between the recognition result of the number type and the understandable display content.

[0030] The bound image recognition algorithm function is designed using an object-oriented design method to design the image recognition algorithm interface specification and build an extended algorithm management module. First, an abstract interface for algorithm processing is defined. Then, for each new device type image recognition algorithm (algorithm subclass), the image algorithm interface is implemented to complete its corresponding image recognition algorithm.

[0031] The image recognition algorithm interface specification provides method functions for specific recognition algorithms, including calculating affine transformation matrices, drawing results, extracting JSON format data, cropping images, image matching, image segmentation, coordinate selection, and image recognition execution.

[0032] Different algorithm subclasses inherit from the algorithm interface specification. Different methods within the same algorithm category can be handled using different branch functions or defined with different processing classes, depending on the actual design. This satisfies the open / closed principle and facilitates future algorithm upgrades and replacements.

[0033] After completing information annotation, recognition algorithm binding, and parameter settings for multiple devices to be identified on the image, the image recognition result can be previewed using the image pre-recognition function. Once confirmed to be correct, the currently annotated information can be saved to the annotation result file, and the image can be saved as an image recognition template.

[0034] Step 3: Use image recognition services to perform image recognition on different types of substation equipment and provide feedback on the results.

[0035] The image recognition service can reuse the image recognition algorithm interface specification and its different algorithm subclasses.

[0036] The image recognition service requires authorization to start normally, and then enters a loop to wait for image recognition requests from external clients. Afterwards, if an image recognition request is received, the following processing flow is executed.

[0037] First, the request data is parsed. If it conforms to the agreed-upon parameter request format, the recognized image is loaded.

[0038] Secondly, load the image modeling template. If template information exists, perform feature matching between the image to be identified and the image modeling template.

[0039] Then, based on the information annotated by the image modeling, the recognition algorithms and parameters set for different types of devices in the image are determined, and the corresponding recognition algorithm processing class is obtained according to the algorithm number to identify the device status or device range and reading.

[0040] Finally, the recognition results are returned to the client.

[0041] The recognition results include two formats: image-based and text-based. The image-based results will select different types of devices within the image and label their status, range, and readings above their respective locations. The text-based results primarily include:

[0042] channelID: The camera's channel number, representing a uniquely coded camera device, usually provided by the video manufacturer.

[0043] channelName: Image name, usually set as a unique identifier, following the rule of "installation location_installation position_camera type and preset position number", such as "control room_row wall mount 4#_PTZ camera 46".

[0044] `result` is an array of data objects, where each element of the array is the recognition result data for all different types of devices at a preset position. Its main key items include:

[0045] presetNum: The number of the preset bit;

[0046] presetName: The name of the preset bit;

[0047] The key items used to represent the identification results of different types of devices are in the form of "device category (key): array of identification result information (result-value)".

[0048] The possible values ​​of the identification category (key) are included in the value set of deviceclass in the bound image recognition algorithm function. The specific value depends on the device category in the image modeling annotation information loaded when processing the client request.

[0049] The "result-value" is an array structure. Each element of the array is a device identification result object of the same type, and its main values ​​include:

[0050] objectId: Represents the number of the external system data point object;

[0051] attrId: This is mainly used by third-party systems as the attribute number of data point objects. If the external system does not need it, this item is reserved.

[0052] raw: indicates the original value of the recognition result;

[0053] value: Represents the displayed value of the recognition result, that is, the mapping value corresponding to the original value of the recognition result.

[0054] dataType: Indicates the type of the identified result value. Possible values ​​include: Bool, Int, Float, Long, and String. This is used by the client to decode the data.

[0055] The values ​​of objectId, attrId, and dataType all come from the common annotation information used in image modeling.

[0056] Compared with the prior art, the technical solution of this application has the following advantages:

[0057] 1. This method can customize intelligent image recognition algorithms for different equipment image recognition scenarios based on data samples at different stages of project implementation, and can flexibly adjust the recognition algorithm strategy as the data samples are gradually improved. This is applicable to the numerous different types of equipment in rail-connected power supply substations.

[0058] 2. Based on the aforementioned collaborative framework, this method employs image template modeling and feature annotation, which not only enables flexible concurrent identification and anomaly detection of different types of devices contained in the identification object, facilitating engineering implementation, but also offers higher recognition efficiency and accuracy compared to the method that relies entirely on grayscale processing, edge detection, image information extraction, and other means to first determine the device contour region from numerous images of different types and then match and identify it.

[0059] 3. This method does not need to concern itself with whether the specific recognition algorithm uses image processing technology or deep learning technology. Instead, it focuses on how to uniformly manage specific recognition algorithms through the agreement of algorithm interface specifications and the modeling of image templates, and apply them to recognition scenarios suitable for their application, so as to achieve batch simultaneous recognition and anomaly detection of a wide variety of equipment images at the engineering site. Attached Figure Description

[0060] Figure 1 This is the overall collaborative framework for image modeling and recognition in an embodiment of the present invention.

[0061] Figure 2 This is a flowchart illustrating the image recognition processing of different types of devices in embodiments of the present invention.

[0062] Figure 3 This is a schematic diagram of the global configuration for different device types in an embodiment of the present invention.

[0063] Figure 4 This is an example of an accident scene processing flow according to an embodiment of the present invention. Detailed Implementation

[0064] To enable those skilled in the art to further understand the features and technical content of the present invention, please refer to the following detailed description and accompanying drawings. The drawings are provided for reference and explanation and are not intended to limit the present invention.

[0065] In this embodiment, the system includes an image recognition service program that obtains device information (denoted as "machineInfo"), an image recognition service program that grants authorization (denoted as "genLicense"), an image recognition modeling tool, and an image recognition service program. The image recognition modeling tool is implemented using Python and PyQt, and the image recognition service program is implemented using Python. It handles inspection command requests, image recognition, and result feedback.

[0066] In this embodiment, the overall collaborative framework for image template modeling and recognition is as follows: Figure 1 As shown in the figure. To illustrate the implementation process more concisely, this embodiment selects equipment in the cabinet inside the substation equipment room, taking the simultaneous identification of indicator lights, circuit breakers, single pointer meters, and rotary knobs as an example, and describes the features and technical implementation scheme of the present invention in conjunction with the accompanying drawings.

[0067] Step 1: Authorize the image recognition service program.

[0068] First, on the machine where the image recognition service program needs to run, run "machineInfo" to obtain information about the running machine and generate an encrypted "computer_info.dat" file.

[0069] Then, the "computer_info.dat" file is used as input to the "genLicense" program, which is run on a machine dedicated to licensing to generate an encrypted "License.dat" file.

[0070] Finally, the image recognition service program needs to be started normally based on "License.dat".

[0071] Step 2: Collect image data of various types of equipment in the power supply substation using the data acquisition function.

[0072] The data acquisition function is completed by the camera PTZ management system and the image recognition request client working together. The camera PTZ management system is used to set the preset positions of the PTZ cameras to be inspected, and the image recognition request client is responsible for calling the camera PTZ management system interface to perform inspections and capture equipment image data.

[0073] Step 1: Select two PTZ cameras and assign a preset position to each. The naming convention for the preset positions is "Camera Installation Area_Based on Location_Camera Name + Preset Position Number". For ease of description in this embodiment, the preset positions of the two PTZ cameras are designated as "Preset Position One" and "Preset Position Two", respectively, and are assigned to the indicator lights and circuit breakers in "Cabinet One", and the knobs and voltage / current meters in "Cabinet Two", respectively.

[0074] Step 2: Using PTZ control, set the preset position of the first PTZ camera to "Preset Position One" and capture an image, saving it as an image. The naming rule is: based on the preset position naming rule, add the shooting time and channel number information. For example, the actual name of a captured image is: "Control Room_Wall Mounted 3#_PTZ Camera 02_20221222131946_003.jpg". For ease of description in this embodiment, the image captured by "Preset Position One" in this step will be denoted as "PresetOne_Image". Then, control the second PTZ camera to set its preset position to "Preset Position Two" and capture an image, denoting the captured image as "PresetTwo_Image".

[0075] Step 3: Use the equipment image recognition template modeling tool to complete the image recognition template modeling for different types of equipment in the substation.

[0076] Step 1 sets up global configuration information for the modeling tool for different types of devices. This mainly includes settings for key information such as device category, algorithm number, recognition methods included for the same algorithm number, and different value options for device status. A schematic diagram of the global configuration for different device types involved in this embodiment is shown below. Figure 3 As shown. This configuration information only needs to be set once, and can be flexibly expanded or updated according to the new device types and algorithms.

[0077] Step 2: Load "PresetOne_Image" and "PresetTwo_Image" into the image recognition modeling tool. A schematic diagram of the image template modeling tool in this embodiment is shown below. Figure 4 As shown.

[0078] Step 3 involves feature annotation and recognition algorithm binding for the device being identified in the image.

[0079] Step 3_1: First, feature information is labeled and recognition algorithms are bound to the indicator light type devices in the "PresetOne_Image". Then, feature information is labeled and recognition algorithms are bound to the circuit breaker type devices.

[0080] Step 3_1_1 The process of marking feature information and binding recognition algorithms for indicator light type devices is as follows.

[0081] 1. In the operation bar on the right side of the modeling tool, select "Indicator Light" from the "Major Category" (i.e., equipment category) selection box, and select the corresponding sub-category (i.e., equipment sub-category) according to the type of indicator light in the actual image. If there are no multiple recognition methods for the corresponding equipment type, then select the recognition algorithm number corresponding to the equipment sub-category.

[0082] 2. In the “Method” (i.e., recognition algorithm number) selection box, select the “Indicator Light-DL” option, which means that in this embodiment, the open and closed state of the indicator light device in the image is recognized based on the deep learning intelligent recognition algorithm; at the same time, in the “Mapping” selection box, set the recognition result with display significance.

[0083] 3. Click the "Annotate" operation, then use the mouse to select the area around the indicator light in the preset image bar on the left. After that, confirm the annotation information and display the annotated information in the "Annotation Results" table.

[0084] 4. After the “Annotation” operation, set auxiliary parameter information in the pop-up information box. For indicator light devices, you only need to enter the device ID.

[0085] Step 3_1_2, the process of labeling feature information and binding recognition algorithms for circuit breaker type devices is similar, as detailed below.

[0086] 1. In the operation bar on the right side of the modeling tool, select "Break Switch" in the "Major Category" (i.e., equipment category) selection box and "Break Switch" in the "Sub-category" selection box.

[0087] 2. In the “Method” selection box, select the “Break-DL” option, which means that in this embodiment, the open / closed state of the circuit breaker device in the image is identified based on the deep learning intelligent recognition algorithm; at the same time, in the “Mapping” selection box, set the recognition result with display significance.

[0088] 3. Click “Annotation”, use the mouse to select the area around the circuit breaker in the preset position image bar on the left, then confirm the annotation information and display the annotated information in the “Annotation Results” table.

[0089] 4. After the “label” operation, enter the device ID in the pop-up information box and click “OK”.

[0090] Step 3_2: First, feature information is labeled and recognition algorithms are bound to the voltage and current meter type devices in the "PresetTwo_Image" image. Then, feature information is labeled and recognition algorithms are bound to the knob type devices within it.

[0091] Step 3_2_1 The process of marking feature information and binding recognition algorithms for voltage and current meter type equipment is as follows.

[0092] 1. In this embodiment, the identification of pointer tables is based on image processing. Different identification algorithms are used for different types of single and double pointer tables. In the operation bar on the right side of the modeling tool, select "Pointer Table" in the category selection box, and then select the corresponding subcategory according to the type of single pointer table in the image.

[0093] 2. Select the corresponding mapping relationship according to the marked positions in the actual image;

[0094] 3. Click “Annotate”, and use the mouse to click in the preset position image column on the left to locate the center of the single pointer meter and the starting range point;

[0095] 4. Enter the device ID and start and end range values ​​in the pop-up information box.

[0096] Step 3_2_2 The process of feature information annotation and recognition algorithm binding for knob-type devices is similar, as detailed below.

[0097] 1. In this embodiment, the knob recognition is based on image processing. Different recognition algorithms are used for knobs with different gear positions and shapes. In the operation selection bar on the right, select "Knob" as the main category, and then select the corresponding subcategory according to the knob type in the actual image.

[0098] 2. Select the corresponding mapping relationship according to the marked positions in the actual image;

[0099] 3. Click “Annotation”, use the mouse to select the area around the knob in the preset image panel on the left, and confirm.

[0100] Step 4: Image pre-recognition. Click the "Recognize" button to recognize all the labeled devices on the image. This operation can be used to check whether the modeling and recognition are accurate. If there are individual recognition problems, the labeling information can be readjusted.

[0101] Step 5: After verifying the accuracy of the image pre-recognition, click "Save Template Image" and "Save Annotation Information" on the right side of the interface.

[0102] Step 6: Distribute the image template and feature annotation information files to the "templateimg" and "configs" directories of the image recognition service program.

[0103] Step 4: Use image recognition services to complete image recognition of different types of substation equipment and provide feedback on the results.

[0104] The overall processing flow of the image recognition service program in this embodiment is as follows: Figure 2 As shown.

[0105] Step 1: The image recognition simulation request program simulates a timed inspection function. First, it invokes the data acquisition function to sequentially capture images from the preset positions set for "Control Room_Wall-mounted 3#_PTZ Camera" and saves them to a specified path for loading by the image recognition service program. Then, it initiates an image recognition request to the image recognition service program. The request parameters include the image name (containing camera and preset position encoding information), the query period, and the path where the recognized image is saved.

[0106] Step 2 describes an image recognition service program that starts an image recognition worker thread for each request to perform image recognition and then returns the processing results.

[0107] Step 2_1: Load all preset images that need to be identified within the requested time period under the corresponding path according to the request parameters.

[0108] Step 2_2 parses the image name information and retrieves the corresponding preset image template and annotation information created by the image modeling tool from the "templateimg" and "configs" directories.

[0109] Step 2_3: Based on the parsed annotation information, obtain the object to be identified and call the corresponding algorithm according to the bound algorithm code to identify the device status or reading.

[0110] The recognition algorithm used in this embodiment mainly includes two processes. First, feature matching is performed between the image to be recognized and the template image. Then, specific recognition processing is performed on the device to be recognized.

[0111] Step 2_4 involves directly labeling the recognition results onto the image and saving them, while simultaneously generating agreed-upon JSON format text data, and then returning all the results to the client.

[0112] Faced with the challenges of numerous different types of equipment, various camera shooting angles, and environmental interference factors in rail-connected substations, and considering the limited number of data samples from each substation during the project implementation phase, this embodiment constructs a flexible and easily expandable intelligent image recognition algorithm management and collaboration framework to ensure relatively accurate detection results even in various non-ideal scenarios. Image template modeling enables unified management of image processing-based and deep learning-based artificial intelligence algorithms. The image recognition service uses modeling information to uniformly schedule recognition algorithms and perform image recognition. This not only effectively addresses the issue of varying data sample quantities for different types of equipment in substations at different implementation stages of the actual project, providing adaptive recognition algorithm strategies, but also allows for easy and quick maintenance and iterative optimization of the recognition algorithm as the data sample size gradually increases. Furthermore, it effectively enables concurrent recognition of the status and readings of different types of equipment within the target object, as well as the identification of abnormal scenarios.

[0113] The above embodiments are merely illustrative of the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solution based on the technical concept proposed in this invention shall fall within the scope of protection of the claims of this invention. Technologies not covered in this invention can be implemented using existing technologies.

Claims

1. An image modeling and recognition method based on image templates, characterized in that, Includes the following steps: Step 1: Collect image data of different types of equipment in the power supply substation using the data acquisition function; Step 2: Use the equipment image recognition template modeling tool to create image recognition templates for different types of equipment in the substation; Step 3: Use image recognition services to perform image recognition on different types of substation equipment and provide feedback on the results; The device image recognition template modeling tool in step two has the functions of marking recognition auxiliary information for the image to be recognized, binding image recognition algorithms, image pre-recognition, saving marked information and template images, and importing image recognition template modeling marked information and corresponding template images. It can mark information, bind recognition algorithms and set parameters for multiple identical or different types of devices on the image to be recognized. In step two, the equipment image recognition template modeling tool first annotates the preset position image acquired by the data acquisition function with image recognition auxiliary information, then binds different image recognition algorithms to different identified devices by binding the image recognition algorithm function, and finally exports the substation equipment object image recognition template modeling annotation information file and its corresponding template image. The identification assistance information includes public annotation information and private annotation information; the public annotation information includes annotation coordinates, preset position numbers, data point numbers agreed upon by the external system, and data type information; the private annotation information is designed according to the characteristics of different types of devices. The bound image recognition algorithm function contains a core tuple, denoted as<deviceclass,algorithm,ways,map> Its meaning is: device category, algorithm number, identification methods included in the same algorithm number, and different value options for device status; When binding the image recognition algorithm, the device category is first set, then the algorithm category is set. If the device to which the recognition algorithm is to be bound has different recognition methods, then one of the recognition methods is set according to the configuration options. Finally, the recognition result mapping information is set, that is, the mapping setting between the recognition result of the number type and the understandable display content is completed. The value of deviceclass is used as the substation equipment category, and is named according to the actual category. It also serves as the keyword for the entire equipment category configuration item. The `algorithm` and `ways` are used to control the granularity and flexibility of the bound recognition algorithm. Different device subcategories correspond to one recognition algorithm. When the recognition requirements are met, `ways` is not further specified. When the same device subcategory uses multiple deep learning or image processing recognition methods, the set of optional image recognition algorithms can be expanded by configuring `ways`. The algorithm, as a subcategory of the major category of devices, corresponds to a specific algorithm number for each subcategory. It is itself a configuration object, and its configuration items take the form of "key: algorithm number". Here, the key is named according to the actual category, and the algorithm number corresponds to a unique number. The `ways` array, which contains different recognition methods for the same algorithm ID, is an array structure. Each element of the array is an object with a value in the form of "{recognition method key: recognition algorithm ID identify-way-number, name: recognition method display name}". The `identify-way-number` specifies a unique ID. The map transforms the recognition results into understandable content and labels them on the recognized image. It is an array structure, and each element of the array is an object with a value in the form of "recognition result key in numeric form: recognition result mapping name map-name". Here, key is the image recognition result represented by numbers, and map-name corresponds to a mapping name with different meanings. Multiple groups can be configured according to actual needs, and one group can be selected as needed during modeling.

2. The image modeling and recognition method based on image templates according to claim 1, characterized in that, The data acquisition function in step one is completed by the camera PTZ management system and the image recognition request client working together; the camera PTZ management system is used to set the preset position of the PTZ camera to be inspected, and the image recognition request client is responsible for calling the camera PTZ management system interface to perform inspection and capture images of the equipment.

3. The image modeling and recognition method based on image templates according to claim 1, characterized in that: The bound image recognition algorithm function is designed using an object-oriented design method to design the image recognition algorithm interface specification and build an extended algorithm management module. First, an abstract interface for algorithm processing is defined. Then, for each new device type, the image recognition algorithm implements the image algorithm interface to complete its corresponding image recognition algorithm.

4. The image modeling and recognition method based on image templates according to claim 1, characterized in that: In step three, the image recognition service can reuse the image recognition algorithm interface specification and its different algorithm subclasses; the image recognition service requires authorization to start normally, and then enters a loop to wait for image recognition requests from external clients; after that, if an image recognition request is received, the following processing flow is performed: First, the request data is parsed. If it conforms to the agreed-upon parameter request format, the recognized image is loaded. Secondly, load the image modeling template. If template information exists, perform feature matching between the image to be identified and the image modeling template. Then, based on the information annotated by the image modeling, the recognition algorithms and parameters set for different types of devices in the image are determined, and the corresponding recognition algorithm processing class is obtained according to the algorithm number to identify the device status or device range and reading. Finally, the recognition results are returned to the client.

5. The image modeling and recognition method based on image templates according to claim 4, characterized in that: The recognition results include two forms: one is the recognition result in image format, and the other is the recognition result in text format. The image format result will select different types of devices in the recognized image and mark the status, range, and reading of the device above the device's location. The recognition results of the text format include: channelID: The camera's channel number, representing a uniquely coded camera device, provided by the video manufacturer; channelName: Image name, set as a unique identifier, following the rule "installation location_installation position_camera type and preset position number"; result: is an array of data objects, where each element of the array is the recognition result data of all different types of devices on a preset bit.

Images