Power cabinet inspection method and device based on machine vision and inspection robot

Abstract

The application provides a kind of power cabinet inspection method, device and inspection robot based on machine vision, it is related to power inspection technical field.The method comprises: obtaining the image to be identified of power cabinet, input to target detection model, detect the equipment information and equipment frame of each instrument and equipment;The image to be identified is input to optical character recognition model, detects text content and text frame;According to the positional relationship of each text frame, each text frame is merged and processed;According to the positional relationship of equipment frame and text frame, equipment frame and text frame are preliminarily matched, global optimal matching is carried out through global matching algorithm, and the unique corresponding relationship of equipment frame and text frame is obtained;The equipment frame and text frame with unique corresponding relationship are subjected to semantic verification to identify the instrument and equipment and label related to semantics.The application can solve the problem that the existing inspection robot automatic inspection process is highly dependent on off-line naming and calibration of inspection equipment.

Description

Technical Field

[0001] This invention relates to the field of power inspection technology, and in particular to a method, device and inspection robot for power cabinet inspection based on machine vision. Background Technology

[0002] Electrical cabinets typically integrate various primary and secondary equipment, including switchboards, indicator lights, circuit breakers, and air switches. To achieve standardized management and accurate status identification, all types of equipment must be equipped with physical identification tags containing key information such as equipment name, model, and unique serial number. During routine inspections, maintenance personnel must use these tags to check and record the equipment's operating status, model configuration, and connections to ensure the safe and stable operation of the power system.

[0003] With the development of inspection robot technology, it has been gradually applied to the inspection of large-scale equipment clusters such as power equipment rooms, effectively reducing the workload of manual inspection. However, the current operation mode of inspection robots still has significant technical bottlenecks: the prerequisite for robots to perform automated inspection, verification and recording tasks is that each inspection device must be named and calibrated offline. When carrying out inspections or asset inventory work on large-scale equipment clusters, the current operation mode still requires maintenance personnel to manually photograph, verify and enter information on each equipment tag, which generally suffers from low operation efficiency, high manual labor intensity and easy error in data recording, seriously restricting the realization of full automation of the inspection process. Summary of the Invention

[0004] This invention provides a machine vision-based method, device, and inspection robot for power cabinet inspection, which solves the problem that existing automatic inspection robot processes heavily rely on offline naming and calibration of inspection equipment.

[0005] In a first aspect, embodiments of the present invention provide a machine vision-based method for inspecting power cabinets, including:

[0006] Acquire the image of the power cabinet to be identified, which contains multiple instruments and equipment and their corresponding labels;

[0007] The image to be identified is input into a preset target detection model to detect the equipment information and equipment frames of each instrument and device in the image to be identified;

[0008] The image to be recognized is input into a preset optical character recognition model to detect the text content and text boxes in the image;

[0009] Merge the text boxes according to their positional relationship;

[0010] Based on the positional relationship between the device frame and the text frame, a preliminary matching is performed on the device frame and the text frame. Based on the preliminary matching results, a global optimal matching is performed through a global matching algorithm to obtain a unique correspondence between the device frame and the text frame.

[0011] For device boxes and text boxes with unique correspondences, semantic verification is performed based on device information and text content to identify semantically related instruments, equipment and labels.

[0012] In one possible implementation, the text boxes are merged based on their positional relationships, including:

[0013] For at least two text boxes of the same size that are on the same horizontal line, calculate the spacing between the text boxes and determine a dynamic spacing threshold based on the size of the text boxes.

[0014] Merge text boxes on the horizontal line whose spacing is less than the dynamic spacing threshold.

[0015] In one possible implementation, merging the text boxes based on their positional relationships also includes:

[0016] Calculate the vertical distance and horizontal overlap between each text box;

[0017] If the vertical distance between any two text boxes is less than a preset vertical distance threshold and the horizontal overlap is greater than a preset horizontal overlap threshold, then the two text boxes are merged.

[0018] In one possible implementation, preliminary matching of the device box and the text box is performed based on their positional relationship, including:

[0019] For any device frame, determine the text boxes within a preset range centered on the device frame, calculate the intersection-union ratio (IUU) between the device frame and each text box within the preset range, and select the first text box corresponding to the device frame from each text box based on the IUU.

[0020] Based on the degree to which each first text box deviates from the central axis of the device frame, the second text box corresponding to the device frame is selected from each first text box;

[0021] Based on the number of device boxes matched by each second text box, filter the third text boxes corresponding to each device box to obtain the preliminary matching results between the device box and the text boxes.

[0022] In one possible implementation, filtering the first text box corresponding to the device frame from each text box according to the intersection-union ratio (IU) includes: determining the text boxes with IU ratios less than a preset IU threshold as the first text box corresponding to the device frame.

[0023] In one possible implementation, the second text box corresponding to the device frame is selected from the first text boxes based on the degree to which each first text box deviates from the central axis of the device frame, including:

[0024] The first text box whose deviation from the horizontal or vertical center axis of the device frame is less than a preset deviation threshold is identified as the second text box corresponding to the device frame.

[0025] In one possible implementation, the third text box corresponding to each device box is filtered based on the number of device boxes matched by each second text box, including: determining the second text box that only matches the device box as the third text box corresponding to the device box.

[0026] In one possible implementation, semantic verification is performed based on device information and text content to identify semantically relevant instruments, devices, and tags, including:

[0027] The device information and text content are input into a preset semantic verification network to obtain the semantic relevance between the device information and the text content;

[0028] If the semantic relevance is greater than the preset semantic relevance threshold, the corresponding device information and text content will be identified as semantically related instruments and labels.

[0029] Secondly, embodiments of the present invention provide a machine vision-based power cabinet inspection device, comprising:

[0030] The acquisition module is used to acquire the image of the power cabinet to be identified, which contains multiple instruments and equipment and their corresponding labels.

[0031] The equipment detection module is used to input the image to be identified into the preset target detection model and detect the equipment information and equipment frame of each instrument in the image to be identified;

[0032] The text detection module is used to input the image to be recognized into a preset optical character recognition model to detect the text content and text boxes in the image to be recognized;

[0033] The text preprocessing module is used to merge the text boxes according to their positional relationship.

[0034] The matching module is used to perform preliminary matching of the device box and the text box based on their positional relationship, and then perform global optimal matching based on the preliminary matching results through a global matching algorithm to obtain a unique correspondence between the device box and the text box.

[0035] The verification module is used to perform semantic verification on device boxes and text boxes that have a unique correspondence, based on device information and text content, in order to filter semantically related instruments, equipment and labels.

[0036] Thirdly, embodiments of the present invention provide an inspection robot, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described in the first aspect or any possible implementation of the first aspect.

[0037] The beneficial effects of the embodiments of the present invention compared with the prior art are as follows:

[0038] In this embodiment of the invention, the target detection model can accurately locate each instrument and device in the image to be identified of the power cabinet and extract the device information and device frame. The optical character recognition model simultaneously captures the label text content and text frame. Furthermore, to address issues such as irregular line spacing, character spacing, and scattered multi-character fragments that may exist in industrial labels, the text frame position relationship merging process can integrate multiple text frames split from the same label into a complete label unit, effectively avoiding the problems of incomplete recognition and semantic breaks caused by label text fragmentation, and ensuring the integrity and coherence of label text information. Moreover, through the dual matching mechanism of "preliminary position matching + global optimal matching", the precise and unique correspondence between the device and the label is achieved, significantly improving the accuracy and stability of the matching. Finally, the semantic verification stage forms a closed-loop verification of the recognition and matching, further filtering matching errors. This invention, through a complete process design of extraction, merging, matching, and verification, achieves automation, high precision, and high stability in equipment and tag identification and matching in industrial scenarios. It significantly reduces the cost of manual intervention, improves the efficiency of inspection, asset inventory, and other tasks, and effectively adapts to the unique needs of industrial sites with dense tags, complex backgrounds, uneven lighting, and irregular text layout. It has strong engineering practicality and scenario adaptability. Attached Figure Description

[0039] Figure 1 This is a flowchart illustrating the machine vision-based power cabinet inspection method provided in an embodiment of the present invention.

[0040] Figure 2 This is a matching illustration provided in the embodiments of the present invention. Figure 1 ;

[0041] Figure 3 This is a matching illustration provided in the embodiments of the present invention. Figure 2 ;

[0042] Figure 4 This is a matching illustration provided in the embodiments of the present invention. Figure 3 ;

[0043] Figure 5 This is a schematic diagram of the structure of the power cabinet inspection device based on machine vision provided in an embodiment of the present invention;

[0044] Figure 6 This is a schematic diagram of the inspection robot provided in an embodiment of the present invention. Detailed Implementation

[0045] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of the invention. However, those skilled in the art will understand that the invention can be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of the invention with unnecessary detail.

[0046] To make the objectives, technical solutions, and advantages of the present invention clearer, specific embodiments will be described below in conjunction with the accompanying drawings.

[0047] While image-based text recognition technology has made significant progress, existing methods are primarily applied to objects with fixed layouts and simple background features, such as certificates, documents, and license plates. However, industrial equipment labels present unique challenges, including dense label arrangement, complex background environments, irregular line or character spacing, and uneven lighting conditions. These challenges significantly increase the difficulty of accurately matching the recognized label text with the corresponding equipment.

[0048] See Figure 1 The document illustrates a flowchart of the implementation of a machine vision-based power cabinet inspection method provided in an embodiment of the present invention, detailed below:

[0049] Step S101: Obtain the image of the power cabinet to be identified.

[0050] In this embodiment, an imaging device mounted on an inspection robot is used to acquire images. The imaging device can be a high-definition camera, which can be fixed to the inspection robot body and acquire one or more images covering the target inspection area according to the actual inspection needs and the site environment.

[0051] The acquired images must meet core content requirements, namely, clearly showing multiple instruments and equipment to be identified, as well as labels corresponding to each device and installed on the device panel. Label information typically includes key identifiers such as device name, model, and serial number. To optimize the accuracy and efficiency of subsequent target detection and character recognition processing, a frontal vertical perspective is prioritized during acquisition to ensure neat and detailed imaging of the equipment and labels. Furthermore, this method offers flexibility in scene adaptation; even with slight deviations in the shooting angle, subsequent algorithms can handle the imaging effects of angle changes, eliminating the need for strict limitations on shooting posture and improving the convenience of on-site inspections.

[0052] Step S102: Input the image to be identified into the preset target detection model to detect the equipment information and equipment frames of each instrument in the image to be identified.

[0053] The collected raw images are input into a pre-trained object detection model for processing. Commonly used models include classic algorithms based on deep learning frameworks, such as the YOLO series. To ensure the model can adapt to the complex and ever-changing environment of industrial sites, its training dataset is constructed using a large number of real industrial scene images. Each image is labeled with the actual bounding box of the equipment, covering different equipment models, sizes, installation layouts, and various background interference scenarios (such as intersecting lines, panel textures, and changes in lighting). Through sufficient sample training, the model's feature extraction and generalization capabilities for complex scenes are strengthened, effectively avoiding false detections and missed detections caused by background interference.

[0054] After the model completes inference processing on the input image, it outputs the relevant information of the detected instruments and equipment, the precise bounding box coordinates of each device in the image coordinate system, and the corresponding detection confidence (used to evaluate the reliability of the detection results and provide support for subsequent screening of high-quality detection results).

[0055] Step S103: Input the image to be recognized into a preset optical character recognition model to detect the text content and text boxes in the image to be recognized.

[0056] The image to be processed is input into an optical character recognition (OCR) model to complete text extraction and recognition. Considering the real-time and integrated requirements of label recognition in industrial scenarios, an end-to-end optical character recognition (OCR) model based on deep learning is preferred. Typical solutions include convolutional recurrent neural networks and differential binarization networks. Such models can break through the traditional OCR step-by-step mode of "localization first, recognition later" and achieve integrated completion of text region localization and character content recognition, significantly reducing error propagation between steps and improving overall processing efficiency.

[0057] After the model inference is completed, the output results cover the core information corresponding to all the identified text lines in the image, specifically the bounding box coordinates of each text line (precisely outlining the position range of the text in the image, consistent with the bounding box coordinate system of the device mentioned earlier, facilitating subsequent position association analysis), and the recognized content of the corresponding text line. Crucially, the OCR model used in this invention is not a general-purpose model, but rather has been specifically trained and optimized for the unique recognition needs of industrial environments: on the one hand, it focuses on adapting to industrial-specific fonts, including dot matrix fonts and special engineering fonts, addressing the low recognition accuracy of such fonts in conventional models; on the other hand, it optimizes the shallow feature extraction structure of the model for small characters commonly found in industrial labels, enhancing the ability to capture details of tiny characters. Simultaneously, it improves the model's anti-interference capabilities by considering the uneven lighting and cluttered backgrounds characteristic of industrial scenes, ultimately significantly enhancing its text recognition accuracy and stability in complex industrial environments.

[0058] Step S104: Merge the text boxes according to their positional relationships.

[0059] In one possible implementation, for at least two text boxes of the same size on the same horizontal line, the spacing between the text boxes is calculated, and a dynamic spacing threshold is determined based on the size of the text boxes; text boxes on the horizontal line with a spacing smaller than the dynamic spacing threshold are merged.

[0060] To address scenarios where character spacing is abnormal in single-line text due to printing quality and layout requirements (e.g., when the label content has few characters, justified layout is used, resulting in gaps between characters that are much larger than normal character spacing, leading to OCR models recognizing them as multiple independent text boxes), this embodiment optimizes the merging effect through a character-level spatial clustering algorithm. Specifically, the bounding boxes corresponding to each character are first extracted, and the horizontal distance between adjacent character bounding boxes is calculated. Simultaneously, a dynamic spacing threshold positively correlated with the average character size is set. This threshold can adaptively adjust with character size, avoiding insufficient adaptation of a fixed threshold for text of different sizes. Based on this dynamic threshold, even if the distance between characters is too great, as long as they are on the same horizontal baseline, they can still be considered part of the same text paragraph. Finally, according to the horizontal coordinate order of each character bounding box, the text content is spliced ​​and the text boxes are merged from left to right, restoring the label text.

[0061] In one possible implementation, the vertical distance and horizontal overlap between each text box are calculated; if the vertical distance between any two text boxes is less than a preset vertical distance threshold and the horizontal overlap is greater than a preset horizontal overlap threshold, then the two text boxes are merged.

[0062] In this embodiment, if the vertical distance between two text boxes is less than a preset vertical distance threshold and the horizontal overlap is greater than a preset horizontal overlap threshold, they are determined to be adjacent text lines of the same multi-line label and are merged into a single text box.

[0063] By employing the aforementioned text box merging strategy, this embodiment can effectively address complex situations such as abnormal text spacing and multi-line distribution in industrial labels, maximizing the restoration of the original text information of the label, avoiding recognition and matching errors caused by text fragmentation, and improving the stability and adaptability of the overall solution.

[0064] Step S105: Based on the positional relationship between the device frame and the text frame, perform preliminary matching on the device frame and the text frame, and based on the preliminary matching results, perform global optimal matching through a global matching algorithm to obtain a unique correspondence between the device frame and the text frame.

[0065] This embodiment first performs preliminary matching based on location relationships. The core purpose is to narrow down the matching range and eliminate irrelevant interference through spatial location correlation filtering, thereby reducing the computational cost for global matching. This step takes the device frames obtained from target detection, OCR processing, and merged text frames as the processing objects. For each device frame, a preset spatial range centered on it is first defined (adapting to the conventional installation distance between equipment and labels in industrial scenarios), and potential text frames within this range are filtered out. Then, the spatial overlap is quantified by calculating the intersection-union ratio (IUGR) between the device frame and each potential text frame. Text frames with IUGR below a set threshold are retained as candidate text frames, forming a preliminary matching result of "single device frame - multiple candidate text frames".

[0066] Then, the core of finding the optimal solution based on the global matching algorithm is to achieve a unique mapping between all device frames and text frames from a holistic perspective, building upon the initial matching results. Here, a global matching algorithm (such as the Hungarian algorithm or the KM algorithm) is introduced to transform the matching problem into a globally optimal solution problem. The global matching algorithm can perform a global, comprehensive calculation of the association between all device frames and candidate text frames, ultimately outputting a set of conflict-free and complete unique correspondences. That is, each device frame corresponds to only one text frame, and each text frame belongs to only one device frame, completely solving the limitations of local matching and ensuring accurate binding between devices and their corresponding labels in complex industrial scenarios.

[0067] Step S106: For device boxes and text boxes with unique correspondences, perform semantic verification based on device information and text content to identify semantically related instruments, equipment and labels.

[0068] For each pair of device frames and text frames with a unique correspondence, core information is extracted: device type information (such as specific device categories like pressure plate switches, indicator lights, and circuit breakers) is extracted from the output of the object detection model, and label text content is extracted from the OCR recognition and merging results. Subsequently, the extracted device type information and label text content are input into a pre-trained semantic verification network, which calculates the semantic correlation between the two through feature modeling and association analysis, and quantifies the reasonableness of the matching between the device and the label at the content level.

[0069] To achieve accurate screening, a semantic relevance threshold needs to be preset (this threshold can be determined by debugging based on the semantic association patterns between industrial equipment and tags, combined with a large amount of sample data, and can be flexibly fine-tuned according to actual scenarios). If the semantic relevance of a certain group of equipment and tags is greater than the preset threshold, it is determined that there is a reasonable semantic association between the two, the group of correspondences is confirmed as valid, and it is retained as the final matching result; if the semantic relevance is less than or equal to the preset threshold, it is determined as an invalid match, and the group of correspondences is removed.

[0070] In this embodiment of the invention, the target detection model can accurately locate each instrument and device in the image to be identified and extract device information and device frames. The optical character recognition model simultaneously captures the label text content and text frames. Furthermore, to address issues such as irregular line spacing, character spacing, and scattered multi-character fragments that may exist in industrial labels, the text frame position relationship merging process can integrate multiple text frames split from the same label into a complete label unit, effectively avoiding the problems of incomplete recognition and semantic breaks caused by label text fragmentation, and ensuring the integrity and coherence of label text information. Moreover, through the dual matching mechanism of "preliminary position matching + global optimal matching", a precise and unique correspondence between the device and the label is achieved, significantly improving the accuracy and stability of the matching. Finally, the semantic verification stage forms a closed-loop verification of the recognition and matching, further filtering out matching errors. This invention, through a complete process design of extraction, merging, matching, and verification, achieves automation, high precision, and high stability in equipment and tag identification and matching in industrial scenarios. It significantly reduces the cost of manual intervention, improves the efficiency of inspection, asset inventory, and other tasks, and effectively adapts to the unique needs of industrial sites with dense tags, complex backgrounds, uneven lighting, and irregular text layout. It has strong engineering practicality and scenario adaptability.

[0071] In one possible implementation, the detailed implementation process of step S105 includes:

[0072] Step S1051: For any device frame, determine the text boxes within a preset range centered on the device frame, calculate the intersection-union ratio (IUU) between the device frame and each text box within the preset range, and select the first text box corresponding to the device frame from each text box based on the IUU.

[0073] This step is the core of the initial matching between the device and the label text. Its purpose is to narrow down the matching range based on spatial location relationships and filter out the candidate text boxes (i.e., the first text box) that are most relevant to the target device frame. This reduces the computational complexity for subsequent global optimal matching and also reduces interference from irrelevant text boxes. The specific logic and details are explained below:

[0074] First, for each device bounding box output by the object detection model mentioned above, a process is performed: a preset spatial range is defined with the geometric center of the device bounding box as the origin (this range can be set according to the conventional installation distance between the device and its corresponding label in an industrial scenario, such as the common distribution radius of labels based on a large amount of field data statistics, and also supports flexible fine-tuning according to different device types and cabinet layouts, ensuring that the label corresponding to the device is covered while avoiding the inclusion of too many irrelevant text boxes). Then, all text boxes recognized and merged by OCR are traversed, and text boxes that fall completely or partially within this preset range are selected as the potential matching text box set for that device bounding box.

[0075] Next, the intersection-union ratio (IUR) of the device frame with each text box in the set of potential matching text boxes is calculated. The core calculation logic of the IUR is the ratio of the "area of ​​the overlapping region between the two frames" to the "area of ​​the merged region between the two frames", and the result ranges from 0 to 1. Usually, labels are not placed on the device, that is, the device frame and the text box usually do not intersect. Therefore, the text box with an IUR less than the preset IUR threshold is identified as the first text box corresponding to the device frame.

[0076] This step, through the dual logic of "range filtering + intersection-union ratio quantization", achieves the initial association between the device and the text box. It not only uses spatial location characteristics to exclude a large number of irrelevant text boxes, but also uses intersection-union ratio quantization to determine the matching priority. This lays a precise candidate set foundation for the subsequent global matching algorithm to efficiently solve for the optimal correspondence, effectively improving the efficiency and reliability of the overall matching process.

[0077] Step S1052: Based on the degree to which each first text box deviates from the central axis of the device frame, filter the second text box corresponding to the device frame from each first text box.

[0078] like Figure 2 As shown, to further eliminate interference from irrelevant text boxes and strengthen the positional correlation between devices and labels, after obtaining the first text box based on intersection-union ratio (IU), a new axis range determination step is added. This step combines the typical installation layout of labels in industrial scenarios (e.g., labels are usually located directly below or to the side of the corresponding device) to perform a secondary screening of the first text box. Specifically, using the vertical and horizontal center axes of the target device frame as references, preset allowable offset ranges are defined, and it is determined whether the centerline of each first text box falls within these allowable ranges.

[0079] This step accurately filters out text boxes that are too far from the device frame or whose positional relationship is obviously unreasonable, significantly reducing the probability of mismatches in subsequent matching. Finally, the first text box whose deviation from the horizontal or vertical center axis of the device frame is less than a preset offset threshold is identified as the second text box corresponding to that device frame. This provides a more accurate and relevant set of candidate text boxes for the subsequent global matching algorithm, further improving the reliability of the overall matching process.

[0080] Step S1053: Based on the number of device frames matched by each second text box, filter the third text boxes corresponding to the device frames to obtain the preliminary matching results between the device frames and the text boxes.

[0081] After obtaining the second text box through the above two-stage screening, a special "coverage check" needs to be carried out on the candidate label boxes. In actual industrial inspections, due to factors such as improper label installation (e.g., labeling across equipment) and image perspective distortion, the spatial range of a single label box may cover two or more equipment boxes at the same time. If such labels participate in subsequent matching, it is very easy to cause the risk of mismatch between multiple devices.

[0082] Therefore, this invention performs coverage verification on each candidate second text box individually: from the vertical direction (e.g.) Figure 3 (as shown) and horizontal direction (as shown) Figure 4 As shown in the diagram, it is determined whether the label box simultaneously covers multiple device boxes. If a label box is found to have cross-device coverage in any direction, it is marked as an invalid candidate label box and directly excluded from the candidate set of that device box to avoid interfering with subsequent matching processes. After excluding invalid candidates, only the second text box that does not have cross-device coverage and meets the secondary filtering conditions is retained, and it is identified as the third text box corresponding to that device box.

[0083] The method proposed in this application can automatically process a picture of a power cabinet and complete the identification and accurate association of dozens of devices and their tags within seconds, with an accuracy rate of over 95%. This greatly improves inspection efficiency and data accuracy, and reduces labor costs and error rates.

[0084] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0085] The following are device embodiments of the present invention. For details not described in detail, please refer to the corresponding method embodiments described above.

[0086] Figure 5 The diagram shows a schematic of the structure of a machine vision-based power cabinet inspection device provided in an embodiment of the present invention. For ease of explanation, only the parts related to the embodiment of the present invention are shown.

[0087] like Figure 5 As shown, the machine vision-based power cabinet inspection device 5 includes:

[0088] The acquisition module 51 is used to acquire the image to be identified of the power cabinet, which contains multiple instruments and equipment and their corresponding labels.

[0089] The equipment detection module 52 is used to input the image to be identified into the preset target detection model and detect the equipment information and equipment frame of each instrument in the image to be identified;

[0090] The text detection module 53 is used to input the image to be recognized into a preset optical character recognition model to detect the text content and text boxes in the image to be recognized.

[0091] The text preprocessing module 54 is used to merge the text boxes according to their positional relationship.

[0092] The matching module 55 is used to perform preliminary matching of the device box and the text box based on their positional relationship, and to perform global optimal matching based on the preliminary matching results through a global matching algorithm to obtain a unique correspondence between the device box and the text box.

[0093] The verification module 56 is used to perform semantic verification on device boxes and text boxes that have a unique correspondence, based on device information and text content, in order to identify semantically related instruments, equipment and labels.

[0094] In one possible implementation, the text preprocessing module 54 is used for:

[0095] For at least two text boxes of the same size that are on the same horizontal line, calculate the spacing between the text boxes and determine a dynamic spacing threshold based on the size of the text boxes.

[0096] Merge text boxes on the horizontal line whose spacing is less than the dynamic spacing threshold.

[0097] In one possible implementation, the text preprocessing module 54 is also used for:

[0098] Calculate the vertical distance and horizontal overlap between each text box;

[0099] If the vertical distance between any two text boxes is less than a preset vertical distance threshold and the horizontal overlap is greater than a preset horizontal overlap threshold, then the two text boxes are merged.

[0100] In one possible implementation, the matching module 55 is used for:

[0101] For any device frame, determine the text boxes within a preset range centered on the device frame, calculate the intersection-union ratio (IUU) between the device frame and each text box within the preset range, and select the first text box corresponding to the device frame from each text box based on the IUU.

[0102] Based on the degree to which each first text box deviates from the central axis of the device frame, the second text box corresponding to the device frame is selected from each first text box;

[0103] Based on the number of device boxes matched by each second text box, filter the third text boxes corresponding to each device box to obtain the preliminary matching results between the device box and the text boxes.

[0104] In one possible implementation, the matching module 55 is used to: identify text boxes with an intersection-to-union ratio (IoU) less than a preset IoU threshold as the first text box corresponding to the device box.

[0105] In one possible implementation, the matching module 55 is used to: determine the first text box whose deviation from the horizontal or vertical center axis of the device frame is less than a preset deviation threshold as the second text box corresponding to the device frame.

[0106] In one possible implementation, the matching module 55 is used to: determine the second text box that only matches the device frame as the third text box corresponding to the device frame.

[0107] In one possible implementation, the verification module 56 is used for:

[0108] The device information and text content are input into a preset semantic verification network to obtain the semantic relevance between the device information and the text content;

[0109] If the semantic relevance is greater than the preset semantic relevance threshold, the corresponding device information and text content will be identified as semantically related instruments and labels.

[0110] In this embodiment of the invention, the target detection model can accurately locate each instrument and device in the image to be identified and extract device information and device frames. The optical character recognition model simultaneously captures the label text content and text frames. Furthermore, to address issues such as irregular line spacing, character spacing, and scattered multi-character fragments that may exist in industrial labels, the text frame position relationship merging process can integrate multiple text frames split from the same label into a complete label unit, effectively avoiding the problems of incomplete recognition and semantic breaks caused by label text fragmentation, and ensuring the integrity and coherence of label text information. Moreover, through the dual matching mechanism of "preliminary position matching + global optimal matching", a precise and unique correspondence between the device and the label is achieved, significantly improving the accuracy and stability of the matching. Finally, the semantic verification stage forms a closed-loop verification of the recognition and matching, further filtering out matching errors. This invention, through a complete process design of extraction, merging, matching, and verification, achieves automation, high precision, and high stability in equipment and tag identification and matching in industrial scenarios. It significantly reduces the cost of manual intervention, improves the efficiency of inspection, asset inventory, and other tasks, and effectively adapts to the unique needs of industrial sites with dense tags, complex backgrounds, uneven lighting, and irregular text layout. It has strong engineering practicality and scenario adaptability.

[0111] Figure 6 This is a schematic diagram of the inspection robot provided in an embodiment of the present invention. Figure 6 As shown, the inspection robot 6 in this embodiment includes a processor 60 and a memory 61. The memory 61 stores a computer program 62. When the processor 60 executes the computer program 62, it implements the steps in the various method embodiments described above. Alternatively, when the processor 60 executes the computer program 62, it implements the functions of each module in the various device embodiments described above.

[0112] For example, computer program 62 can be divided into one or more modules / units, which are stored in memory 61 and executed by processor 60 to complete the present invention. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of computer program 62 in inspection robot 6.

[0113] The inspection robot 6 may include, but is not limited to, a processor 60 and a memory 61. Those skilled in the art will understand that... Figure 6 This is merely an example of the inspection robot 6 and does not constitute a limitation on the inspection robot 6. It may include more or fewer parts than shown, or combine certain parts, or different parts. For example, the inspection robot 6 may also include input / output devices, network access devices, buses, etc.

[0114] For the sake of simplicity and clarity, only the above-described functional modules / units are used as examples. In practical applications, the functions described above can be assigned to different functional modules / units as needed. These modules / units can be implemented in hardware, software, or a combination of both.

[0115] In the above embodiments, the descriptions of each embodiment have their own emphasis. Parts not detailed or described in a particular embodiment can be referred to in the relevant descriptions of other embodiments. Unless otherwise specified or in conflict with logic, the terminology and / or descriptions between different embodiments are consistent and can be referenced interchangeably. Technical features in different embodiments can be combined to form new embodiments based on their inherent logical relationships.

[0116] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A machine vision-based method for inspecting power cabinets, characterized in that, include: Acquire an image of the power cabinet to be identified, wherein the image contains multiple instruments and equipment and corresponding tags; The image to be identified is input into a preset target detection model to detect the equipment information and equipment frames of each instrument and device in the image to be identified; The image to be recognized is input into a preset optical character recognition model to detect the text content and text boxes in the image to be recognized; Merge the text boxes according to their positional relationship; Based on the positional relationship between the device frame and the text frame, a preliminary matching is performed on the device frame and the text frame. Based on the preliminary matching results, a global optimal matching is performed through a global matching algorithm to obtain a unique correspondence between the device frame and the text frame. For device boxes and text boxes with unique correspondences, semantic verification is performed based on device information and text content to identify semantically related instruments, equipment and labels; The step of merging the text boxes according to their positional relationships includes: For at least two text boxes of the same size on the same horizontal line, calculate the spacing between the text boxes and determine a dynamic spacing threshold based on the size of the text boxes; merge text boxes on the horizontal line whose spacing is less than the dynamic spacing threshold. Calculate the vertical distance and horizontal overlap between each text box; if the vertical distance between any two text boxes is less than a preset vertical distance threshold and the horizontal overlap is greater than a preset horizontal overlap threshold, then merge the two text boxes. The preliminary matching of the device frame and the text frame based on their positional relationship includes: For any device frame, determine the text frames within a preset range centered on the device frame, calculate the intersection-union ratio (IUU) between the device frame and each text frame within the preset range, and filter the first text frame corresponding to the device frame from each text frame according to the IUU. Based on the degree to which each first text box deviates from the central axis of the device frame, the second text box corresponding to the device frame is selected from each first text box; Based on the number of device boxes matched by each second text box, filter the third text boxes corresponding to each device box to obtain the preliminary matching results between the device box and the text boxes.

2. The machine vision-based power cabinet inspection method according to claim 1, characterized in that, The step of filtering the first text box corresponding to the device frame from each text box according to the intersection-union ratio includes: determining the text boxes with an intersection-union ratio less than a preset intersection-union ratio threshold as the first text box corresponding to the device frame.

3. The machine vision-based power cabinet inspection method according to claim 1, characterized in that, The step of filtering the second text box corresponding to the device frame from each of the first text boxes based on the degree to which each first text box deviates from the central axis of the device frame includes: The first text box whose deviation from the horizontal or vertical center axis of the device frame is less than a preset deviation threshold is identified as the second text box corresponding to the device frame.

4. The machine vision-based power cabinet inspection method according to claim 1, characterized in that, The step of filtering the third text box corresponding to the device box based on the number of device boxes matched by each second text box includes: determining the second text box that only matches the device box as the third text box corresponding to the device box.

5. The machine vision-based power cabinet inspection method according to claim 1, characterized in that, The semantic verification based on device information and text content to identify semantically related instruments, equipment, and tags includes: The device information and the text content are input into a preset semantic verification network to obtain the semantic correlation between the device information and the text content; If the semantic relevance is greater than the preset semantic relevance threshold, the corresponding device information and text content will be identified as semantically related instruments and tags.

6. A machine vision-based power cabinet inspection device, characterized in that, Used to implement the machine vision-based power cabinet inspection method as described in any one of claims 1-5; The device includes: The acquisition module is used to acquire an image of the power cabinet to be identified, wherein the image contains multiple instruments and equipment and corresponding tags; The device detection module is used to input the image to be identified into a preset target detection model and detect the device information and device frame of each instrument and device in the image to be identified; The text detection module is used to input the image to be recognized into a preset optical character recognition model to detect the text content and text boxes in the image to be recognized; The text preprocessing module is used to merge the text boxes according to their positional relationship. The matching module is used to perform preliminary matching of the device box and the text box based on their positional relationship, and then perform global optimal matching based on the preliminary matching results through a global matching algorithm to obtain a unique correspondence between the device box and the text box. The verification module is used to perform semantic verification on device boxes and text boxes that have a unique correspondence, based on device information and text content, in order to identify semantically related instruments, equipment and labels.

7. An inspection robot, characterized in that, It includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method as described in any one of claims 1 to 5.

Images