Screen cabinet equipment label automatic verification method and system based on visual identification

Abstract

The invention relates to the technical field of image recognition, in particular to a screen cabinet equipment label automatic verification method and system based on visual recognition, and the method comprises the following steps: obtaining an input image, and carrying out the label detection of the input image through a target detection model, and obtaining a visual label; performing text recognition on the detected visual label by using an optical character recognition model; according to the distribution condition of the screen cabinet equipment labels, dividing the reference label into a plurality of label areas in advance, and confirming the current label area to which the input image belongs; performing area verification on the visual label by using the reference label of the current label area; obtaining a reasoning label corresponding to each visual label; and performing position verification on the visual label by using the reasoning label. A topological position voting reasoning mechanism is introduced. After label area matching is completed and the real physical area to which the input image belongs is determined, the actual position of the visual label in the area where the visual label is located can be effectively determined, so that the position matching problem in visual label matching is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition, and in particular to a visual recognition-based cabinet equipment label automatic checking method and system. BACKGROUND

[0002] With the rapid development of the power field, more and more newly built or expanded substations will be put into use. Acceptance task is an important link to ensure the safe and efficient operation of the substation, and the accurate identification and checking of the power cabinet equipment label is a key link of equipment management. If the label of a certain device is incorrectly or omitted, it will affect the normal operation and safety of the entire power system. Traditionally, the checking of these labels is completed manually, i.e., the staff checks the label content and position according to the equipment account one by one. The cabinet equipment label is an important carrier for equipment identity recognition, state recording and maintenance management. The label usually contains device number, name, model and color coding (such as white background black, white background red and yellow background black) information, which is used to indicate the ownership, function and current state (such as running, standby, maintenance, etc.) of the device. The cabinet equipment label checking not only needs to check whether the position of the equipment label is incorrect, but also needs to check whether the content and color information of the equipment label are incorrect.

[0003] However, with the expansion of the power grid and the dramatic increase in the number of equipment, a substation may have thousands of labels, and the manual checking method has low checking efficiency and the human eye is easily disturbed by light, visual fatigue and visually similar characters (such as "1" and "I", "0" and "O" confusion), which leads to a high missed detection rate in the equipment label checking process. Furthermore, the manual checking method cannot record each equipment label comparison information in the record for personnel to perform secondary review. Therefore, how to efficiently and accurately check the content and position of the power cabinet equipment label has become a technical problem in electrical acceptance tasks.

[0004] Although the existing OCR technology can automatically identify text, there is no good solution to implement the automatic checking task of the power cabinet equipment label. The current power cabinet label automatic checking task has three major difficulties: ① visually similar character missed identification and misidentification problem: the matching error caused by visual confusion of characters or missed detection of character recognition in the label number. ② Installation position checking problem: how to establish a one-to-one correspondence between the actual installation position of the equipment and the position in the design drawing. Using a target detection model to locate the label area and combining OCR to recognize the text can only complete the text recognition, without establishing the spatial relationship between the labels, and cannot find the position disorder problem. ③ Multi-dimensional checking missing: lack of consistency checking of color, content and account record. SUMMARY

[0005] In view of at least one of the above defects, the purpose of the present application is to provide a visual recognition-based screen cabinet equipment label automatic verification method and system.

[0006] To achieve this purpose, the present application adopts the following technical solutions: a visual recognition-based screen cabinet equipment label automatic verification method, comprising the following steps: Step S1: acquire an input image, use a target detection model to detect the input image for label detection, and obtain a visual label; Step S2: use an optical character recognition model to recognize the text of the detected visual label, and extract the number key and name value in the visual label; Step S3: according to the distribution of the screen cabinet equipment label, pre-divide the reference label into several label regions, confirm the current label region to which the input image belongs, and use the reference label of the current label region to verify the visual label in the region; Step S4: use the input image and the label topological relationship of the current label region to perform voting reasoning on the visual label, and obtain the reasoning label corresponding to each visual label; use the reasoning label to verify the position of the visual label; Step S5: use the reference label of the current label region to verify the name of the visual label.

[0007] Preferably, the step S2 specifically comprises: Step S21. first use a super-resolution model to reconstruct the image of the detected visual label; then use an optical character recognition model to recognize the characters of the reconstructed image; Step S22. group and sort the recognized characters by row according to a preset pixel width; Step S23. based on the printing rules of the screen cabinet equipment label, format the multiple lines of text to obtain the number key and name value in the visual label.

[0008] Preferably, the step S3 specifically comprises: Step S31. according to the distribution of the screen cabinet equipment label, pre-divide the reference label into several label regions; Step S32. count the number of matching between the number key of the reference label in each label region and the number key of the visual label in the input image; Step S33. select the label region with the most matching number as the current label region to which the input image belongs.

[0009] Optionally, in the step S32, when the matching number of the label region appears the same or the maximum matching number is less than the threshold value, the number key of the visual label in the input image is firstly corrected using the number key of the reference label and the fuzzy matching algorithm, and the matching number of each label region is re-counted.

[0010] Preferably, the step S3 further comprises: Step S34. Comparing the number key of the visual label of the input image with the number key of the reference label of the current label region and the remaining label regions one by one; If the number key of the visual label is the same as the number key of the reference label in the current label region, the visual label passes the region check; If the number key of the visual label is the same as the number key of the reference label in the remaining label regions, the visual label is marked as “position error”; If the number key of the visual label is not the same as the number key of the reference label in any label region, the visual label is marked as “manual review”; If the visual label corresponding to the reference label in the current label region is missing in the input image, a “missing error” is recorded.

[0011] Further, when the number key of the visual label is not the same as the number key of the reference label in any label region, the number key of the visual label is secondly corrected using the number key of the reference label in the current label region and the fuzzy matching algorithm; If the correction is successful, the visual label passes the region check; If the correction fails, the visual label is marked as “manual review”.

[0012] Preferably, the step S4 specifically comprises: Step S41. Constructing the reference KDTree of each label region according to the center point coordinates of the reference label in the label region; obtaining the ROI center point coordinates of the visual label passing the region check in the input image, and constructing the visual KDTree of the input image; Step S42. For each visual label passing the region check, querying its neighbor points and relative directions in the visual KDTree and the reference KDTree of the current label region respectively; grouping the neighbor points according to the relative directions and sorting them according to the distance, and respectively forming the visual neighbor list and the reference neighbor list; Step S43. Taking any of the above visual labels as the query center, pairing the neighbor points in the same direction and the same sequence number in the visual neighbor list and the reference neighbor list one by one, and voting the neighbor points in the visual neighbor list according to the neighbor points in the reference neighbor list. Step S44. Traverse the query above visual label, count the number of votes received by each visual label and the number of votes; Select the most number of votes as the reasoning label corresponding to the voting label of the visual label; Step S45. Use the reasoning label to verify the position of the visual label; If the number key of the visual label is the same as the number key of the corresponding reasoning label, mark the visual label as "position correct"; If the number key of the visual label is not the same as the number key of the corresponding reasoning label, mark the visual label as "position error".

[0013] Further, in step S44, when the number of votes of the visual label received is the same, temporarily skip the position verification of the visual label, and after the reasoning label of the remaining visual label is confirmed, use the confirmed reasoning label to exclude the voting label of the visual label; If there is a voting label with the most number of votes, it is used as the reasoning label of the visual label; If the voting label is still the same number of votes, mark the visual label as "manual review".

[0014] Preferably, the step S5 specifically comprises: Calculate the Levenshtein edit distance between the name value of the visual label in the input image and the name value of the reference label in the current label area; If the Levenshtein edit distance is less than the threshold value, mark the visual label as "content correct"; Otherwise, mark the visual label as "content error".

[0015] Preferably, the screen cabinet equipment label automatic verification method further comprises color verification; Step S1 further comprises: using an image classification model to identify the color of the detected visual label, to obtain the color category of the visual label; Step S5 further comprises: using the reference label of the current label area to verify the color of the visual label, if the color category of the visual label is the same as the color category of the reference label, mark the visual label as "color correct", otherwise mark the visual label as "color error".

[0016] A screen cabinet equipment label automatic verification system based on visual recognition, comprising: A visual module for obtaining a screen cabinet local image containing a screen cabinet equipment label; A processing module for executing the above screen cabinet equipment label automatic verification method; A visualization output module for superimposed display of the visualized screen cabinet equipment label automatic verification result based on the screen cabinet local image.

[0017] One of the above technical solutions has the following advantages or beneficial effects: for the label position verification problem, the application introduces a topological position voting reasoning mechanism. The application obtains the visual label in the screen cabinet equipment image through visual recognition, divides the screen cabinet equipment label into multiple label regions, then confirms the current label region to which the input image belongs and completes the region verification of the visual label, so as to reduce the data amount of verification calculation and system resource occupation, compares the label topological relationship of the input image and the current label region, votes for the position of the visual label, and then completes the specific position verification of the visual label. The application performs coarse and fine double position verifications on the visual label in the input image through region verification and position verification, can effectively verify the position difference between the visual label of the input image and the reference label of the screen cabinet equipment, and then solves the position verification problem in the automatic verification of the screen cabinet equipment label. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 is a flowchart of an embodiment of the method of the application; Figure 2 is a detection effect diagram of a target detection model in an embodiment of the application; Figure 3 is a style display diagram of a screen cabinet equipment label in an embodiment of the application; Figure 4 is an OCR recognition effect display diagram of a visual label in an embodiment of the application; Figure 5 is a label region schematic diagram of a screen cabinet equipment label in an embodiment of the application; Figure 6 is a verification result schematic diagram of a visual output module in an embodiment of the application. DETAILED DESCRIPTION

[0019] Embodiments of the application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the application and cannot be understood as limiting the application.

[0020] In the description of the embodiments of the application, the terms "first", "second" are only used for description purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features. In the description of the embodiments of the application, the meaning of "multiple" is two or more, unless otherwise specifically limited.

[0021] In addition, the terms "first", "second", "third", etc. are used only for descriptive purposes and are not to be construed as indicating or implying relative importance or an ordered ranking of the indicated technical features. Thus, features defined with "first", "second" or "third" can explicitly or implicitly include one or more of such features. In the description of the present application, the meaning of "a plurality of" is two or more, unless otherwise stated. The specific meaning of the above terms in the present application can be understood on a case-by-case basis by those of ordinary skill in the art.

[0022] As shown in Figures 1-4 A visual recognition-based screen cabinet equipment label automatic verification method includes the following steps: Step S1: Obtain an input image, use a target detection model to detect the input image, obtain a visual label, and the detection effect of the target detection model is as shown in Figure 2 ; Step S2: Use an optical character recognition model to recognize the text of the detected visual label, and extract the number key and name value in the visual label; Step S3: According to the distribution of the screen cabinet equipment label, the reference label is pre-divided into several label regions, and the current label region to which the input image belongs is confirmed; the reference label of the current label region is used to verify the region of the visual label; Step S4: Using the input image and the label topological relationship of the current label region, the voting reasoning of the visual label is carried out, and the reasoning label corresponding to each visual label is obtained; the reasoning label is used to verify the position of the visual label; Step S5: The reference label of the current label region is used to verify the name of the visual label.

[0023] After the preliminary visual label is obtained, the content in the visual label has not been obtained, so it cannot continue to be matched, and therefore an optical character recognition model is used to recognize the text of the detected visual label, and the number key and name value are extracted from the visual label for subsequent matching.

[0024] Due to the pixel limitation of the camera and the overall size limitation of the screen cabinet equipment, the input image is usually a part of the screen cabinet equipment, and before the number key and name value are matched, it is necessary to determine which region of the screen cabinet equipment the input image is located in, so as to lay a solid foundation for subsequent matching. Therefore, in the present application, the reference label of the current label region is used to verify the region of the visual label, and the current label region to which the input image belongs is confirmed.

[0025] For the problem of label position matching, the application introduces a topological position voting reasoning mechanism. After determining the real physical area to which the input image belongs through label area matching, the position and direction of the visual label are voted and collectively reasoned, which can effectively determine the actual position of the visual label in the area where it is located, thereby solving the problem of position matching in visual label matching.

[0026] Finally, since the specific position of the visual label has been determined through the key of the visual label, subsequent matching between the reference label and the visual label based on the key of the visual label can be performed to complete the verification of other data such as the name value and label color of the visual label. If the name value of the visual label does not match the reference label during the matching process, it indicates that the content of the visual label is incorrect, and the staff needs to be reminded to perform manual maintenance.

[0027] As can be understood by those skilled in the art, during the verification process of the screen cabinet equipment label, the reference label is the screen cabinet equipment label correctly set on the screen cabinet, and the reference label is generally recorded in the screen cabinet equipment account for ready calling out and comparison and verification with the visual label.

[0028] In some embodiments, system initialization and data preparation are generally required before step S1, specifically including: loading the target detection model, image detection model, optical character recognition model, and super-resolution model required for use; reading the key, value, color category, center point coordinate, and label area of the reference label from the screen cabinet equipment account; and using the center point coordinate of each reference label to construct a KDTree spatial index for subsequent voting reasoning in step S4.

[0029] In some embodiments, in step S1, the YOLOv8n model is used for label detection and color recognition of the input image. Specifically, the YOLOv8n-label-det target detection model is used for label detection of the input image to obtain the image and ROI center point coordinate of the visual label; and the YOLOV8n-label-color-cls image classification model is used for color recognition of the detected visual label to obtain the color category of the visual label.

[0030] The YOLOv8n-label-det target detection model is trained using the YOLOv8n model on a self-built screen cabinet equipment label dataset, and the detection effect of the YOLOv8n-label-det target detection model is as shown in Figure 2YOLOV8n-label-color-cls image classification model is trained by using a YOLOv8n-cls model on a classification data set constructed from the cropped screen cabinet equipment label, which is divided into three categories, i.e., white background black font, white background red font and yellow background black font, as shown in the following table. Figure 3

[0031] The general annotation process of the screen cabinet equipment label data set includes: using a third-party annotation tool LabelImg to annotate the images containing screen cabinet equipment labels collected on site, and storing the annotated data in the form of an XML file, which contains the annotation data and category information of each image. The XML file saving the annotation information is converted into a TXT file in YOLO format required for training the YOLOv8n model by a data format conversion script, so as to facilitate the final training of the target detection model. The annotation process of the classification data set is similar and will not be repeated.

[0032] In the present application, the YOLOv8n model is used for the detection task of the screen cabinet equipment label. If other target detection models such as YOLOv11 or RT-DERT are used, similar target detection tasks can also be achieved. In the present application, the YOLOv8n-cls model is used for the color classification task of the screen cabinet equipment label. If other image classification models such as MobileNet or Vit are used, similar image classification tasks can also be achieved.

[0033] Preferably, the step S2 specifically comprises: Step S21. First, an image of the detected visual label is reconstructed using a super-resolution model, and then the reconstructed image is subjected to character recognition using an optical character recognition model. Specifically, the image of the detected visual label can be first reconstructed using an ESPCx4 super-resolution model, and then the reconstructed image is subjected to character recognition using a PaddleOCR optical character recognition model. The recognition accuracy of the characters can be improved by reconstructing the image of the visual label. In the present application, the ESPCx4 super-resolution model is used to reconstruct the corresponding high-resolution image of the visual label, which facilitates the subsequent character recognition task. If other super-resolution models such as FSRCNN or VDSR are used, similar super-resolution image reconstruction tasks can also be achieved. In the present application, the PaddleOCR optical character recognition model is used to recognize the characters on the visual label. If other optical character recognition models such as TesseractOCR or MMOCR are used, similar OCR character recognition tasks can also be achieved.

[0034] Step S22. The recognized characters are sorted by row grouping according to a preset pixel width. ​Step S23. Format the multiple lines of text based on the printing rule of the cabinet equipment label to obtain the number key and the name value in the visual label.

[0035] In one embodiment, when the identified characters are sorted by row grouping, the y coordinate of the text box center is used for row grouping, allowing an error of ±5 pixels to be grouped into the same row, and then the x coordinate of the characters in each row is used for sorting, so as to ensure the correct reading order of the multiple lines of text.

[0036] The printing rule of the cabinet equipment label clearly records the format of the information in the label, such as Figure 4 As shown, for example, the first line is the number key, and the others are the name value. When dividing the information, the first line of text without Chinese in the visual label is taken as the number key, and the remaining part is combined and removed after the line break to be taken as the name value. By dividing the text according to the printing rule of the cabinet equipment label, the number key and the name value in the visual label can be clearly extracted.

[0037] In some embodiments, if the above OCR recognition fails or fails to extract the number key, the visual label is marked as "manual review" for subsequent reminding of the staff to perform manual verification. The record data of all recognized visual labels include: number key, name value, ROI center point coordinates, color category, so as to facilitate subsequent verification.

[0038] In some embodiments, the PaddleOCR optical character recognition model is trained on the PP-OCRv4-server-rec model based on the self-built cabinet equipment label character dataset. The PPOCRLabel labeling software is used to label the cropped cabinet equipment label pictures, thereby constructing the required dataset for training, and creating the required txt character set for the optical character recognition model.

[0039] Preferably, the step S3 specifically includes: Step S31. According to the distribution of the cabinet equipment label, the reference label is pre-divided into a plurality of label regions; Step S32. Count the number of matching between the number key of each label region in the reference label and the number key of the visual label in the input image; Step S33. Select the label region with the largest number of matches as the current label region to which the input image belongs.

[0040] Specifically, the reference label in the cabinet equipment account can be divided into a plurality of label regions according to the aggregation degree of the center point coordinates of the cabinet equipment label. As shown in Figure 5As shown, in one exemplary screen cabinet image, the screen cabinet equipment tags are concentrated in 5 regions of the screen cabinet, and the reference tags in the screen cabinet equipment account are correspondingly divided into 5 groups. It should be noted that step S31 is usually completed in the system initialization and data preparation stage described above.

[0041] When the current tag region is confirmed, the reference tags in the screen cabinet equipment account are called, and the number of matching between the key of each reference tag in each tag region and the key of each visual tag is counted. For example, 8 visual tags are recognized in the input image, and the number of matching between the key of the reference tags in the A, B, C, D, E tag regions and the key of the visual tags is 2, 3, 6, 7, and 4 respectively. At this time, the matching number in the D tag region is the largest, and therefore the D tag region is selected as the current tag region to which the input image belongs.

[0042] Optionally, in step S32, when the matching number of the tag region is the same or the largest matching number is less than the threshold value, the key of the visual tag in the input image is initially corrected using the key of the reference tag and the fuzzy matching algorithm, and the matching number of each tag region is re-counted.

[0043] In the case that the character recognition accuracy in step S2 is low, or the visual tags in the input image are mostly wrong, the matching number of the tag region will be the same or the largest matching number will be less than the threshold value. At this time, the effect of continuing to perform subsequent tag verification is not good, and the input image can be re-identified by repeating steps S1 and S2, or a "manual verification" prompt can be directly output for the input image.

[0044] In some embodiments, the key of the visual tag in the input image can be initially corrected using the key of the reference tag corresponding to the tag region and the fuzzy matching algorithm, and the matching number of each tag region is re-counted. Specifically, for the visual tags in the input image that cannot be correctly matched, the Levenshtein distance is used to calculate the similarity between two strings, and when the Levenshtein distance between the key of the visual tag and the key of the reference tag is less than 2, the key of the reference tag is used to replace the key of the corrected visual tag. After the key of the visual tag is corrected, the matching number of each tag region is re-counted to find the tag region with the largest matching number as the current tag region to which the input image belongs.

[0045] The skilled person understands that the Levenshtein distance, also known as the edit distance, refers to the minimum number of editing (such as replacement, addition, deletion) operations required to convert one string into another between two strings. In the present application, the Levenshtein distance is preferably used as a fuzzy matching algorithm to calculate the similarity between two strings. Alternatively, the Jaccard similarity or the Jaro-Winkler algorithm can be used.

[0046] Preferably, after determining the current label area to which the input image belongs, the visual label of the input image is regionally verified by using the reference label of the current label area. The step S3 further comprises: Step S34. Comparing the number key of the visual label of the input image with the number key of the reference label of the current label area and the remaining label areas one by one in sequence; If the number key of the visual label is the same as the number key of the reference label in the current label area, the visual label passes the regional verification, indicating that the visual label belongs to the label of the screen cabinet equipment and is placed in the correct label area; If the number key of the visual label is the same as the number key of the reference label in the remaining label areas, the visual label is marked as "position error", indicating that the visual label belongs to the label of the screen cabinet equipment but is placed in the wrong label area; If the number key of the visual label is not the same as the number key of the reference label in any label area, the visual label is marked as "manual review", indicating that the visual label does not belong to the label of the screen cabinet equipment; If the visual label corresponding to the reference label in the current label area is missing in the input image, a "missing error" is recorded, indicating that the visual label recognition fails or the corresponding label on the screen cabinet equipment is missing.

[0047] Preferably, when the number key of the visual label is not the same as the number key of the reference label in any label area, the number key of the visual label is secondarily corrected using the number key of the reference label in the current label area and the fuzzy matching algorithm; If the correction is successful, the visual label passes the regional verification; If the correction fails, the visual label is marked as "manual review".

[0048] In the processing of area verification, the area position of the label can be determined. For the visual label of "position error" or "missing error" which has been confirmed as a definite error, subsequent verification can be omitted, and it is not necessary to participate in the subsequent topological position reasoning to reduce the data processing amount. For the visual label of "manual review", it is possible that the matching error of part of the visual label is caused by the visual recognition error due to the reason of shooting or model recognition, such as the letter o and the number 0, the letter l and the number 1, etc. Therefore, the number key of the visual label is modified again by using the fuzzy matching algorithm to reduce the number of labels that need to be "manually reviewed" and reduce the misjudgment rate.

[0049] In some embodiments, the secondary modification also uses the Levenshtein distance to calculate the similarity between two strings. For the above-mentioned "manual review" visual label, when the Levenshtein distance between the number key of the visual label and the number key of the reference label is less than 2, or when the number key of the visual label is a substring of the number key of the reference label, the number key of the modified visual label is replaced by the number key of the reference label, and the corresponding visual label also passes the area verification.

[0050] Preferably, the step S4 specifically comprises: Step S41. Construct the reference KDTree of each label area according to the center point coordinates of the reference label in the label area; obtain the ROI center point coordinates of the visual label passing the area verification in the input image, and construct the visual KDTree of the input image; Step S42. For each visual label passing the area verification, query its near neighbor points and relative directions in the visual KDTree and the reference KDTree of the current label area, respectively; group the near neighbor points by relative direction and sort them by distance, and form the visual neighbor list and the reference neighbor list, respectively; Step S43. Take any of the above-mentioned visual labels as a query center, and pair the near neighbor points in the same direction and the same sequence number in the visual neighbor list and the reference neighbor list one by one, and vote for the near neighbor points in the visual neighbor list by the near neighbor points in the reference neighbor list; Step S44. Traverse the above-mentioned visual labels, and count the number of votes and the number of votes received by each visual label; select the voting label with the most votes as the reasoning label corresponding to the visual label; Step S45. Perform position verification on the visual label using the reasoning label; if the number key of the visual label is the same as the number key of the corresponding reasoning label, the visual label is marked as "position correct"; if the number key of the visual label is not the same as the number key of the corresponding reasoning label, the visual label is marked as "position error".

[0051] For the visual tags that pass the step S3 area verification, it is necessary to further confirm whether the position of the visual tag in the current tag area is correct. The present scheme uses the spatial topological relationship of the cabinet equipment tags to vote and reason, obtains the reasoning tag corresponding to each visual tag, and then compares the visual tag with the reasoning tag to finally complete the specific position verification of the visual tag in the tag area. The following is described with an embodiment: The visual KDTree exists a numbered key sequence: A1, C1, B1, D1, E1, key…; The reference KDTree exists a numbered key sequence: A1, B1, C1, D1, E1, key…; Taking the first visual tag A1 as the query center, the visual neighbor list and the reference neighbor list in a certain direction are obtained: Visual neighbor list: [1, C1], [2, B1], [3, D1], [distance, key]…; Reference neighbor list: [1, B1], [2, C1], [3, D1], [distance, key]…; The neighbor points in the reference neighbor list vote for the neighbor points in the visual neighbor list one by one. Except for the visual tag A1 as the query center, each visual tag can receive one vote. The voting result is as follows: Visual tag C1: [B1: 1]; Visual tag B1: [C1: 1]; Visual tag D1: [D1: 1]; Visual tag key: [voted tag: number of votes]; The next visual tag is taken as the query center, and the above voting process is repeated until all visual tags are queried. The number of votes received by each visual tag is counted, and the final voting result is as follows: Visual tag A1: [A1: 5]; Visual tag B1: [C1: 5]; Visual tag C1: [B1: 5]; Visual tag D1: [D1: 5]; [F1: 2] Visual tag E1: [E1: 3]; [D1: 3] Visual tag key: [voted tag: number of votes]; For the visual tag A1, the number of votes of the voting tag A1 is the largest, A1 is the corresponding reasoning tag, the visual tag and the reasoning tag are the same, and the visual tag is marked as “position correct”. It indicates that the visual tag is correctly placed in a specific position in the current tag area. For the visual label B1, the voting number of the voting label C1 is the most, C1 is the corresponding reasoning label, the visual label is different from the reasoning label, and the visual label is marked as "position error"; For the visual label C1, the voting number of the voting label B1 is the most, B1 is the corresponding reasoning label, the visual label is different from the reasoning label, and the visual label is marked as "position error"; For the visual label D1, the voting number of the voting label D1 is the most, D1 is the corresponding reasoning label, the visual label is the same as the reasoning label, and the visual label is marked as "position correct"; For the visual label E1, the voting number of the voting label D1 and E1 is the same, and the corresponding reasoning label cannot be obtained, and the visual label is marked as "manual review".

[0052] In some embodiments, in step S42, the queried neighbor points are grouped in eight main directions (east, southeast, south, southwest, west, northwest, north, northeast, each direction deviates by 45 degrees), and in each direction group, the distance from the queried visual label is sorted from near to far, thereby forming a visual neighbor list and a reference neighbor list.

[0053] In some embodiments, in step S44, each visual label can receive voting verification from different relative directions through cross-voting of multiple query centers. In addition, a distance threshold can be set when querying neighbor points, and neighbor points exceeding the threshold do not participate in voting. Finally, the voting results received by each visual label are accumulated, and the voting label with the most votes is selected as the corresponding reasoning label of the visual label.

[0054] The present application can effectively verify the position difference between the visual label of the input image and the reference label of the screen cabinet device through coarse-to-fine two specific position verifications of the screen cabinet device label, and enhance the robustness and accuracy of the verification, thereby solving the position verification problem in the automatic verification of the screen cabinet device label.

[0055] Further, in step S44, when the voting labels received by the visual label have the same number of votes, the position verification of the visual label is temporarily skipped, and after the reasoning labels of the remaining visual labels are confirmed, the confirmed reasoning labels are used to exclude the voting labels of the visual label; If there is a voting label with the most votes, it is used as the reasoning label of the visual label; If the voting labels still have the same number of votes, the visual label is marked as "manual review".

[0056] Continuing with the above-mentioned visual label E1 as an example, the voting quantity of the voting label D1 and E1 appears the same, temporarily skipping the position verification of the visual label E1, since the number key in the label area is unique, when the reasoning label of the visual label D1 is confirmed as D1, the voting label D1 can be excluded, taking E1 as the voting label with the most voting quantity, and finally confirming E1 as the reasoning label corresponding to the visual label E1. The scheme adds the visual label with the same voting quantity to the processing queue first, and then processes the visual label with the same voting quantity again after completing the voting reasoning of the remaining visual labels, and removes the target occupied by other labels from the voting options of the visual label with the same voting quantity, if only one available option is left after the removal, the option is taken as the reasoning label of the visual label; thereby reducing the number of labels for "manual review", and reducing the misjudgment rate of the position verification process.

[0057] Compared with the traditional voting mechanism, when there is a tie, it may be directly marked as an error or randomly selected, resulting in misjudgment. The scheme uses the "skip-backtrack-exclude" strategy to exclude the interference options in reverse using the confirmed reasoning label, thereby improving the verification accuracy in the tie scenario.

[0058] Preferably, the step S5 specifically comprises: calculating the Levenshtein edit distance between the name value of the visual label in the input image and the name value of the reference label in the current label area; If the Levenshtein edit distance is less than a threshold value, the visual label is marked as "content correct"; otherwise, the visual label is marked as "content error".

[0059] The Levenshtein edit distance can accurately quantify common errors such as missing words, wrong words, and multiple words generated by OCR, providing a simple and efficient automated text verification method. In the present application, the Levenshtein distance is used to calculate the similarity between two strings. If other fuzzy matching algorithms such as Jaccard similarity or Jaro-Winkler algorithm are used, the similarity calculation task between strings can also be realized.

[0060] In some embodiments, the visual label that completes the position verification is subjected to name verification, when the Levenshtein distance between the name value of the visual label and the name value of the reference label is less than 2, or when the name value of the visual label is a substring of the name value of the reference label, it is considered that the contents of the two are consistent, and the visual label is marked as "content correct".

[0061] Preferably, the cabinet equipment label automatic verification method further comprises color verification. The step S1 further comprises: color recognition of the detected visual label using an image classification model to obtain a color category of the visual label. The step S5 further comprises: color verification of the visual label using a reference label of the current label region, if the color category of the visual label is the same as the color category of the reference label, the visual label is marked as “color correct”, otherwise the visual label is marked as “color error”.

[0062] In some embodiments, the color verification of the visual label that has passed the position verification, the color category of the visual label obtained in step S1 is called, the color category of the corresponding reference label is called, if the color category of the visual label is the same as the color category of the reference label, the visual label is marked as “color correct”, otherwise the visual label is marked as “color error”.

[0063] A visual recognition-based cabinet equipment label automatic verification system, comprising: a visual module, configured to acquire a cabinet local image containing cabinet equipment labels; a processing module, configured to execute the cabinet equipment label automatic verification method described above; a visual output module, configured to superimpose and display the visualized cabinet equipment label automatic verification result on the basis of the cabinet local image.

[0064] As Figure 6 shown, in the visual output, the program will mark the detection frame of the visual label on the original input image using different colors according to the record content, and add corresponding brief explanation text above or below the detection frame. For example, when the color category is inconsistent with the cabinet equipment account, the label is circled with a red frame, and the correct color category is marked beside the frame. When the label is missing, the undetected equipment label number is listed in the upper right corner of the input image. In this way, a verification result image with all annotations and explanations, and a structured error warning list are finally output, thereby completing the entire cabinet equipment label automatic verification process.

[0065] In the description of the present specification, the description of the terms “one embodiment”, “some embodiments”, “illustrative embodiment”, “example”, “specific example”, or “some examples” means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0066] While embodiments of the application have been shown and described, it is to be understood that the embodiments described are merely divergences of the principles and application of the present application and that numerous modifications, changes, substitutions, and alterations can be made thereto without departing from the spirit and scope of the present application, which is defined by the following claims and their equivalents.

Claims

1. A visual recognition-based automatic checking method for cabinet equipment labels, characterized in that, The method comprises the following steps: Step S1: obtaining an input image, performing label detection on the input image using a target detection model to obtain visual labels; Step S2: performing text recognition on the detected visual labels using an optical character recognition model to extract the number key and the name value in the visual labels; Step S3: according to the distribution of the cabinet equipment labels, the reference labels are pre-divided into a plurality of label regions, and the current label region to which the input image belongs is determined; the reference labels in the current label region are used to perform region verification on the visual labels; Step S4: using the input image and the label topological relationship of the current label region, voting reasoning is performed on the visual labels to obtain a reasoning label corresponding to each visual label; the reasoning label is used to perform position verification on the visual labels; Step S5: using the reference labels in the current label region to perform name verification on the visual labels.

2. The method of claim 1, wherein the method further comprises: The step S2 specifically comprises: Step S21: first, performing image reconstruction on the detected visual labels using a super-resolution model; and then, performing character recognition on the reconstructed image using an optical character recognition model; Step S22: grouping and sorting the recognized characters by row according to a preset pixel width; Step S23: based on the printing rules of the cabinet equipment labels, the multi-row text is formatted to obtain the number key and the name value in the visual labels.

3. The visual recognition-based cabinet equipment label automatic verification method according to claim 1, wherein The step S3 specifically comprises: Step S31: according to the distribution of the cabinet equipment labels, the reference labels are pre-divided into a plurality of label regions; Step S32: the number of matching between the number key of the reference labels in each label region and the number key of the visual labels in the input image is counted; Step S33: the label region with the largest number of matches is selected as the current label region to which the input image belongs.

4. The method of claim 3, wherein the method further comprises: In the step S32, when the matching number of the label regions is the same or the largest matching number is less than a threshold value, the number key of the visual labels in the input image is initially corrected using the number key of the reference labels and a fuzzy matching algorithm, and the matching number of each label region is counted again.

5. The method of claim 1, wherein the method further comprises: The step S3 further comprises: Step S34: the number key of the visual labels in the input image is compared with the number key of the reference labels in the current label region and the remaining label regions one by one; if the number key of the visual labels is the same as the number key of the reference labels in the current label region, the visual labels pass the region verification; if the number key of the visual labels is the same as the number key of the reference labels in the remaining label regions, the visual labels are marked as "position error"; if the number key of the visual labels is not the same as the number key of the reference labels in any label region, the visual labels are marked as "manual review"; if the visual labels corresponding to the reference labels in the current label region are missing in the input image, a "missing error" is recorded.

6. The method of claim 5, wherein the method further comprises: When the number key of the visual tag is not the same as the number key of the reference tag in any tag area, the number key of the visual tag is secondarily corrected using the number key of the reference tag in the current tag area and the fuzzy matching algorithm; If the correction is successful, the visual tag passes the area verification; If the correction fails, the visual tag is marked as "manual review".

7. The method of claim 1, wherein the method further comprises: The step S4 specifically comprises: Step S41. According to the center point coordinates of the reference tags in the tag area, a reference KDTree of each tag area is constructed in advance; the ROI center point coordinates of the visual tags passing the area verification in the input image are obtained to construct a visual KDTree of the input image; Step S42. For each visual tag passing the area verification, its near neighbor points and relative directions are respectively queried in the visual KDTree and the reference KDTree of the current tag area; the near neighbor points are grouped according to the relative directions and sorted according to the distance closeness to form a visual neighbor list and a reference neighbor list respectively; Step S43. Taking any of the above visual tags as a query center, the near neighbor points in the same direction and the same sequence number in the visual neighbor list and the reference neighbor list are correspondingly paired, and the near neighbor points in the reference neighbor list vote for the near neighbor points in the visual neighbor list; Step S44. The above visual tags are traversed, and the voting tags and the number of votes received by each visual tag are counted; the voting tag with the largest number of votes is selected as the reasoning tag corresponding to the visual tag; Step S45. The visual tag is subjected to position verification using the reasoning tag; if the number key of the visual tag is the same as the number key of the corresponding reasoning tag, the visual tag is marked as "position correct"; if the number key of the visual tag is not the same as the number key of the corresponding reasoning tag, the visual tag is marked as "position error".

8. The method of claim 7, wherein the method further comprises: In the step S44, when the voting tags received by the visual tag have the same number of votes, the position verification of the visual tag is temporarily skipped, and after the reasoning tags of the remaining visual tags are confirmed, the confirmed reasoning tags are used to exclude the voting tags of the visual tag; If there is a voting tag with the largest number of votes, it is used as the reasoning tag of the visual tag; If the voting tags still have the same number of votes, the visual tag is marked as "manual review".

9. The method of claim 1, wherein the method further comprises: The step S5 specifically comprises: The Levenshtein edit distance between the name value of the visual tag in the input image and the name value of the reference tag in the current tag area is calculated; If the Levenshtein edit distance is less than a threshold value, the visual tag is marked as "content correct"; otherwise, the visual tag is marked as "content error".

10. The method of claim 1, wherein the method further comprises: The cabinet equipment tag automatic verification method further comprises color verification; The step S1 further comprises: using an image classification model to perform color recognition on the detected visual tag to obtain the color category of the visual tag; Step S5 further comprises: color checking the visual label using the reference label of the current label area, if the color type of the visual label is the same as the color type of the reference label, marking the visual label as "color correct", otherwise marking the visual label as "color error".

11. A visual recognition based automatic verification system for cabinet equipment labels, characterized in that, The method comprises: a visual module configured to acquire a local image of the screen cabinet containing a screen cabinet device label; a processing module configured to execute the screen cabinet device label automatic checking method according to any one of claims 1-10; a visualization output module configured to superimpose and display the visualized screen cabinet device label automatic checking result on the basis of the local image of the screen cabinet.

Images