Visual detection method and device for appearance of screen cabinet, electronic equipment and storage medium

Abstract

The invention provides a screen cabinet appearance visual detection method and device, electronic equipment and a storage medium, and relates to the technical field of intelligent visual detection. The screen cabinet appearance visual detection method comprises the following steps: acquiring a to-be-detected screen cabinet appearance image; according to a preset appearance shooting mode, dividing the screen cabinet appearance image to determine an area appearance image, and according to the area appearance image, determining a corresponding target detection model; based on the target detection model, a preset character recognition model, the area appearance image and preset screen cabinet design information, determining a screen cabinet text comparison result, a type comparison result and a screen cabinet color comparison result, and determining the screen cabinet text comparison result, the type comparison result and the screen cabinet color comparison result as visual detection results. According to the method, the appearance image of the screen cabinet can be divided, the target detection model is selected in a targeted manner, the consistency comparison requirement of the screen cabinet and the screen cabinet design information is met based on detection and text recognition, and the visual detection effect of the screen cabinet is improved.

Description

Technical Field

[0001] The present application relates to the technical field of intelligent vision detection, for example, to a visual detection method, device, electronic device and storage medium for the appearance of a switchgear cabinet. Background Art

[0002] The secondary electrical switchgear cabinet refers to the switchgear cabinet used for installing secondary equipment such as control, protection, and signal in the power system, and is an important part of the power system. The consistency check between the secondary electrical switchgear cabinet and the switchgear cabinet design drawing is an important link in the switchgear cabinet factory. If the actual installation does not match the drawing, it may cause misoperation, short circuit or other electrical faults in the actual project, or cause some functions to be unable to be correctly realized, affecting the operation of the entire power system. The consistency check of the secondary switchgear cabinet includes appearance inspection and function inspection. The inspection items of the switchgear cabinet appearance include dozens of sub-items such as the switchgear cabinet nameplate, air switch, pressure plate, terminal block, etc.

[0003] Currently, the visual inspection process of the secondary electrical switchgear cabinet generally adopts the method of manual visual inspection. This method has low efficiency and is prone to errors. For inspection items such as air switches and pressure plates, the quantity is usually large and the content to be inspected is also large. Manual inspection takes a lot of time, resulting in low detection efficiency. Summary of the Invention

[0004] The present application aims to provide a visual detection method, device, electronic device and storage medium for the appearance of a switchgear cabinet.

[0005] According to one aspect of the present application, a visual detection method for the appearance of a switchgear cabinet is proposed, including: obtaining an appearance image of the switchgear cabinet to be detected; dividing the appearance image of the switchgear cabinet according to a preset appearance shooting method to determine a regional appearance image, and determining a corresponding target detection model according to the regional appearance image; determining a switchgear cabinet text comparison result, a type comparison result and a switchgear cabinet color comparison result based on the target detection model, a preset character recognition model, the regional appearance image and preset switchgear cabinet design information, and determining the switchgear cabinet text comparison result, the type comparison result and the switchgear cabinet color comparison result as visual detection results.

[0006] According to one aspect of the present application, a visual detection device for the appearance of a switchgear cabinet is proposed, including:

[0007] An image acquisition module, configured to obtain an appearance image of the switchgear cabinet to be detected;

[0008] A model determination module, configured to divide the appearance image of the switchgear cabinet according to a preset appearance shooting method to determine a regional appearance image, and determine a corresponding target detection model according to the regional appearance image;

[0009] A result determination module, configured to determine a cabinet text comparison result, a type comparison result, and a cabinet color comparison result based on a target detection model, a preset character recognition model, a regional appearance image, and preset cabinet design information, and determine the cabinet text comparison result, the type comparison result, and the cabinet color comparison result as a visual detection result.

[0010] According to an aspect of the present application, an electronic device is provided, which includes: a processor; a memory storing a computer program, and when the computer program is executed by the processor, the processor is caused to execute the visual detection method of the cabinet appearance as described above.

[0011] According to an aspect of the present application, a non-transitory computer-readable medium is provided, on which readable instructions are stored, and when the instructions are executed by the processor, the processor is caused to execute the visual detection method of the cabinet appearance as described above.

[0012] It should be understood that the above general description and the following detailed description are only exemplary and do not limit the present application.

[0013] Beneficial effects:

[0014] Through the above embodiments provided by the present application, by dividing the cabinet appearance image according to the preset appearance shooting method and determining the regional appearance image, more refined detection can be performed on different regions of the cabinet. Compared with the overall detection, this method can capture and identify the detailed features of each region more accurately, thereby improving the detection accuracy. For different regional appearance images, determining the corresponding target detection model can make full use of the advantages of different models in specific situations and achieve more efficient detection for elements and element types. Through the preset character recognition model, the text information on the cabinet can be recognized efficiently and accurately. By comparing the recognized cabinet text with the preset cabinet design information, errors or inconsistencies in the text information can be discovered in a timely manner, thereby ensuring the accuracy and compliance of the cabinet. By comparing the actual color and element type of the cabinet with the preset cabinet design information, type and color differences or situations that do not meet the design requirements can be discovered in a timely manner. This is of great significance for ensuring the appearance quality and consistency of the cabinet. Integrating the cabinet text comparison result, the element type comparison result, and the cabinet color comparison result into a visual detection result can provide users with comprehensive cabinet appearance detection information, facilitating users to quickly understand the actual situation of the cabinet. Description of the Drawings

[0015] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the accompanying drawings required for the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other accompanying drawings can be obtained based on these drawings without exceeding the scope of protection required by the present application.

[0016] Figure 1 It is a flowchart of the visual inspection method for the appearance of the switchgear cabinet provided by the embodiment of the present application;

[0017] Figure 2 It is a block diagram of the visual inspection device for the appearance of the switchgear cabinet provided by the embodiment of the present application;

[0018] Figure 3 It is a schematic structural diagram of the electronic device provided by the embodiment of the present application. Detailed implementation manners

[0019] Now, the exemplary embodiments will be described more comprehensively with reference to the accompanying drawings. However, the exemplary embodiments can be implemented in various forms and should not be construed as limited to the embodiments described herein; on the contrary, these embodiments are provided so that the present application will be comprehensive and complete, and the concept of the exemplary embodiments will be fully conveyed to those skilled in the art. The same reference numerals in the figures denote the same or similar parts, and thus their repeated description will be omitted.

[0020] In addition, the described features, structures or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, many specific details are provided to give a full understanding of the embodiments of the present application. However, those skilled in the art will realize that the technical solutions of the present application can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, well-known methods, devices, implementations or operations are not shown or described in detail to avoid obscuring various aspects of the present application.

[0021] The block diagrams shown in the accompanying drawings are only functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0022] The flowcharts shown in the accompanying drawings are only exemplary illustrations, not necessarily including all the content and operations / steps, nor necessarily executed in the described order. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined. Therefore, the actual execution order may change according to the actual situation.

[0023] It should be understood that although terms such as first, second, and third may be used herein to describe various components, these components should not be limited by these terms. These terms are used to distinguish one component from another. Thus, the first component discussed below may be referred to as the second component without departing from the teachings of the concepts of this application. As used herein, the term "and / or" includes any one of the associated listed items and all combinations of one or more of them.

[0024] Specific implementation manners can refer to the following embodiments.

[0025] Figure 1 As shown in the flowchart of the visual detection method for the appearance of the cabinet provided by the embodiments of this application, the method of this embodiment can be applied to a visual detection server. Figure 1 As shown, the method includes: step S10, step S11, and step S12.

[0026] In step S10, an image of the appearance of the cabinet to be detected is acquired.

[0027] In this application, the image of the appearance of the cabinet to be detected is an image obtained by photographing the secondary electrical cabinet at the current moment.

[0028] In some implementation manners, image acquisition devices can be arranged in various directions of the secondary electrical cabinet to acquire images of various directions of the cabinet. In other implementation manners, one image acquisition device can be arranged to acquire images of the appearance of various directions of the cabinet in a preset order. The visual detection server can receive the images acquired and sent by the image acquisition device.

[0029] In step S11, according to the preset appearance photographing manner, the image of the cabinet appearance is divided to determine the regional appearance image, and according to the regional appearance image, the corresponding target detection model is determined.

[0030] In this application, regardless of whether the number of image acquisition devices is one or more, the appearance photographing manner can be preset. For example, different image acquisition devices are used to photograph different cabinet appearance elements, and the cabinet appearance elements include cabinet eyebrows, nameplates, quality tracking sheets, pressure plates, grounding copper bars, buttons, handles, air switches, terminal blocks, and small busbars, etc.

[0031] According to the exemplary embodiment, according to the preset appearance photographing manner, the photographing process of the image acquisition device and the relevant information of image uploading can be obtained, and then the received image of the cabinet appearance is divided to obtain the regional appearance images corresponding to different regions, and one or more cabinet appearance elements are included in the regional appearance image. The regional appearance image can include a front panel image, a rear panel image, a front panel image, a rear upper rail image, a rear left rail image, a rear right rail image, and a rear lower rail image.

[0032] Since different cabinet appearance elements are located at different positions on the cabinet, they may appear flat or tilted when captured and displayed in the image. Different object detection models can be used for image analysis according to different display situations. Therefore, the corresponding relationship between different regional appearance images and object detection models can be preset.

[0033] In step S12, based on the object detection model, the preset character recognition model, the regional appearance image, and the preset cabinet design information, determine the cabinet text comparison result, the type comparison result, and the cabinet color comparison result, and determine the cabinet text comparison result, the type comparison result, and the cabinet color comparison result as the visual detection result.

[0034] In this application, the object detection model can include the YOLOX detection model and the YOLOv7-OBB detection model. When dividing the cabinet appearance image, one or two object detection models may be used. The character recognition model can be an optical character recognition model, and specifically, PaddleOCR can be used. The cabinet design information can be the pre-obtained cabinet design drawings, which can contain the parameter information of different elements of the cabinet and the corresponding relevant characters.

[0035] Different elements on the cabinet have corresponding characters such as names and types. Comparing these characters with the actual cabinet design information gives the cabinet text comparison result. The cabinet text comparison result in this application is used to represent the comparison result between the recognized text and the actual text, that is, whether the recognized text is correct. For some elements on the cabinet, there may be multiple colors, and the types of the same element in different colors are different. Comparing the colors with the actual cabinet design information gives the cabinet color comparison result. The different types of elements can be directly output by the object detection model.

[0036] According to the exemplary embodiment, the regional appearance image can be input into the object detection model, the positions and relevant attributes of different elements can be framed, and an image with a data frame and the element type can be output. The relevant attributes can include, for example, color, name, etc. Then, the image with the data frame is input into the character recognition model, the text in the image can be extracted, and further, the extracted text and the element type are compared with the cabinet design information to obtain the visual detection result.

[0037] In some implementations, a large number of appearance images can be collected in advance. These appearance images are taken from various angles and various application environments. The labelme annotation tool is built into the object detection model to annotate a large number of appearance images, so that the trained object detection model has the ability to output the desired data frame. The PPOCRLabel annotation tool is built on the character recognition model to annotate the appearance images, and data augmentation operations such as flipping, rotating, cropping, and color transformation are performed on the text in the images to expand the samples, so that the trained character recognition model has the ability to extract text at various angles.

[0038] In this application, the cabinet appearance images are divided according to the preset appearance shooting method, and the regional appearance images are determined, so that more refined detection can be performed on different regions of the cabinet. Compared with the overall detection, this method can capture and identify the detailed features of each region more accurately, thereby improving the detection accuracy. For different regional appearance images, the corresponding object detection models are determined, which can make full use of the advantages of different models in specific situations to achieve more efficient detection for elements and element types. Through the preset character recognition model, the text information on the cabinet can be recognized efficiently and accurately. By comparing the recognized cabinet text with the preset cabinet design information, errors or inconsistencies in the text information can be found in time, thereby ensuring the accuracy and compliance of the cabinet. By comparing the actual color and element type of the cabinet with the preset cabinet design information, type and color differences or situations that do not meet the design requirements can be found in time. This is of great significance for ensuring the appearance quality and consistency of the cabinet. Integrating the cabinet text comparison result, element type comparison result, and cabinet color comparison result into the visual detection result can provide users with comprehensive cabinet appearance detection information, facilitating users to quickly understand the actual situation of the cabinet.

[0039] According to some embodiments, the shooting process for multiple positions of the cabinet can be determined according to the appearance shooting method; according to the shooting process, the regional appearance images corresponding to each of the multiple positions can be determined; and according to the regional appearance images, the corresponding object detection models can be determined.

[0040] In this application, the appearance shooting method may include specific shooting steps for the cabinet, such as what position to shoot first, how many images to shoot at that position, then what position to shoot next, how many images to shoot, until the shooting is completed, and in what way to upload the pictures to the visual detection server, such as for different positions, first upload all the images of position A, and then upload the images of position B... These contents can be regarded as the shooting process as a whole.

[0041] Based on the shooting process, the situation of each image can be clarified. In some implementation manners, in combination with detection requirements, for example, detecting the images at each position separately, the appearance images of the switchgear cabinet can be divided to obtain regional appearance images. Then, the target detection model corresponding to the regional appearance image is searched for.

[0042] In this application, for the appearance images of different regions, a dedicated target detection model is selected or trained, which can more accurately identify the unique features and information of that region. The matching of this model with the region significantly improves the pertinence and accuracy of detection. The preset shooting process provides clear guidance for the appearance shooting of the switchgear cabinet, reducing the randomness and uncertainty during the shooting process. This not only ensures the consistency of image quality but also improves the shooting efficiency and reduces detection anxiety. This application is applicable to switchgear cabinets of different models, specifications, and uses. By adjusting the preset shooting method and selecting or training the corresponding target detection model, the detection requirements of various switchgear cabinets can be easily met.

[0043] According to some embodiments, the appearance images of the switchgear cabinet can be divided according to the appearance shooting method to determine the regional appearance images; the switchgear cabinet appearance elements included in the regional appearance images are obtained; and the corresponding target detection model is determined according to the switchgear cabinet appearance elements.

[0044] In this application, the division method in the appearance shooting method can be to divide the regional appearance images according to the switchgear cabinet appearance elements, so that each divided regional appearance image contains the same switchgear cabinet appearance elements to be detected.

[0045] In some implementation manners, according to the appearance shooting method, the appearance images of the switchgear cabinet are divided to obtain multiple groups of regional appearance images. Then, through image analysis, or by extracting the image and element corresponding requirements during the division from the appearance shooting method, the switchgear cabinet appearance elements included in the regional appearance images are determined. Then, according to the positions of the preset switchgear cabinet appearance elements on the switchgear cabinet, it can be determined which switchgear cabinet appearance elements are skewed, and then the corresponding target detection model is selected. For example, for elements such as the screen eyebrow and nameplate, whose positions on the switchgear cabinet are normal, YOLOX can be used as the target detection model; the position of the terminal row on the switchgear cabinet may be relatively distorted, and YOLOv7-OBB can be used as the target detection model.

[0046] Through the detailed division of the appearance images of the switchgear cabinet, this application can identify and focus on each key area of the switchgear cabinet, thereby improving the pertinence and accuracy of detection. Obtaining the specific switchgear cabinet appearance elements included in each regional image enables the target detection model to more accurately identify these elements, reducing false alarms and missed detections. Customizing or selecting a suitable target detection model according to the differences in the switchgear cabinet appearance elements improves the accuracy of the visual detection of the switchgear cabinet appearance.

[0047] According to some embodiments, the regional appearance image can be input into the corresponding target detection model to output the element box information of the corresponding cabinet appearance element; according to the element box information and the cabinet design information, determine the element type and the corresponding type comparison result; obtain the preset detection content of the cabinet appearance element to determine the visual detection direction according to the preset detection content and the element type; according to the visual detection direction, the element box information and the cabinet design information, perform corresponding detection on the regional appearance image to determine the cabinet text comparison result and the cabinet color comparison result; determine the cabinet text comparison result, the element type comparison result and the cabinet color comparison result as the visual detection result.

[0048] The element box information in this application can specifically be the vertex coordinates of the element box. In some implementation manners, the element box is a rectangle, with the image center as the origin, and the element box information can be the coordinates of the four vertices of the rectangle in the image. The image content in the element box can include the cabinet appearance element itself, the label of the cabinet appearance element, etc. There may be differences in the types of cabinet appearance elements and also in colors. In response to this situation, different detection contents can be preset, and then different visual detection directions can be correspondingly set. The visual detection direction can include the text detection direction and the cabinet color detection direction.

[0049] In some implementation manners, inputting the regional appearance image into the corresponding target detection model can output the element box information. The element box information can include the position information of the element box and the element type corresponding to the element box. Compare the element type with the element type at the corresponding position in the cabinet design information to obtain the type comparison result. Then, the preset detection content of the cabinet appearance element can be searched and obtained, and the corresponding matching visual detection direction can be searched from the preset detection content based on the element type. Then, on the visual detection direction, compare the element box information and the cabinet design information to obtain the cabinet text comparison result and the cabinet color comparison result. Integrate the type comparison result, the cabinet text comparison result and the cabinet color comparison result, and take the whole as the visual detection result, which can specifically include which comparison result of a certain cabinet appearance element passes or fails.

[0050] This application integrates a target detection model and a character recognition model. This method realizes the automatic recognition and detection of cabinet appearance elements, greatly improving the automation level of the detection process. Without manual intervention or manual operation, the detection result can be quickly obtained, thereby reducing the labor cost and improving the detection efficiency. Using the preset cabinet design information as a reference benchmark, the consistency between the actual appearance of the cabinet (including text, type and color) and the design requirements can be accurately compared. Through accurate positioning of the element box information and detailed setting of the visual detection direction, the accuracy and reliability of the detection result are ensured.

[0051] According to some embodiments, the visual detection directions include a text detection direction and a cabinet color detection direction. The regional appearance image can be cropped according to the element box information to determine an element sub-image; the element sub-image is input into a character recognition model to output corresponding element text information; according to the element text information and the cabinet design information, a cabinet text comparison result is determined; in the case where the visual detection direction includes the cabinet color detection direction, based on a preset HSV threshold segmentation method and a preset pixel point number threshold, the color information of the corresponding cabinet appearance element is determined, and according to the color information and the cabinet design information, a cabinet color comparison result is determined; the cabinet text comparison result and the cabinet color comparison result are determined as the visual detection result.

[0052] In this application, the HSV (Hue-Saturation-Value) threshold segmentation method is a method for segmenting an image based on the HSV color space. It can be preset that when the number of pixel points is greater than this threshold, it can be determined as the target color. In the specific implementation process, it can be set in combination with the color of the cabinet appearance element of the cabinet.

[0053] In some implementation manners, first, the positions of different cabinet appearance elements in the regional appearance image are determined according to the element box information, and then the image frames where the cabinet appearance elements are located are cropped, and each sub-image containing the cabinet appearance element is used as an element sub-image. The element sub-image is input into a character recognition model, and if there is text in the element sub-image, this text can be extracted.

[0054] Then, the text in the cabinet design information can be compared with the extracted text, that is, the element text information, to obtain a cabinet text comparison result.

[0055] If the visual detection direction also includes the cabinet color detection direction, the element sub-image can be segmented by using the HSV threshold segmentation method first. For each part after segmentation, the pixel point number threshold is used to determine the color information, and then comparison is performed to obtain a cabinet color comparison result.

[0056] This application uses element box information to crop the regional appearance image, effectively extracts the key element sub-images, reduces the interference of irrelevant information, and improves the accuracy of subsequent text recognition and color detection. By processing the element sub-images through a character recognition model, the text information on the cabinet can be accurately recognized and compared with the cabinet design information to ensure the consistency of the text content. Whether to perform cabinet color detection is selected according to the actual situation, increasing the flexibility of detection. When the color information is not a key detection index, only text detection can be performed to improve the detection efficiency. The application of the HSV threshold segmentation method enables color detection to accurately distinguish different colors, and the preset pixel point quantity threshold helps to further screen the effective color information and improve the reliability of color detection.

[0057] According to some embodiments, the cabinet appearance elements include labeled elements. When the cabinet appearance element is a labeled element, the corresponding element body box information, or the element body box information and the corresponding label box information can be extracted from the element text information; when the element text information only contains the element body box information, based on a preset perspective transformation method, the element box body information is rotationally corrected, and character segmentation is performed on the corrected element box information, and characters are extracted to determine the cabinet text comparison result according to the characters and the cabinet design information; when the element text information contains the element body box information and the corresponding label box information, the preset sorting method corresponding to the labeled element is obtained, and based on the sorting method and the cabinet design information, the label recognition information and the relative position information between the body and the label corresponding to the labeled element are determined to determine the cabinet text comparison result according to the label recognition information, the relative position information, and the cabinet design information.

[0058] In this application, the cabinet appearance elements can be divided into two categories. One category is labeled elements, and the other category is body elements. Labeled elements can be used to indicate that the element body and the label are separated, and the multiple element boxes output by the target detection model respectively correspond to the element body and the corresponding label. The body elements can be used to indicate that the label is set on the element body, and the multiple element boxes output by the target detection model correspond to the element body. At this time, the element body and the label overlap. Examples of labeled elements include air switches, pressure plates, terminal blocks, etc.

[0059] The sorting method can be preset. Taking the air switch as an example, the label boxes of the air switch are sorted from small to large according to the numerical value of the central abscissa in the label box information. The label recognition information can be used to characterize whether the detected label text is correct, the type, stage number, current intensity, etc. of the element body. The relative position information between the body and the label can be used to characterize whether the body and the label correspond.

[0060] In some implementations, if the cabinet appearance element is a tagged element, the element body box information can be extracted from the element text information, that is, the text information of the data box corresponding to the tagged element, or the element body box information and the corresponding tag box information can be extracted, that is, the text information of the data box corresponding to the tagged element and the text information of the data box of the tag corresponding to the tagged element.

[0061] If the element text information only contains the element body box information, that is, the tag box information cannot be extracted, at this time, the perspective transformation method can be used to rotate and correct the element box body information, and then character segmentation is performed on the corrected element box information. The segmented characters are extracted and compared with the cabinet design information to obtain the cabinet text comparison result.

[0062] If the element text information contains the element body box information and the corresponding tag box information, then obtain the preset sorting method of the tagged element, and then compare it with the cabinet design information one by one in the order obtained according to this sorting method to obtain the tag recognition information and the relative position information, and obtain the final cabinet text comparison result.

[0063] In this application, by distinguishing whether the cabinet appearance element is a tagged element and adopting different processing strategies according to the specific content of the element text information (only containing the element body box information or containing both the element body box information and the tag box information), the adaptability and flexibility of cabinet text comparison are enhanced. This fine-grained processing method can more accurately reflect the actual design situation of the cabinet and improve the accuracy of comparison. When the element text information only contains the element body box information, this technology rotates and corrects the element box body information through a preset perspective transformation method, and performs character segmentation and character extraction on the corrected element box information. This process effectively solves problems such as character distortion and overlap caused by factors such as shooting angle and light, and improves the accuracy and reliability of character recognition. For tagged elements containing element body box information and corresponding tag box information, this technology determines the tag recognition information and the relative position information between the body and the tag by obtaining the preset sorting method and the cabinet design information. This step not only simplifies the tag recognition process, but also improves the accuracy and efficiency of tag recognition by combining the cabinet design information, which helps to quickly and accurately complete the comparison of cabinet text.

[0064] According to some embodiments, when the visual detection direction includes the cabinet color detection direction, based on the HSV threshold segmentation method, the element subgraph can be segmented to determine multiple element color subgraphs; the number of pixel points corresponding to each of the multiple element color subgraphs is detected, and the color information is determined based on the number of pixel points and the pixel point number threshold; the actual color of the cabinet appearance element is extracted from the cabinet design information, and the color information is compared with the actual color to determine the cabinet color comparison result.

[0065] In some implementations, if the visual detection direction includes the cabinet color detection direction, according to the above embodiments, the HSV threshold segmentation method can be used to segment the element subgraph to obtain multiple color subgraphs. Then, the number of pixel points in each color subgraph is detected, and the number of pixel points is compared with the pixel point number threshold to correspondingly determine the color information.

[0066] The actual color of the corresponding cabinet appearance element is extracted from the cabinet design information and compared with the color information obtained above to obtain the cabinet color comparison result.

[0067] When the visual detection direction of this application includes cabinet color detection, through the preset HSV threshold segmentation method, the element subgraph is finely segmented to obtain multiple element color subgraphs representing different color regions. This segmentation method fully considers the distribution characteristics of colors in the HSV space, can more accurately identify the color information in the cabinet appearance elements, and avoids color confusion or misjudgment. By detecting the number of pixel points corresponding to each of the multiple element color subgraphs and based on the preset pixel point number threshold, the color information is further determined, enhancing the reliability of color comparison. At the same time, comparing the detected color information with the actual color in the cabinet design information can intuitively reflect whether the cabinet color meets the design requirements.

[0068] According to other embodiments, in the specific visual detection process, for the inspection of the cabinet brow and nameplate: the cabinet brow types are divided into stainless steel brushed cabinet brow (Brow_StainlessSteel) and ordinary steel plate cabinet brow (Brow_Steel), and the nameplate types are PVC nameplate (PanelPrint_PVC) and aluminum plate nameplate (PanelPrint_AL), which are the corresponding labels for the cabinet brow and nameplate. The front-of-cabinet image is input into the YOLOX detection model, and the element box information is output, including the body boxes of the cabinet brow and nameplate, and the label boxes corresponding to the types of the cabinet brow and nameplate. The element subgraph is cropped on the front-of-cabinet image according to the element box information and input into the optical character recognition model for OCR recognition, and the output text is compared with the text in the design drawing for inspection.

[0069] For the inspection of the quality tracking form: the classification label of the quality tracking form is TrackingForm. The front-of-cabinet image is input into the YOLOX detection model, and the element box with the classification label of TrackingForm is found from the output information, and the corresponding subgraph is cropped. The subgraph is input into the optical character recognition model for OCR recognition to obtain multiple lines of text. The label pattern of the quality tracking form is fixed, and the text prefixes are "specification", "color", "project name", "factory number". These text prefixes are extracted and compared with the water machine text in the design drawing for inspection.

[0070] Inspection of air switches: The label of the air switch is Label_AirSwitch. The types of air switches are divided into single-pole air switches AirSwitch_1P, two-pole air switches AirSwitch_2P, and three-pole air switches AirSwitch_3P. The number of poles of the air switch is already distinguished in the type. Input the image of the rear upper guide rail into the YOLOX detection model, find all air switch labels and the element boxes of the air switches in the output information, crop the sub-images, perform OCR recognition, extract the text, and record the number of poles of the air switches at the same time.

[0071] Sort the air switch labels in ascending order according to the abscissa of the center of the label rectangle. After sorting, judge whether the label text at each position is the same as the label text at the corresponding position in the drawing in sequence. Take out all the air switch labels with the same text match. According to the rule that the abscissa of the center of the body box of the air switch needs to be within the label box of the corresponding air switch label, find the air switch corresponding to each air switch label. Design regular expressions according to the type, number of poles, current intensity, and tripping characteristics of each air switch in the design drawing, and compare and check the extracted text.

[0072] Inspection of pressure plates: The label of the pressure plate is Label_Link. The types of pressure plates are divided into new-style pressure plates Link_XH17-2T / Z, old-style pressure plates Link_YY1-D1-A, double-link new-style pressure plates Link_XH49W4T-DKZ, and double-link old-style pressure plates Link_RSH2.5-2S. Input the image in front of the panel into the YOLOX detection model, find all the element boxes of the pressure plate labels and pressure plates of the corresponding types in the output information, and record the type of the pressure plate. If the pressure plate label is detected, it is identified as a pressure plate with a label. Crop the sub-image of the pressure plate label and the sub-image of the pressure plate, and input the sub-image of the pressure plate label into the optical character recognition model to extract the text.

[0073] For the sub-image of the pressure plate, based on the HSV threshold segmentation method, preset the HSV thresholds of red, yellow, blue, and green for segmentation, then count the number of pixel points to determine the color of the pressure plate. According to the rule that the abscissa of the center of the body box of the pressure plate needs to be within the label box of the corresponding pressure plate label, and the body box of the pressure plate needs to be above the pressure plate label, find the pressure plate corresponding to each pressure plate label. Classify the pressure plates with the center ordinate within the preset threshold range into one row to obtain multiple rows of pressure plates, and sort each row of pressure plates in ascending order according to the abscissa of the center of the body box, and judge whether the text of the pressure plates in each row and each column is consistent with the text of the pressure plates in the corresponding row and column in the design drawing. Then, take out the pressure plates with consistent text matching, and further check whether the color and type of the pressure plates are incorrect.

[0074] If the pressing plate label is not detected, extract the sub - figure of the pressing plate, further input the sub - figure of the pressing plate into the YOLOv7 - OBB detection model to obtain a rotated rectangle figure. Use the method of perspective transformation to correct the rotation of the image, then grayscale and binarize the image, and use horizontal projection and vertical projection to segment each character. Input the segmented character figure into the PaddleOCR model to extract text. Similarly, based on the method of HSV threshold segmentation, determine the color of the pressing plate. Obtain multiple rows of pressing plates in the above - mentioned manner, sort them, and conduct text comparison.

[0075] Regarding the detection of the terminal block: The type of the terminal block label is Label_Terminal. There are many types of terminal blocks, and its type naming rule is "terminal type + color". The terminal types include single - layer terminal Terminal_Single, double - layer terminal Terminal_Double, test terminal Terminal_Test, knife - type terminal Terminal_Knife, and the color types include gray Gray, red Red, blue Blue, yellow Yellow, green Green, and yellow - green earthing Earthing. Take all these information as the classification information of the terminal block. Input the images of the rear left guide rail, rear right guide rail, and rear lower guide rail into the YOLOv7 - OBB detection model respectively, and output the element frames corresponding to the terminal block and the terminal block label respectively. This element frame is rotated. According to the pre - set classification information of the terminal block, extract the color and type. Based on perspective transformation, extract the sub - figures of all terminal block labels, and input the sub - figures into the optical character recognition model to extract text.

[0076] For the terminal blocks and terminal block labels on the rear left and rear right guide rails, those with the center abscissa of the corresponding element frame within the preset threshold range are grouped into one column. In each column of terminal blocks and terminal block labels, for each terminal block label, count all the terminal blocks between this terminal block label and the next terminal block label, sort them in ascending order according to the center ordinate of the body frame of the terminal block, and determine these terminal blocks as the terminal blocks corresponding to this terminal block label.

[0077] For the terminal blocks and terminal block labels on the rear lower guide rail, those with the center ordinate of the corresponding element frame close to each other are grouped into one row. In each row of terminal blocks and terminal block labels, for each terminal block label, count all the terminal blocks between this terminal block label and the next terminal block label, sort them in ascending order according to the center abscissa of the body frame of the terminal block, and determine these terminal blocks as the terminal blocks corresponding to this terminal block label.

[0078] According to the design drawings, compare each label text and the type and color of the terminal block contained in the label.

[0079] Inspection of buttons, handles, and signal lights: The types of buttons are classified as Hongbo green button (Button_HB_Green), Hongbo red button (Button_HB_Red), Electrolux green button (Button_ELX_Green), Electrolux red button (Button_ELX_Red), and Electrolux gray button (Button_ELX_Gray); the types of handles are classified as handle without light and without lock (TrsSwitch_Unlock), handle without lock with circular light (TrsSwitch_Unlock_Circular), handle without lock with square light (TrsSwitch_Unlock_Square), handle with lock without light (TrsSwitch_Lock), handle with lock with circular light (TrsSwitch_Lock_Circular), and handle with lock with square light (TrsSwitch_Lock_Square); the types of signal lights are classified as red signal light (Lamp_Red), green signal light (Lamp_Green), and yellow signal light (Lamp_Yellow).

[0080] Input the front panel image into the YOLOX detection network, find all the element boxes and related information of buttons, handles, and signal lights from the output information, and extract and record the type and position in the related information. A gap determination template can be preset, and this template is used for image matching with the cabinet appearance image to find the installation gaps of the front left panel, front panel, and front right panel, and record the coordinates of the installation gaps. In some implementation manners, there are at least two installation gaps.

[0081] Classify the buttons, handles, and signal lights on the left side of the installation gap as the front left panel, those on the right side of the installation gap as the front right panel, and those in the middle of the installation gap as the front panel. According to the design drawings, check the types of buttons, handles, and signal lights in sequence for the front left panel, front panel, and front right panel.

[0082] Inspection of small busbars and earthing copper bars: The type classification rule of earthing copper bars is "insulated or not + region", insulated earthing copper bar (EarthingBar_Insulated), uninsulated earthing copper bar (EarthingBar_Uninsulated), and the region styles are standard style (Std), State Grid style (GW), Southern Power Grid style (NF), and Jibei style (BW).

[0083] Input the rear panel image into the YOLOX detection model, find all the element boxes of earthing copper bars of all types from the output information, count the models and the number of occurrences of all models based on the number of element boxes and the output information, and determine the copper bar style and whether it is insulated based on the output information, and compare it with the design drawings.

[0084] The types of small busbars are classified into small busbar empty terminals BusBar_TopTerminal, 6mm copper bar small busbars BusBar_6mm, 6mm tinned copper bar small busbars BusBar_Tinned_6mm, 8mm copper bar small busbars BusBar_8mm, and 8mm tinned copper bar small busbars BusBar_Tinned_8mm. The two categories for judging whether the small busbar area is insulated are: small busbar area non-insulated BusBars_Uninsulated and small busbar area insulated BusBars_Insulated.

[0085] Input the image of the top of the panel into the YOLOX detection model, find all the element frames of the type of small busbar from the output information, count the models and the number of occurrences of all models based on the number of element frames and the output information, and determine the number of empty terminals, the number of small busbars, and the styles of copper bars or tinned based on the output information, and then compare with the design drawings.

[0086] The device embodiments of the present application are described below, which can be used to execute the method embodiments of the present application. For the details not disclosed in the device embodiments of the present application, reference can be made to the method embodiments of the present application.

[0087] Figure 2 It is a block diagram of the visual detection device for the appearance of the panel cabinet provided by the embodiments of the present application. As Figure 2 shown, the visual detection device 200 for the appearance of the panel cabinet includes an image acquisition module 201, a model determination module 202, and a result determination module 203.

[0088] The image acquisition module 201 is used to acquire the image of the appearance of the panel cabinet to be detected;

[0089] The model determination module 202 is used to divide the image of the appearance of the panel cabinet according to the preset appearance shooting method to determine the regional appearance image, and determine the corresponding target detection model according to the regional appearance image;

[0090] The result determination module 203 is used to determine the panel cabinet text comparison result, type comparison result, and panel cabinet color comparison result based on the target detection model, preset character recognition model, regional appearance image, and preset panel cabinet design information, and determine the panel cabinet text comparison result, type comparison result, and panel cabinet color comparison result as the visual detection result.

[0091] Optionally, the model determination module 202 is specifically used for:

[0092] Determine the shooting process for multiple positions of the panel cabinet according to the appearance shooting method;

[0093] Determine the regional appearance image corresponding to each of the multiple positions according to the shooting process;

[0094] Determine the corresponding target detection model according to the regional appearance image.

[0095] Optionally, the model determination module 202 is specifically further configured to:

[0096] Divide the cabinet appearance image according to the appearance shooting method to determine the regional appearance image;

[0097] Obtain the cabinet appearance elements included in the regional appearance image;

[0098] Determine the corresponding target detection model according to the cabinet appearance elements.

[0099] Optionally, the result determination module 203 is specifically configured to:

[0100] Input the regional appearance image into the corresponding target detection model to output the element box information of the corresponding cabinet appearance elements;

[0101] Determine the element type and the corresponding type comparison result according to the element box information and the cabinet design information;

[0102] Obtain the preset detection content of the cabinet appearance elements to determine the visual detection direction according to the preset detection content and the element type;

[0103] Perform corresponding detection on the regional appearance image according to the visual detection direction, the element box information and the cabinet design information to determine the cabinet text comparison result and the cabinet color comparison result;

[0104] Determine the cabinet text comparison result, the type comparison result and the cabinet color comparison result as the visual detection result.

[0105] Optionally, the visual detection direction includes the text detection direction and the cabinet color detection direction; when the result determination module 203 performs corresponding detection on the regional appearance image according to the visual detection direction, the element box information and the cabinet design information to determine the cabinet text comparison result and the cabinet color comparison result, it is specifically configured to:

[0106] Crop the regional appearance image according to the element box information to determine the element sub-image;

[0107] Input the element sub-image into the character recognition model to output the corresponding element text information;

[0108] Determine the cabinet text comparison result according to the element text information and the cabinet design information;

[0109] In the case where the visual detection direction includes the cabinet color detection direction, based on a preset HSV threshold segmentation method and a preset pixel point quantity threshold, determine the color information of the corresponding cabinet appearance element, and determine the cabinet color comparison result according to the color information and the cabinet design information;

[0110] Determine the cabinet text comparison result and the cabinet color comparison result as the visual detection result.

[0111] Optionally, the cabinet appearance element includes a labeled element; when the result determination module 203 determines the cabinet text comparison result according to the element text information and the cabinet design information, it is specifically used for:

[0112] In the case where the cabinet appearance element is a labeled element, extract the corresponding element body frame information, or the element body frame information and the corresponding label frame information from the element text information;

[0113] In the case where the element text information only contains the element body frame information, based on a preset perspective transformation method, perform rotation correction on the element frame body information, perform character segmentation on the corrected element frame information, and extract characters, so as to determine the cabinet text comparison result according to the characters and the cabinet design information;

[0114] In the case where the element text information contains the element body frame information and the corresponding label frame information, obtain the preset sorting method corresponding to the labeled element, and determine the label recognition information and the relative position information between the body and the label corresponding to the labeled element based on the sorting method and the cabinet design information, so as to determine the cabinet text comparison result according to the label recognition information, the relative position information and the cabinet design information.

[0115] Optionally, when the result determination module 203 determines the color information of the corresponding cabinet appearance element based on a preset HSV threshold segmentation method and a preset pixel point quantity threshold in the case where the visual detection direction includes the cabinet color detection direction; and determines the cabinet color comparison result according to the color information and the cabinet design information, it is specifically used for:

[0116] In the case where the visual detection direction includes the cabinet color detection direction, based on the HSV threshold segmentation method, segment the element sub - image to determine multiple element color sub - images;

[0117] Detect the pixel point quantity corresponding to each of the multiple element color sub - images, and determine the color information based on the pixel point quantity and the pixel point quantity threshold;

[0118] Extract the actual color of the cabinet appearance element from the cabinet design information, and compare the color information with the actual color to determine the cabinet color comparison result.

[0119] The device performs a function similar to the method provided above. For other functions, please refer to the previous description and will not be elaborated here.

[0120] Figure 3 FIG. is a schematic structural diagram of an electronic device provided by an embodiment of the present application. As Figure 3 shown, the electronic device 300 of this embodiment may include: a memory 301 and a processor 302.

[0121] A computer program is stored on the memory 301. When the computer program is executed by the processor 302, the processor 302 is caused to execute the method in the above embodiment.

[0122] Among them, the processor 302 and the memory 301 are connected, such as through a bus.

[0123] Optionally, the electronic device 300 may further include a transceiver. It should be noted that in practical applications, the transceiver is not limited to one, and the structure of the electronic device 300 does not constitute a limitation on the embodiments of the present application.

[0124] The processor 302 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute various exemplary logic blocks, modules, and circuits described in combination with the disclosure of the present application. The processor 302 may also be a combination that implements computing functions, such as a combination including one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0125] The bus may include a path for transmitting information between the above components. The bus may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For the sake of simplicity, only a thick line is shown in the figure, but it does not mean that there is only one bus or one type of bus.

[0126] The memory 301 can be a ROM (Read Only Memory), or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory), or other types of dynamic storage devices that can store information and instructions. It can also be an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory), or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media, or other magnetic storage devices, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0127] The memory 301 is used to store the application program code for executing the solution of this application, and is controlled and executed by the processor 302. The processor 302 is used to execute the application program code stored in the memory 301 to implement the content shown in the foregoing method embodiments.

[0128] Among them, the electronic device includes but is not limited to: mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablet Computers), PMPs (Portable Multimedia Players), in-vehicle terminals (such as in-vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. It can also be a server, etc. Figure 3 The shown electronic device is only an example and should not bring any restrictions to the functions and usage scope of the embodiments of this application.

[0129] The electronic device of this embodiment can be used to execute the method of any of the foregoing embodiments, and its implementation principle and technical effects are similar, so details are not described herein again.

[0130] This application also provides a non-transitory computer-readable storage medium, on which computer-readable instructions are stored. When the foregoing instructions are executed by a processor, the processor is caused to execute the method in the foregoing embodiments.

[0131] Those of ordinary skill in the art can understand that all or part of the steps for implementing the foregoing method embodiments can be completed by hardware related to program instructions. The foregoing program can be stored in a non-transitory computer-readable storage medium. When this program is executed, it executes the steps including the foregoing method embodiments; and the foregoing storage medium includes: ROM, RAM, magnetic disks, or optical discs and other media that can store program codes.

[0132] The above has introduced the embodiments of the present application in detail. Specific examples are used herein to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application. At the same time, any changes or deformations made by those skilled in the art based on the idea of the present application, within the specific implementation manner and application scope of the present application, fall within the protection scope of the present application. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. A visual inspection method for the appearance of a screen cabinet, characterized in that: include: Obtain the appearance image of the screen cabinet to be inspected; According to a preset appearance shooting method, the appearance image of the screen cabinet is divided to determine a regional appearance image, and a corresponding target detection model is determined according to the regional appearance image; Based on the target detection model, the preset character recognition model, the regional appearance image and the preset screen cabinet design information, the screen cabinet text comparison results, the type comparison results and the screen cabinet color comparison results are determined, and the screen cabinet text comparison results, the type comparison results and the screen cabinet color comparison results are determined as visual detection results.

2. The method according to claim 1, characterized in that: The method of dividing the appearance image of the screen cabinet according to a preset appearance shooting method to determine a regional appearance image, and determining a corresponding target detection model according to the regional appearance image includes: According to the appearance shooting method, determine the shooting process for multiple positions of the screen cabinet; According to the shooting process, determining the regional appearance images corresponding to each of the multiple positions; Determine the corresponding target detection model according to the regional appearance image.

3. The method according to claim 1, characterized in that The method of dividing the appearance image of the screen cabinet according to a preset appearance shooting method to determine a regional appearance image, and determining a corresponding target detection model according to the regional appearance image includes: According to the appearance shooting method, the appearance image of the screen cabinet is divided to determine the regional appearance image; Obtaining the screen cabinet appearance elements corresponding to the regional appearance image; According to the appearance elements of the screen cabinet, a corresponding target detection model is determined.

4. The method according to claim 3, characterized in that The determining of the screen cabinet text comparison result, the type comparison result and the screen cabinet color comparison result based on the target detection model, the preset character recognition model, the regional appearance image and the preset screen cabinet design information, and determining the screen cabinet text comparison result, the type comparison result and the screen cabinet color comparison result as the visual detection result includes: Inputting the regional appearance image into the corresponding target detection model to output the element frame information of the corresponding screen cabinet appearance element; Determine the element type and the corresponding type comparison result according to the element frame information and the screen cabinet design information; Acquire preset detection content of the appearance element of the screen cabinet to determine the visual detection direction according to the preset detection content and the element type; According to the visual detection direction, the element frame information and the screen cabinet design information, corresponding detection is performed on the regional appearance image to determine the screen cabinet text comparison result and the screen cabinet color comparison result; The screen cabinet text comparison result, the type comparison result and the screen cabinet color comparison result are determined as the visual inspection result.

5. The method according to claim 4, characterized in that The visual detection direction includes a text detection direction and a screen cabinet color detection direction; The corresponding detection of the regional appearance image is performed according to the visual detection direction, the element frame information and the screen cabinet design information to determine the screen cabinet text comparison result and the screen cabinet color comparison result, including: According to the element frame information, the regional appearance image is cropped to determine an element sub-image; Inputting the element subgraph into the character recognition model to output corresponding element text information; Determine the screen cabinet text comparison result according to the element text information and the screen cabinet design information; In the case where the visual detection direction includes the screen cabinet color detection direction, based on a preset HSV threshold segmentation method and a preset pixel number threshold, the color information of the corresponding screen cabinet appearance element is determined, and the screen cabinet color comparison result is determined according to the color information and the screen cabinet design information; The screen cabinet text comparison result and the screen cabinet color comparison result are determined as the visual inspection result.

6. The method according to claim 5, characterized in that The screen cabinet appearance elements include label elements; Wherein, determining the screen cabinet text comparison result according to the element text information and the screen cabinet design information includes: In the case where the cabinet appearance element is the labeled element, extracting corresponding element body frame information, or the element body frame information and corresponding label frame information from the element text information; In the case where the element text information only includes the element body frame information, the element frame body information is rotationally corrected based on a preset perspective transformation method, and character segmentation is performed on the corrected element frame information, and characters are extracted to determine the cabinet text comparison result based on the characters and the cabinet design information; In the case that the element text information includes the element body frame information and the corresponding label frame information, a preset sorting method corresponding to the labeled element is obtained, and based on the sorting method and the screen cabinet design information, the label identification information and the relative position information between the body and the label corresponding to the labeled element are determined, so as to determine the screen cabinet text comparison result according to the label identification information, the relative position information and the screen cabinet design information.

7. The method according to claim 5, characterized in that In the case where the visual detection direction includes the screen cabinet color detection direction, based on a preset HSV threshold segmentation method and a preset pixel number threshold, the color information of the corresponding screen cabinet appearance element is determined, and according to the color information and the screen cabinet design information, the screen cabinet color comparison result is determined, including: In the case where the visual detection direction includes the cabinet color detection direction, segmenting the element sub-image based on the HSV threshold segmentation method to determine a plurality of element color sub-images; Detecting the number of pixels corresponding to each of the plurality of element color sub-images, and determining the color information based on the number of pixels and the pixel number threshold; The actual color of the appearance elements of the screen cabinet is extracted from the screen cabinet design information, and the color information is compared with the actual color to determine the color comparison result of the screen cabinet.

8. A visual inspection device for the appearance of a screen cabinet, characterized in that: include: An image acquisition module is used to acquire an appearance image of the screen cabinet to be inspected; A model determination module, used to divide the appearance image of the screen cabinet according to a preset appearance shooting method to determine a regional appearance image, and determine a corresponding target detection model according to the regional appearance image; A result determination module is used to determine the screen cabinet text comparison result, type comparison result and screen cabinet color comparison result based on the target detection model, the preset character recognition model, the regional appearance image and the preset screen cabinet design information, and determine the screen cabinet text comparison result, the type comparison result and the screen cabinet color comparison result as the visual detection result.

9. An electronic device, characterized in that: include: processor; A memory storing a computer program, which, when executed by the processor, enables the processor to execute the visual inspection method for the appearance of the screen cabinet as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium, characterized in that: Computer-readable instructions are stored thereon, and when the instructions are executed by a processor, the processor executes the visual inspection method for the appearance of a screen cabinet as described in any one of claims 1-7.

Images