Switch cabinet inspection method, device, system and electronic equipment

Abstract

The application discloses a kind of switch cabinet inspection method, device, system and electronic equipment, it is related to switch cabinet technical field.The method comprises the following steps: obtaining the first switch cabinet panel image of first switch cabinet, detecting first switch cabinet panel image, and obtaining the detection frame of each first switch in first switch cabinet panel image;First switch cabinet panel image is input into key point detection model pre-trained according to target number sequence, and the first key point and first number of each first switch are obtained;First key point is matched with detection frame, and first matching result is obtained, and the second matching result of first number and detection frame is obtained according to first matching result;According to the second matching result, the inspection result of first switch cabinet is obtained.Therefore, the problem that the number sequence of detection frame is inconsistent with the required sequence is avoided, and high-precision switch cabinet inspection is realized.

Description

Technical Field

[0001] This invention relates to the field of switchgear technology, and in particular to a switchgear inspection method, device, system and electronic equipment. Background Technology

[0002] In related technologies, the automatic inspection method using a sliding rail robot is commonly used when inspecting switchgear. This method involves a sliding rail robot equipped with a camera moving to a preset position to capture images of the switchgear panel. Image recognition algorithms are then used to determine the status of the switches within the switchgear, resulting in a switch status identification result, and finally, the inspection result.

[0003] However, due to cost considerations, the methods for obtaining switch status in related technologies are difficult to implement using high-precision sliding rail robots with high-precision sliding rails and positioning systems. For example, to reduce costs, sliding rail robots equipped with gimbal cameras are used. This makes it easy for the shooting angle and the spatial position of the camera device to deviate when using the sliding rail robot to capture images of the switch cabinet. As a result, after obtaining the detection frames of the switches in the switch cabinet image, the numbering order of the detection frames is often inconsistent with the order in the switch cabinet template. This leads to matching errors when matching the identified switches with the switch cabinet template based on the numbers, resulting in incorrect inspection results. Summary of the Invention

[0004] This invention aims to at least partially solve one of the technical problems in related technologies. Therefore, the first objective of this invention is to provide a switchgear inspection method to avoid inconsistencies between the numbering order of the inspection boxes and the required order, thereby achieving high-precision switchgear inspection.

[0005] The second objective of this invention is to provide an electronic device.

[0006] The third objective of this invention is to provide a switch cabinet inspection device.

[0007] The fourth objective of this invention is to provide a switch cabinet inspection system.

[0008] To achieve the above objectives, a first aspect of the present invention provides a method for inspecting a switchgear. The method includes: acquiring a first switchgear panel image of a first switchgear; detecting the first switchgear panel image to obtain a detection box for each first switch in the first switchgear panel image; inputting the first switchgear panel image into a key point detection model pre-trained according to target number order to obtain a first key point and a first number for each first switch; matching the first key point with the detection box to obtain a first matching result, and obtaining a second matching result between the first number and the detection box based on the first matching result; and obtaining an inspection result for the first switchgear based on the second matching result.

[0009] In addition, the switchgear inspection method according to embodiments of the present invention may also have the following additional technical features: According to an embodiment of the present invention, the training method of the key point detection model includes: acquiring a second switch cabinet panel image of a second switch cabinet, and acquiring a switch detection box label for each second switch in the second switch cabinet panel image, wherein the switch detection box label includes the bounding box coordinates and a second number of the second switch, the second number being a number obtained according to the position of the second switch in the second switch cabinet panel image and the target numbering order; for each second switch, obtaining a second key point corresponding to the second switch according to the bounding box coordinates of the second switch; and using the second key point and the second number to train the model to obtain the key point detection model.

[0010] According to one embodiment of the present invention, obtaining the second key point corresponding to the second switch based on the bounding box coordinates of the second switch includes: obtaining the center point of the bounding box corresponding to the second switch based on the bounding box coordinates, and using the center point as the second key point corresponding to the second switch.

[0011] According to one embodiment of the present invention, the step of training a model using the second keypoint and the second number includes: obtaining the number of the second keypoints; when the number of the second keypoints is equal to a preset number, training a model using the second keypoints and the second number; when the number of the second keypoints is less than the preset number, adding multiple third keypoints, and training a model using the second keypoints, the third keypoints, and the second number, wherein the number of the third keypoints is equal to the difference between the preset number and the number of the second keypoints; when the number of the second keypoints is greater than the preset number, obtaining multiple keypoint groups based on the second keypoints, and training a model using the multiple keypoint groups and the second number; wherein the number of keypoints in each keypoint group is equal to the preset number.

[0012] According to one embodiment of the present invention, matching the first key point with the detection box includes: for each first key point, obtaining a first distance between the first key point and the center point of each detection box; for each first key point, obtaining a second distance between the first key point and the boundary of each detection box; and obtaining a matching result between the key point and the detection box based on the first distance and the second distance.

[0013] According to one embodiment of the present invention, obtaining the matching result between the key point and the detection box based on the first distance and the second distance includes: obtaining a direct distance cost matrix based on the first distance and obtaining an inclusion relation cost matrix based on the second distance; obtaining a combined cost matrix based on the direct distance cost matrix and the inclusion relation cost matrix; and obtaining the matching result based on the combined cost matrix.

[0014] According to an embodiment of the present invention, the combined cost matrix is ​​obtained according to the following formula: , in, The combined cost matrix is... As the first preset weight, For normalization, Let be the direct distance cost matrix. As the second preset weight, The inclusion relation cost matrix is... .

[0015] To achieve the above objectives, a second aspect of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executed on the processor, wherein when the computer program is executed by the processor, it implements the above-described switch cabinet inspection method.

[0016] To achieve the above objectives, a third aspect of the present invention provides a switchgear inspection device, comprising: a first acquisition module for acquiring a first switchgear panel image of a first switchgear; a detection module for detecting the first switchgear panel image to obtain a detection frame for each first switch in the first switchgear panel image; an input module for inputting the first switchgear panel image into a key point detection model pre-trained according to target number order to obtain a first key point and a first number for each first switch; a matching module for matching the first key point with the detection frame to obtain a first matching result, and obtaining a second matching result between the first number and the detection frame based on the first matching result; and a second acquisition module for obtaining the inspection result of the first switchgear based on the second matching result.

[0017] To achieve the above objectives, a fourth aspect of the present invention provides a switch cabinet inspection system, including the switch cabinet inspection device described above.

[0018] According to embodiments of the present invention, a switchgear inspection method, apparatus, system, and electronic device acquire a first switchgear panel image, detect the first switchgear panel image to obtain a detection box for each first switch in the first switchgear panel image; input the first switchgear panel image into a key point detection model pre-trained according to a target numbering order to obtain a first key point and a first number for each first switch; match the first key point with the detection box to obtain a first matching result, and obtain a second matching result between the first number and the detection box based on the first matching result; and obtain the inspection result of the first switchgear based on the second matching result. Thus, by obtaining the first number through a key point detection model trained according to the target numbering order and matching the first number with the detection box, the problem of inconsistent detection box numbering order with the required order is avoided, achieving high-precision switchgear inspection.

[0019] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0020] Figure 1 This is a flowchart of the switchgear inspection method according to an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating the target numbering order of an example of the present invention; Figure 3 This is a flowchart of an example of a switchgear inspection method according to the present invention; Figure 4 This is a flowchart of another example of a switchgear inspection method according to the present invention; Figure 5This is a flowchart of another example of a switchgear inspection method according to the present invention; Figure 6 This is a flowchart of another example of a switchgear inspection method according to the present invention; Figure 7 This is a flowchart of another example of a switchgear inspection method according to the present invention; Figure 8 This is a flowchart of another example of a switchgear inspection method according to the present invention; Figure 9 This is a flowchart of another example of a switchgear inspection method according to the present invention; Figure 10 This is a structural block diagram of an electronic device according to an embodiment of the present invention; Figure 11 This is a structural block diagram of the switchgear inspection device according to an embodiment of the present invention; Figure 12 This is a structural block diagram of the switchgear inspection system according to an embodiment of the present invention. Detailed Implementation

[0021] The following description, with reference to the accompanying drawings, describes a switchgear inspection method, apparatus, system, and electronic equipment according to embodiments of the present invention. Throughout the description, the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions. The embodiments described with reference to the accompanying drawings are exemplary and should not be construed as limiting the present invention.

[0022] Figure 1 This is a flowchart of the switchgear inspection method according to an embodiment of the present invention.

[0023] like Figure 1 As shown, the switchgear inspection method includes: S101, acquire the first switch cabinet panel image of the first switch cabinet, detect the first switch cabinet panel image, and obtain the detection box of each first switch in the first switch cabinet panel image.

[0024] Specifically, the camera can be adjusted to various preset shooting positions, each shooting position corresponding to a switch cabinet panel or a specific area of ​​the panel. Then, the application terminal controls the camera to collect image data and obtain the first switch cabinet panel image.

[0025] The camera type can be a fixed camera or an adjustable-angle PTZ camera. The number of preset positions is set according to the number and layout of the switch cabinets. The image resolution is recommended to be 1920×1080 pixels or higher. The acquisition environment is an indoor environment with controllable lighting conditions.

[0026] After obtaining the image of the first switchgear panel, a switch status detection algorithm is used to detect the switch status of all first switches in the image. The switch status detection result is a list of detection boxes, each containing location coordinates.

[0027] The aforementioned position coordinates include the horizontal and vertical coordinates of the vertices of the detection box. For example, for each detection box, its position coordinates include the horizontal and vertical coordinates of the top left corner and the bottom right corner.

[0028] S102, input the image of the first switch cabinet panel into the key point detection model that has been trained in advance according to the target number order to obtain the first key point and the first number of each first switch.

[0029] Specifically, the model is pre-trained according to the target numbering order to obtain a key point detection model that can number the switch cabinets in the switch cabinet image according to the target numbering order.

[0030] The training method for the aforementioned key point detection model includes: acquiring an image of the second switch cabinet panel of the second switch cabinet, and acquiring a switch detection box label for each second switch in the second switch cabinet panel image, wherein the switch detection box label includes the bounding box coordinates and a second number of the second switch, the second number being a number obtained according to the position of the second switch in the second switch cabinet panel image and the order of target numbers; for each second switch, obtaining a second key point corresponding to the second switch based on the bounding box coordinates of the second switch; and training the model using the second key point and the second number to obtain the key point detection model.

[0031] The method for obtaining the image of the second switchgear panel can be the same as the method for obtaining the image of the first switchgear panel.

[0032] After obtaining the image of the second switchgear panel, all the second switches in the image can be numbered according to their target numbering order. This target numbering order must be consistent with the order of subsequent switchgear templates. (See [link to relevant documentation]). Figure 2 In the example shown, the switches in the switch cabinet template are numbered in a horizontal-to-vertical order. That is, first, the first row of switches to be numbered is determined according to the direction pointed to by the arrow with number 1. Then, the switches in the first row are numbered according to the direction pointed to by the arrow with number 2. The first switch in the first row is numbered 0, the second switch in the first row is numbered 1, the third switch in the first row is numbered 2, and so on. After the switches in the first row are numbered, the switches in the second row are determined according to the direction pointed to by the arrow with number 1, and so on, until all the second switches in the second switch cabinet panel image are numbered. Therefore, the numbering order used when numbering the second switch cabinet panel image must also be the horizontal-to-vertical order.

[0033] Furthermore, it is necessary to generate the bounding box for each second switch in the second switch cabinet panel image, and then obtain the bounding box coordinates.

[0034] Furthermore, for each second switch, a second key point corresponding to the second switch is obtained based on the bounding box coordinates of the second switch. Obtaining the second key point corresponding to the second switch based on the bounding box coordinates includes: obtaining the center point of the bounding box corresponding to the second switch based on the bounding box coordinates, and using the center point as the second key point corresponding to the second switch.

[0035] After obtaining the bounding box coordinates, for each bounding box, the coordinates of the bounding box center point are obtained based on the bounding box coordinates. Specifically, the x-coordinate of the top-left vertex and the x-coordinate of the bottom-right vertex of the bounding box can be used as the x-coordinate of the bounding box center point, and the y-coordinate of the top-left vertex and the bottom-right vertex of the bounding box can be used as the y-coordinate of the bounding box center point.

[0036] Furthermore, each second key point needs to have its visibility annotation, that is, whether it is visible or not. For the above second key points, the default visibility annotation is visible, and the annotation 1 can be used to indicate that the visibility annotation is visible.

[0037] Furthermore, the above-mentioned model training is performed using the second keypoint and the second number.

[0038] The above-mentioned model training using the second keypoint and the second number includes: obtaining the number of second keypoints; when the number of second keypoints equals a preset number, training the model using the second keypoints and the second number; when the number of second keypoints is less than the preset number, adding multiple third keypoints, and training the model using the second keypoints, the third keypoints, and the second number, wherein the number of third keypoints equals the difference between the preset number and the number of second keypoints; when the number of second keypoints is greater than the preset number, obtaining multiple keypoint groups based on the second keypoints, and training the model using the multiple keypoint groups and the second number; wherein the number of keypoints in each keypoint group is equal to the preset number.

[0039] Specifically, after obtaining the second key point, the second number, and the visible annotation, an initial key point training dataset can be constructed based on the second key point, the second number, the visible annotation, and the second switch cabinet panel image.

[0040] To construct the initial keypoint training dataset, it is necessary to first obtain the number of second keypoints, and then compare the number of second keypoints with the preset number.

[0041] See Figure 3The example shown first obtains the initial keypoint dataset. Since there are multiple images of the second switch cabinet panel, and each second switch cabinet panel image has at least one second keypoint, it is necessary to traverse the keypoints of each image.

[0042] The number N of the second key points is obtained. i Then, the number N of the second key points. i With the preset quantity N std Comparison. In Figure 3 In the example shown, the preset quantity N std It is 20.

[0043] The preset quantity N i These are the parameters of the key point detection model mentioned above.

[0044] Specifically, since there are various types of switchgear, and each type has a different number of switches, in order to ensure that the above key point detection model can detect switchgear with different numbers of switches, the number of the first key points output by the above key point detection model at one time is set to a fixed value, and this fixed value is the preset number N. std .

[0045] Therefore, assuming the number of keypoints output by the keypoint detection model in a single output is a preset number Nstd, in order to reduce the difficulty of the model learning keypoints, ensure that the model can fit correctly, and solve the problem that some keypoints may be floating in unknown positions due to an excessive number of keypoints, the number of keypoints used in a single input during training is also a preset number N. std .

[0046] When the number of the above second key points is N i Equal to the preset quantity N std At that time, the original data can be used directly for training.

[0047] When the number of the above second key points is N i Less than the preset number N std When this happens, interpolation and zero-padding must be performed.

[0048] The above interpolation zero-padding process is to add a preset number N. std Subtract the number of second key points and the number of third key points to get the number of zeros padded, N0.

[0049] The third key point mentioned above is a virtual key point. Its horizontal and vertical coordinates can both be set to 0, and the visible label can be set to 0, that is, the confidence level can be set to 0, making it invisible.

[0050] Furthermore, the category of the third key point is set as the default category. Then, the initial keypoint training dataset was constructed using the second keypoint, the third keypoint, the second number, the visible annotation, and the second switch cabinet panel image.

[0051] When the number of second key points N i Greater than the preset number N std At that time, a truncation and bisection process is performed, which results in multiple keypoint groups based on the second keypoint.

[0052] The above-mentioned multiple keypoint groups obtained based on the second keypoint can be achieved by equally dividing the second keypoint to obtain at least two initial keypoint groups. The keypoints in each initial keypoint group are the representative keypoints in the corresponding keypoint group, and the number N of the second keypoints in each initial keypoint group is... i All are less than or equal to the preset quantity N std Then, for each initial keypoint group, a preset number N is added to that initial keypoint group. std Subtract the number of second keypoints and third keypoints from the initial keypoint group to obtain the keypoint group.

[0053] Furthermore, before grouping the second key points, the second key points can be sorted according to their ordinates before being grouped.

[0054] After obtaining the key point group, an initial key point training dataset is constructed based on the key point group, the second number, the visible annotation, and the second switch cabinet panel image.

[0055] After obtaining the initial keypoint training dataset, use the initial keypoint training dataset as the original training data for training. The training method can be seen in the example below.

[0056] First step, see Figure 4 Data augmentation was performed on the initial keypoint training dataset, including image geometric transformation enhancement (random rotation ±15°, random flip probability 0.5, random scaling 0.8~1.2 times, random cropping), color transformation enhancement (brightness ±20%, contrast 0.8~1.2, saturation 0.8~1.2, grayscale probability 0.1), and keypoint-specific enhancement (coordinate noise addition σ=0.5~1.0 pixel Gaussian noise, occlusion simulation 5%~20%, random loss probability 0.05).

[0057] Thus, augmented training data can be obtained, which can then be used for model training.

[0058] The second step involves training a keypoint detection algorithm (such as YOLO-pose series, HRNet, SimpleBaseline, etc.). The training parameters include the backbone network CSPDarknet, input size 640×640, batch size 16~32, training epochs 300~500, optimizer SGD (momentum 0.9, weight decay 0.0005), initial learning rate 0.01, cosine annealing decay to 0.0001, number of keypoints 20, OKS threshold 0.5~0.7, and affinity field range 8~16 pixels.

[0059] The third step is to train and output the keypoint detection model, using a pre-training + fine-tuning strategy, multi-scale training, hot start (the learning rate is increased from 0.001 to 0.01 in the first 5 rounds), and early stopping (the model stops after 20 rounds of verification that the loss does not decrease). The verification metrics include the average precision and recall rate of keypoints.

[0060] Therefore, a keypoint detection model can be trained.

[0061] The following is combined with Figure 5 The example shown illustrates this.

[0062] exist Figure 5 The example shown includes steps S1 (camera image acquisition), S2 (key point data annotation), S3 (data adaptive adjustment), and S4 (model training). Step S1 includes steps S11 (adjusting the shooting position), S12 (corresponding to the switch cabinet panel), and S13 (acquiring image data); step S2 includes step S21 (marking horizontally then vertically in sequence). Figure 2 (The target numbering order is shown), step S22 extracts the detection center point, step S23 constructs the training dataset; step S3 includes step S32 direct use, step S33 interpolation and zero-padding, step S34 truncation and equal division, step S35 summary processing, step S4 includes step S41 data augmentation, step S42 YOLO-pose training, step S43 output detection model.

[0063] Before step S1, it is necessary to select the camera to be used for shooting, which can be a fixed camera or a pan-tilt camera.

[0064] Step S12 is to point the camera at the panel of the second switch cabinet.

[0065] In step S1, the image resolution is set to 1920×1080+.

[0066] Figure 5 In the keypoint labels x, y, c, x is the x-coordinate of the second keypoint, y is the y-coordinate of the second keypoint, and c is the visible label of the second keypoint.

[0067] The construction of the training dataset in step S23 is to obtain the dataset based on the coordinates of the second key point, the visible annotation of the second key point, and the second number. After subsequent adaptive processing, the initial key point training dataset can be obtained.

[0068] The detection model output in step S43 is the key point detection model mentioned above.

[0069] After obtaining the image of the first switch cabinet panel, the image is input into the key point detection model to obtain the detection results of the first switch in the image. The detection results include 20 fourth key points and their corresponding visible annotations, as well as the first number.

[0070] After obtaining the detection results, the standardized fourth keypoint output by the model is restored to the actual first keypoint.

[0071] Specifically, the number of fourth keypoints output by the keypoint detection model at one time is fixed at the aforementioned preset number N. std The preset quantity N std Each keypoint includes the first keypoint marked as visible and the virtual keypoints marked as invisible. Therefore, it is necessary to filter or mark the virtual keypoints. Specifically, for virtual keypoints obtained by interpolation and zero-padding, their indices are recorded and excluded during restoration; for virtual keypoints obtained by truncation and equal division, they are restored in reverse according to the original grouping relationship.

[0072] The number of the first key points output after restoration is the same as the number of the first switches.

[0073] S103, match the first key point with the detection box to obtain the first matching result, and obtain the second matching result of the first number and the detection box based on the first matching result.

[0074] The above-mentioned matching of the first key point with the detection box includes: for each first key point, obtaining a first distance between the first key point and the center point of each detection box; for each first key point, obtaining a second distance between the first key point and the boundary of each detection box; and obtaining the matching result of the key point and the detection box based on the first distance and the second distance.

[0075] The matching results of key points and detection boxes obtained based on the first distance and the second distance include: obtaining a direct distance cost matrix based on the first distance and an inclusion relation cost matrix based on the second distance; obtaining a combined cost matrix based on the direct distance cost matrix and the inclusion relation cost matrix; and obtaining the matching result based on the combined cost matrix.

[0076] Specifically, see Figure 6First, obtain the list of detection boxes and the list of restored detection boxes. The list of detection boxes is composed of the fourth key point mentioned above, and the list of restored detection boxes is composed of the first key point mentioned above.

[0077] After obtaining the first key point, the center point of the detection box is obtained.

[0078] The method for calculating the coordinates of the center point of the detection frame mentioned above is as follows: , .

[0079] After obtaining the center point of the detection frame, a list of device center points is obtained, which is a list composed of the center points of the detection frame corresponding to the first switch.

[0080] Once the list of device center points is obtained, the direct distance cost matrix can be calculated.

[0081] To calculate the direct distance cost matrix, the distance is first calculated using the following formula. : , in, Let be the first distance between the i-th first keypoint and the center point of the j-th detection box. Let x be the x-coordinate of the i-th first key point. Let be the ordinate of the i-th first key point. Let x be the x-coordinate of the center point of the j-th detection box. Let be the ordinate of the center point of the j-th detection box. Let x be the x-coordinate of the top-left vertex of the j-th detection box. Let be the ordinate of the top-left vertex of the j-th detection box. Let x be the x-coordinate of the bottom right vertex of the j-th detection box. Let be the ordinate of the bottom right vertex of the j-th detection box.

[0082] After obtaining all distances The direct distance cost matrix can then be obtained. The direct distance cost matrix It can be used to represent the cost based on the distance between a first keypoint and the center point of a detection box.

[0083] After obtaining the direct distance cost matrix Then, the inclusion relation cost matrix is ​​calculated.

[0084] To calculate the inclusion cost matrix, it is necessary to first determine whether the keypoint is inside the bounding box, and then obtain a second distance between the first keypoint and the boundary of each detection box based on whether the keypoint is inside the bounding box.

[0085] The above method of obtaining the second distance between a first key point and the boundary of each detection box can be achieved by determining whether the first key point is inside the detection box for each detection box. If the key point is inside the detection box, the minimum distance from the point to each side of the detection box is calculated and normalized. If the key point is outside the detection box, the shortest distance from the point to the detection box is calculated.

[0086] Then, the inclusion cost can be obtained based on the distance between the first keypoint and the boundary of the detection box, and the inclusion cost matrix mentioned above can be constructed based on the inclusion cost. The inclusion cost matrix above can represent the cost required to include a first keypoint within a detection box, that is, the extent to which a first keypoint belongs to a detection box.

[0087] Then, the combined cost matrix can be calculated.

[0088] The combined cost matrix can be obtained from the following formula: , in, For the combined cost matrix, As the first preset weight, For normalization, The direct distance cost matrix, As the second preset weight, This is the inclusion relation cost matrix. For example, It is 0.6. It is 0.4.

[0089] After obtaining the combined cost matrix, the optimal matching of the detection box and the first key point can be solved using the Jawley algorithm to obtain a list of optimal matching pairs.

[0090] Once the optimal list of matching pairs is obtained, matching filtering can be performed.

[0091] Specifically, matching pairs with matching costs exceeding a preset threshold (typically 0.4-0.5) are excluded, and matching pairs of virtual keypoints with a confidence level of 0 (which is the visible annotation mentioned above) are discarded. If a match is successful, the detection box result corresponding to the keypoint is set as the matching box; if a match fails, the switch state is output as unknown, ensuring the fixed output order of keypoints and preventing incorrect output of detection boxes in the correct order.

[0092] Therefore, the first matching result can be obtained.

[0093] After obtaining the first matching result, since the first key point and the first number are in one-to-one correspondence, if a certain first key point matches a certain detection box, then the first number corresponding to the first key point matches the detection box.

[0094] For detailed procedures, please refer to [link / reference]. Figure 7 The example shown includes steps S5 (model inference), S6 (cost matrix matching), and S7 (result output). Step S5 (model inference) includes steps S51 (state detection), S52 (keypoint detection), and S54 (keypoint reconstruction). Step S6 (cost matrix matching) includes steps S61 (calculating the center point of the detection box), S62 (direct distance cost matrix), S63 (inclusion relation cost matrix), S64 (combined cost matrix), S65 (Hungarian algorithm solution), and S66 (matching filtering).

[0095] Step S51 involves detecting the first switch cabinet panel image as described above to obtain the detection frame for each first switch in the first switch cabinet panel image.

[0096] Step S52 involves detecting the fourth keypoint, and step S54 involves obtaining the first keypoint based on the fourth keypoint. Step S62 involves calculating the direct distance cost matrix, step S63 involves calculating the inclusion relationship cost matrix, and step S64 involves calculating the combined cost matrix.

[0097] The above process can be found in [reference]. Figure 8 ,in, Figure 8 Step S1, image acquisition, is... Figure 5 In step S1, the camera captures images. Figure 8 The key point annotation in step S2 is... Figure 5 Step S2: Keypoint data annotation.

[0098] S104. Obtain the inspection results of the first switchgear based on the second matching result.

[0099] Specifically, after obtaining the first number corresponding to each detection box, the image in the detection box is matched with the switch cabinet template according to the first number to obtain the switching status recognition result of the first switch in the detection box.

[0100] When the above steps are used for switch cabinet inspection, the inspection results of the switch cabinet can be obtained based on the switch status identification results.

[0101] For details on the data flow, please refer to [link / reference]. Figure 9 The example shown. Figure 9 The camera capture in this context refers to the image of the second switch cabinet panel captured by the camera. Figure 9 The data collection in this process is the same as step S1 described above. Figure 9 The annotation in the text refers to step S2 above. Figure 9 The adaptive processing in this process is step S3 mentioned above. Figure 9 The model training in this process is the same as step S4 described above. Figure 9 The new image in the image is the aforementioned image of the first switch cabinet panel. Figure 9The reasoning in this context is the same as step S5 described above. Figure 9 The restoration in the above process is step S54. Figure 9 The matching in the above is step S6. Figure 9 The result output is the one described in step S7 above. Figure 9 The state detection result in the above detection box is the result of the state detection in the image. Figure 9 The key point detection result is the fourth key point mentioned above.

[0102] In some embodiments of the present invention, a sliding window temporal smoothing method can also be used. That is, for a switch cabinet, after capturing multiple consecutive frames of first switch cabinet panel images, a sliding average smoothing process is performed on the first key points detected based on the multiple consecutive frames of first switch cabinet panel images. Specifically, for the x-coordinate of the i-th first key point in the t-th frame of the first switch cabinet panel image, the sliding average smoothing process can be performed using the following formula: , in This is a smoothing coefficient, with a value ranging from 0.6 to 0.8. The x-axis is after smoothing. Let x be the x-coordinate of the i-th first key point in the first switch cabinet panel image of frame t. Let x be the x-coordinate of the i-th first key point in the first switch cabinet panel image of frame t-1.

[0103] For processing of the vertical axis, please refer to the above explanation of processing the horizontal axis.

[0104] After performing sliding window temporal smoothing on the horizontal and vertical coordinates, the first key point after sliding window temporal smoothing is matched with the detection box.

[0105] In some embodiments of the present invention, a flying point filtering mechanism may also be employed, that is, for a switching transistor, after capturing the first switch cabinet panel image in the t-th frame, the Euclidean distance of the first key point in the first switch cabinet panel image in the t-th frame is calculated according to the following formula: , in, Let be the Euclidean distance to the i-th first key point. Let be the ordinate of the i-th first key point in the first switch cabinet panel image of frame t. Let y be the ordinate of the i-th first key point in the first switch cabinet panel image of frame t-1.

[0106] When Euclidean distance Exceeding the preset threshold (Usually 30-50 pixels), then the i-th first key point is determined to be a flying point.

[0107] In some embodiments of the present invention, a priori template of the switch cabinet model may also be introduced. The approximate area of ​​the switch is determined by template matching, and then key point detection is performed within the area.

[0108] In summary, the switchgear inspection method of this invention involves acquiring a first switchgear panel image, detecting the first switchgear panel image to obtain a detection box for each first switch in the first switchgear panel image, inputting the first switchgear panel image into a key point detection model pre-trained according to the target number order to obtain a first key point and a first number for each first switch, matching the first key point with the detection box to obtain a first matching result, and obtaining a second matching result between the first number and the detection box based on the first matching result, and obtaining the inspection result of the first switchgear based on the second matching result. Thus, by obtaining the first number through a key point detection model trained according to the target number order and matching the first number with the detection box, the problem of inconsistent detection box number order with the required order is avoided, achieving high-precision switchgear inspection. Furthermore, hardware costs are significantly reduced: through algorithm-level key point adaptive compensation, the camera positioning accuracy requirements are relaxed, hardware costs are reduced by approximately 60%, deployment complexity is significantly reduced, and maintenance costs are low. Improved stability of keypoint detection: An adaptive keypoint quantity processing mechanism (interpolation with zero padding / truncated splitting) ensures a fixed number of output keypoints, eliminating fluctuations caused by factors such as lighting, occlusion, and angle, effectively solving the "flying point" problem. Universal adaptation for multiple switchgear models: Standardized keypoint sequence output eliminates reliance on fixed templates, adapting to layout differences in different switchgear models without requiring template re-creation after model change. Increased inspection efficiency: Utilizing a PTZ camera solution or a low-precision sliding rail robot, inspection speed is faster, enabling rapid switching between multiple preset positions. Modular algorithm design: Decoupling keypoint detection from state detection, the keypoint model focuses on position learning, while the state detection model focuses on state classification, facilitating independent optimization and iterative upgrades.

[0109] Furthermore, the present invention proposes an electronic device.

[0110] Figure 10 This is a structural block diagram of an electronic device according to an embodiment of the present invention.

[0111] like Figure 10 As shown, the electronic device 500 includes a processor 501 and a memory 503. The processor 501 and the memory 503 are connected, for example, via a bus 502. Optionally, the electronic device 500 may also include a transceiver 504. It should be noted that in practical applications, the transceiver 504 is not limited to one type, and the structure of this electronic device 500 does not constitute a limitation on the embodiments of the present invention.

[0112] Processor 501 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this invention. Processor 501 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0113] Bus 502 may include a pathway for transmitting information between the aforementioned components. Bus 502 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 502 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 10 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0114] The memory 503 stores a computer program corresponding to the switchgear inspection method of the above embodiments of the present invention. This computer program is controlled and executed by the processor 501. The processor 501 executes the computer program stored in the memory 503 to implement the content shown in the aforementioned method embodiments.

[0115] in, Figure 10 The electronic device 500 shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0116] According to the electronic device of the present invention, by implementing the switch cabinet inspection method of the above embodiments, the problem of inconsistent numbering order of the detection boxes with the required order is avoided, thereby achieving high-precision switch cabinet inspection.

[0117] Furthermore, the present invention proposes a switch cabinet inspection device.

[0118] Figure 11 This is a structural block diagram of the switchgear inspection device according to an embodiment of the present invention.

[0119] like Figure 11As shown, the switch cabinet inspection device 100 includes: a first acquisition module 101, used to acquire a first switch cabinet panel image of a first switch cabinet; a detection module 102, used to detect the first switch cabinet panel image to obtain a detection box for each first switch in the first switch cabinet panel image; an input module 103, used to input the first switch cabinet panel image into a key point detection model pre-trained according to the target number order to obtain a first key point and a first number for each first switch; a matching module 104, used to match the first key point with the detection box to obtain a first matching result, and obtain a second matching result between the first number and the detection box based on the first matching result; and a second acquisition module 105, used to obtain the inspection result of the first switch cabinet based on the second matching result.

[0120] It should be noted that other specific embodiments of the switchgear inspection device of the present invention can refer to the switchgear inspection method of the above embodiments.

[0121] The switchgear inspection device according to an embodiment of the present invention can avoid the problem of the detection frame numbering order being inconsistent with the required order, and achieve high-precision switchgear inspection.

[0122] Furthermore, this invention proposes a switchgear inspection system.

[0123] Figure 12 This is a structural block diagram of the switchgear inspection system according to an embodiment of the present invention.

[0124] like Figure 12 As shown, the switchgear inspection system 10 includes the aforementioned switchgear inspection device 100.

[0125] The switchgear inspection system according to an embodiment of the present invention can avoid the problem of the detection box numbering order being inconsistent with the required order, and achieve high-precision switchgear inspection.

[0126] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein can be considered as a ordered list of executable instructions for implementing logical functions, which can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0127] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0128] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0129] In the description of this specification, the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and should not be construed as limiting the present invention.

[0130] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0131] In this specification, unless otherwise stated, the terms "installation," "connection," "joining," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise explicitly defined. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0132] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "over," and "on top" of the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0133] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for inspecting switchgear, characterized in that, The method includes: Obtain the first switch cabinet panel image of the first switch cabinet, and perform detection on the first switch cabinet panel image to obtain the detection box of each first switch in the first switch cabinet panel image; The first switch cabinet panel image is input into a key point detection model pre-trained according to the target number order to obtain the first key point and first number of each first switch. The first key point is matched with the detection box to obtain a first matching result, and a second matching result between the first number and the detection box is obtained based on the first matching result; The inspection results of the first switchgear are obtained based on the second matching result; The training method for the keypoint detection model includes: Obtain an image of the second switch cabinet panel of the second switch cabinet, and obtain a switch detection box label for each second switch in the second switch cabinet panel image. The switch detection box label includes the bounding box coordinates and a second number of the second switch. The second number is a number obtained according to the position of the second switch in the second switch cabinet panel image and the target number order. For each of the second switches, a second key point corresponding to the second switch is obtained based on the bounding box coordinates of the second switch. The key point detection model is obtained by training the model using the second key point and the second number; The step of matching the first key point with the detection box includes: For each of the first key points, a first distance is obtained between the first key point and the center point of each detection box; For each of the first key points, a second distance is obtained between the first key point and the boundary of each detection box; The matching result between the key point and the detection box is obtained based on the first distance and the second distance.

2. The switchgear inspection method according to claim 1, characterized in that, The step of obtaining the second key point corresponding to the second switch based on the bounding box coordinates of the second switch includes: The center point of the bounding box corresponding to the second switch is obtained based on the bounding box coordinates, and the center point is used as the second key point corresponding to the second switch.

3. The switchgear inspection method according to claim 2, characterized in that, The step of training the model using the second key point and the second number includes: Obtain the number of the second key points; When the number of the second key points is equal to the preset number, the model is trained using the second key points and the second number. When the number of the second key points is less than the preset number, multiple third key points are added, and the model is trained using the second key points, the third key points, and the second number, wherein the number of the third key points is equal to the difference between the preset number and the number of the second key points; When the number of the second key points is greater than the preset number, multiple key point groups are obtained based on the second key points, and model training is performed using the multiple key point groups and the second number; wherein the number of key points in each key point group is equal to the preset number.

4. The switchgear inspection method according to claim 1, characterized in that, The step of obtaining the matching result between the key point and the detection box based on the first distance and the second distance includes: The direct distance cost matrix is ​​obtained based on the first distance, and the inclusion relation cost matrix is ​​obtained based on the second distance; The combined cost matrix is ​​obtained based on the direct distance cost matrix and the inclusion relation cost matrix; The matching result is obtained based on the combined cost matrix.

5. The switchgear inspection method according to claim 4, characterized in that, The combined cost matrix is ​​obtained according to the following formula: ， in, The combined cost matrix is... As the first preset weight, For normalization, Let be the direct distance cost matrix. As the second preset weight, The inclusion relation cost matrix is... .

6. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executed on the processor, wherein when the computer program is executed by the processor, it implements the switch cabinet inspection method according to any one of claims 1-5.

7. A switchgear inspection device, characterized in that, include: The first acquisition module is used to acquire the first switch cabinet panel image of the first switch cabinet; The detection module is used to detect the first switch cabinet panel image and obtain the detection box of each first switch in the first switch cabinet panel image; The input module is used to input the image of the first switch cabinet panel into a key point detection model that has been trained in advance according to the target number order, so as to obtain the first key point and the first number of each of the first switches; The matching module is used to match the first key point with the detection box to obtain a first matching result, and to obtain a second matching result between the first number and the detection box based on the first matching result; The second acquisition module is used to obtain the inspection result of the first switch cabinet based on the second matching result; The training method for the keypoint detection model includes: Obtain an image of the second switch cabinet panel of the second switch cabinet, and obtain a switch detection box label for each second switch in the second switch cabinet panel image. The switch detection box label includes the bounding box coordinates and a second number of the second switch. The second number is a number obtained according to the position of the second switch in the second switch cabinet panel image and the target number order. For each of the second switches, a second key point corresponding to the second switch is obtained based on the bounding box coordinates of the second switch. The key point detection model is obtained by training the model using the second key point and the second number; The step of matching the first key point with the detection box includes: For each of the first key points, a first distance is obtained between the first key point and the center point of each detection box; For each of the first key points, a second distance is obtained between the first key point and the boundary of each detection box; The matching result between the key point and the detection box is obtained based on the first distance and the second distance.

8. A switchgear inspection system, characterized in that, Includes the switchgear inspection device according to claim 7.

Images