A knob switch state detection method, device, equipment and storage medium

Abstract

The present application relates to image processing technology, disclose a kind of knob switch state detection method, comprising: obtaining inspection switch image, and it is judged whether the inspection switch image quality meets the pre-set picture quality standard;When not satisfied, filter out the inspection switch image;When satisfied, the target detection model of pre-construction is used to detect the switch coordinate frame in the inspection switch image;According to switch coordinate frame, the position information of knob switch is positioned, and according to position information, the inspection switch image is cut to obtain cut switch image;The segmentation network of pre-training is used to carry out target segmentation to the cut switch image, obtains switch state recognition result based on target segmentation result, and switch recognition result is mapped to the inspection switch image, obtains the switch state of knob switch.The present application also proposes a kind of knob switch state detection device, equipment and storage medium.The present application can improve the efficiency and accuracy of knob switch state detection.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method, apparatus, device, and storage medium for detecting the state of a rotary switch. Background Technology

[0002] Currently, in high-risk industries such as chemical and power, the following problems often arise when inspecting rotary switches in scenarios such as heat tracing cabinets, electrical boxes, and high-voltage distribution cabinets: 1) During on-site inspections, poorly chosen shooting angles can easily cause the rotary switches in the images to appear tilted, leading to false detections. 2) When multiple switches are present in an image, it is usually necessary to perform status checks on each switch individually. This approach is not conducive to analyzing and recording the status of all switch knobs in the current scene. 3) Unclear images can also affect the judgment of results.

[0003] Traditional inspection methods rely on manual inspection, which poses certain safety hazards in some high-risk environments. Robots can effectively avoid this problem by replacing humans and save a lot of labor costs. However, due to the complexity of the on-site environment, robots may interfere with the judgment of the state of knob switches, thus failing to accurately identify the state of the knob switches and resulting in low efficiency in knob switch state recognition. Summary of the Invention

[0004] This invention provides a method, apparatus, device, and storage medium for detecting the status of a rotary switch, which can improve the efficiency and accuracy of rotary switch status detection.

[0005] To achieve the above objectives, the present invention provides a method for detecting the state of a rotary switch, comprising:

[0006] Acquire an image of the inspection switch and determine whether the quality of the inspection switch image meets the preset image quality standard;

[0007] When the image quality of the inspection switch does not meet the preset image quality standard, the inspection switch image is filtered out.

[0008] When the quality of the inspection switch image meets the preset image quality standard, the switch coordinate frame in the inspection switch image is detected using a pre-constructed target detection model;

[0009] The position information of the rotary switch is located according to the switch coordinate frame, and the inspection switch image is cropped according to the position information to obtain the cropped switch image;

[0010] The cutter switch image is segmented using a pre-trained segmentation network. The switch state recognition result is obtained based on the target segmentation result. The switch recognition result is then mapped to the inspection switch image to obtain the switch state of the rotary switch.

[0011] Optionally, determining whether the image quality of the inspection switch meets a preset image quality standard includes:

[0012] Calculate the information entropy of the inspection switch image in the RGB color components to obtain the color information entropy;

[0013] The inspection switch image is divided into multiple image blocks according to a preset segmentation size;

[0014] Calculate the information entropy of the spatial component of each of the multiple image blocks to obtain the spatial information entropy;

[0015] The color information entropy and the spatial information entropy are integrated to obtain the total information entropy. The total information entropy is used to calculate the evaluation score. The quality of the inspection switch image is judged to meet the image quality standard by determining whether the evaluation score meets the preset evaluation score threshold.

[0016] Optionally, calculating the information entropy of the spatial components of each of the plurality of image blocks includes:

[0017] The information entropy of the spatial component of each of the plurality of image blocks is calculated using the following formula:

[0018]

[0019] Among them, H space Let H(I) be the spatial information entropy, N be the number of image patches, and H(I) be the spatial information entropy. j ) represents the color information entropy of the j-th image patch.

[0020] Optionally, the step of detecting the switch coordinate frame in the inspection switch image using a pre-built target detection model includes:

[0021] The pre-built target detection model is used to detect the switch images in the inspection switch images, and the fixed-point coordinates of the switch images are recorded, wherein the fixed-point coordinates include the coordinates of the upper left corner and the lower right corner of the switch images;

[0022] The coordinate frame of the switch image is constructed based on the fixed point coordinates, thus obtaining the switch coordinate frame.

[0023] Optionally, the step of locating the position information of the rotary switch based on the switch coordinate frame includes:

[0024] Calculate the center point coordinates based on the fixed point coordinates in the switch coordinate frame;

[0025] The position of the center point is located in the inspection switch image to obtain the position information of the rotary switch.

[0026] Optionally, before performing switch state recognition on the cutter switch image using a pre-trained segmentation network, the cutter switch image is further subjected to tilt correction processing.

[0027] Optionally, the tilt correction processing of the shear switch image includes:

[0028] Feature point information of the shear switch image is extracted using a feature detection algorithm;

[0029] The feature point information is clustered along the four corners of the shear switch image using a preset clustering algorithm to obtain four feature point information clusters;

[0030] The cluster center point of each of the four feature point information clusters is selected as the feature coordinate point to obtain four feature coordinate points, and the image tilt angle is calculated based on the four feature coordinate points.

[0031] The shear switch image is corrected using a transmission transformation algorithm based on the image tilt angle.

[0032] To address the above problems, the present invention also provides a rotary switch status detection device, the device comprising:

[0033] The image acquisition module is used to acquire images of the inspection switch and determine whether the quality of the inspection switch image meets the preset image quality standard.

[0034] The image cropping module is used to filter out the inspection switch image when the quality of the inspection switch image does not meet the preset image quality standard; when the quality of the inspection switch image meets the preset image quality standard, it uses a pre-built target detection model to detect the switch coordinate frame in the inspection switch image; it locates the position information of the rotary switch according to the switch coordinate frame, and crops the inspection switch image according to the position information to obtain the cropped switch image.

[0035] The state recognition module is used to perform target segmentation on the cutter switch image using a pre-trained segmentation network, obtain the switch state recognition result based on the target segmentation result, and map the switch recognition result to the inspection switch image to obtain the switch state of the rotary switch.

[0036] To address the above problems, the present invention also provides an electronic device, the electronic device comprising:

[0037] At least one processor; and,

[0038] A memory communicatively connected to the at least one processor; wherein,

[0039] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the rotary switch state detection method described above.

[0040] To address the aforementioned problems, the present invention also provides a computer-readable storage medium storing at least one computer program, which is executed by a processor in an electronic device to implement the rotary switch state detection method described above.

[0041] This invention ensures the clarity of images captured by the inspection robot by judging whether the quality of the inspection switch image meets a preset image quality standard, thereby guaranteeing the accuracy of knob switch status detection. Furthermore, by introducing color entropy and spatial entropy into the image quality judgment, the spatial distribution of pixels in the inspection switch image and their distribution in the color image are guaranteed, ensuring the comprehensiveness of the captured inspection switch image. In addition, by locating the position information of the knob switch based on the switch coordinate frame and cropping the inspection switch image according to the position information, a cropped switch image is obtained, ensuring a one-to-one correspondence between the cropped knob switch image and the inspection switch image, and guaranteeing the accuracy of the knob switch status detection result corresponding to the inspection switch image. Moreover, by mapping the switch recognition result to the inspection switch image, the comprehensiveness of all knob switch status recognitions in the inspection switch image is guaranteed, improving the efficiency and accuracy of knob switch status recognition in the inspection switch image. Attached Figure Description

[0042] Figure 1 This is a flowchart illustrating a rotary switch state detection method according to an embodiment of the present invention.

[0043] Figure 2 This is an inspection switch image provided by a rotary switch state detection method according to an embodiment of the present invention;

[0044] Figure 3 This is a target segmentation result image of a rotary switch state detection method provided in an embodiment of the present invention;

[0045] Figure 4 The original rotary switch image is provided by a rotary switch state detection method according to an embodiment of the present invention.

[0046] Figure 5 The image of a rotary switch after tilt correction processing is shown in the original rotary switch image of the rotary switch state detection method provided in an embodiment of the present invention.

[0047] Figure 6The rotary switch state recognition result is provided by a rotary switch state detection method according to an embodiment of the present invention.

[0048] Figure 7 This is a functional block diagram of a rotary switch status detection device according to an embodiment of the present invention;

[0049] Figure 8 This is a schematic diagram of the structure of an electronic device that implements the rotary switch state detection method according to an embodiment of the present invention.

[0050] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0051] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0052] This application provides a method for detecting the state of a rotary switch. The executing entity of this method includes, but is not limited to, at least one electronic device that can be configured to execute the method provided in this application, such as a server or a terminal. In other words, the method can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster. The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0053] Reference Figure 1 The diagram shown is a flowchart illustrating a rotary switch state detection method according to an embodiment of the present invention. In this embodiment, the rotary switch state detection method includes:

[0054] S1. Obtain the inspection switch image and determine whether the quality of the inspection switch image meets the preset image quality standard.

[0055] In this embodiment of the invention, the inspection switch image refers to a switch image taken by the inspection robot when performing switch status inspection work.

[0056] As an embodiment of the present invention, determining whether the image quality of the inspection switch meets the preset image quality standard includes:

[0057] Calculate the information entropy of the inspection switch image in the RGB color components to obtain the color information entropy;

[0058] The inspection switch image is divided into multiple image blocks according to a preset segmentation size;

[0059] Calculate the information entropy of the spatial component of each of the multiple image blocks to obtain the spatial information entropy;

[0060] The color information entropy and the spatial information entropy are integrated to obtain the total information entropy. The total information entropy is used to calculate the evaluation score. The quality of the inspection switch image is judged to meet the image quality standard by determining whether the evaluation score meets the preset evaluation score threshold.

[0061] In this embodiment of the invention, the preset segmentation size refers to a pre-set aspect ratio in pixels.

[0062] In this embodiment of the invention, the color information entropy includes the information entropy of the RGB three channels.

[0063] This invention calculates the information entropy of the inspection switch image in the RGB color components using the following formula:

[0064]

[0065] Where, p R (r i ) represents the gray level r in the red channel. i The probability of occurrence, p G (g i ) represents the gray level g in the green channel. i The probability of occurrence, p B (b i ) represents the gray level b in the blue channel. i The probability of occurrence, r i For the grayscale value in the red channel, g i b represents the grayscale value in the green channel. i This represents the grayscale value in the blue channel.

[0066] Further, calculating the information entropy of the spatial components of each of the plurality of image blocks includes:

[0067] The information entropy of the spatial component of each of the plurality of image blocks is calculated using the following formula:

[0068]

[0069] Among them, H space Let H(I) be the spatial information entropy, N be the number of image patches, and H(I) be the spatial information entropy. j) represents the color information entropy of the j-th image patch.

[0070] Furthermore, the total information entropy is obtained by integrating the color information entropy and the spatial information entropy using the following calculation formula:

[0071]

[0072] In the formula, α is the first weighting coefficient, = is the second weighting coefficient, and γ is the third weighting coefficient. It is the fourth weighting coefficient.

[0073] In this embodiment of the invention, the weighting coefficient is adjusted based on the image feature information within the scene, and must meet the following requirements: For example, α, β, γ, Set them to 0.3, 0.2, 0.2, and 0.3 respectively.

[0074] In this embodiment of the invention, the preset evaluation score threshold refers to a reference value for the amount of information in the image obtained by experimental statistics on multiple historical data images.

[0075] This invention, by determining whether the quality of the inspection switch image meets a preset image quality standard, can ensure the clarity of the images captured by the inspection robot, thereby ensuring the accuracy of the knob switch status detection. In addition, by introducing color information entropy and spatial information entropy into the image quality judgment, the spatial distribution of pixels in the inspection switch image and their distribution in the color image can be guaranteed, ensuring the comprehensiveness of the captured inspection switch image.

[0076] S2. When the image quality of the inspection switch does not meet the preset image quality standard, the inspection switch image is filtered out.

[0077] S3. When the quality of the inspection switch image meets the preset image quality standard, the switch coordinate frame in the inspection switch image is detected using a pre-constructed target detection model.

[0078] In this embodiment of the invention, the pre-built target detection model refers to a target detection model that can achieve the detection of a specific target after training, such as the YOLOv8 target detection model.

[0079] As an embodiment of the present invention, the step of detecting the switch coordinate frame in the inspection switch image using a pre-constructed target detection model includes:

[0080] The pre-built target detection model is used to detect the switch images in the inspection switch images, and the fixed-point coordinates of the switch images are recorded, wherein the fixed-point coordinates include the coordinates of the upper left corner and the lower right corner of the switch images;

[0081] The coordinate frame of the switch image is constructed based on the fixed point coordinates, thus obtaining the switch coordinate frame.

[0082] In this embodiment of the invention, the coordinate frame of the switch image is a coordinate array composed of fixed point coordinates. For example, (x1, y1) is the coordinate of the upper left corner of the coordinate frame, and (x2, y2) is the coordinate of the lower right corner of the coordinate frame. Then the switch coordinate frame is (x1, y1, x2, y2).

[0083] S4. Locate the position information of the rotary switch according to the switch coordinate frame, and cut the inspection switch image according to the position information to obtain the cut switch image.

[0084] In this embodiment of the invention, the location information refers to the position of multiple rotary switches contained in an inspection switch image within the inspection switch image.

[0085] As an embodiment of the present invention, the step of locating the position information of the rotary switch according to the switch coordinate frame includes:

[0086] Calculate the center point coordinates based on the fixed point coordinates in the switch coordinate frame;

[0087] The position of the center point is located in the inspection switch image to obtain the position information of the rotary switch.

[0088] As an embodiment of the present invention, the step of cutting the inspection switch image according to the location information to obtain the cut switch image includes:

[0089] Based on the location information, the position of a single rotary switch is located, and the position and switch coordinate frame are combined to cut out the single rotary switch image from the inspection switch image.

[0090] In this embodiment of the invention, the position information of the rotary switch is located according to the switch coordinate frame, and the inspection switch image is cropped according to the position information to obtain a cropped switch image. This ensures that the cropped rotary switch image corresponds one-to-one with the inspection switch image and guarantees the accuracy of the rotary switch status detection result corresponding to the inspection switch image.

[0091] S5. The cutter switch image is segmented using a pre-trained segmentation network. The switch state recognition result is obtained based on the target segmentation result. The switch recognition result is then mapped to the inspection switch image to obtain the switch state of the rotary switch.

[0092] In this embodiment of the invention, the cut-out image of the cutter switch is tilted to a certain extent due to the influence of the shooting angle.

[0093] In this embodiment of the invention, the pre-trained segmentation network refers to a network used to identify the on / off state of an image. For example, the pre-trained segmentation network may be a Mask R-CNN network, where Mask R-CNN (Mask Region-based Convolutional Neural Network) is a deep learning model for image segmentation.

[0094] As an embodiment of the present invention, before performing switch state recognition on the cutter switch image using the pre-trained segmentation network, the cutter switch image is further subjected to tilt correction processing.

[0095] Furthermore, the tilt correction processing of the shear switch image includes:

[0096] Feature point information of the shear switch image is extracted using a feature detection algorithm;

[0097] The feature point information is clustered along the four corners of the shear switch image using a preset clustering algorithm to obtain four feature point information clusters;

[0098] The cluster center point of each of the four feature point information clusters is selected as the feature coordinate point to obtain four feature coordinate points, and the image tilt angle is calculated based on the four feature coordinate points.

[0099] The shear switch image is corrected using a transmission transformation algorithm based on the image tilt angle.

[0100] In this embodiment of the invention, the feature detection algorithm may be the SIFT feature detection algorithm, which is an algorithm used to describe local features of an image.

[0101] In this embodiment of the invention, the preset clustering algorithm refers to an algorithm that aggregates discrete feature points into multiple feature clusters. For example, the preset clustering algorithm may be the K-means clustering algorithm.

[0102] In this embodiment of the invention, the rotary switch states include manual state, stop state, and automatic state.

[0103] For example, in this embodiment of the invention, the shearing switch image is corrected using a transmission transformation algorithm based on the image tilt angle, including the following steps:

[0104] Step 1: After obtaining the four feature coordinate points, starting from the top left and proceeding clockwise, assume the four feature coordinate points are P1(x1, y1), P2(x2, y2), P3(x3, y3), and P4(x4, y4). Calculate the angle u1 between P1 and P2 in the horizontal direction and the angle u2 between P1 and P4 in the vertical direction. If both angles u1 and u2 are less than 10°, the image will not be corrected; otherwise, the image needs to be corrected.

[0105] Step 2: Image correction is achieved using the transmission transformation method. After processing in Step 1, the four coordinate points of the area to be corrected in the image are P1(x1, y1), P2(x2, y2), P3(x3, y3), and P4(x4, y4). The four coordinate points after correction are P1'(x1, y1), P2'(x3, y1), P3'(x3, y3), and P4'(x1, y3). That is, the coordinates of the upper left P1 and the lower right P3 remain unchanged, while P2 and P4 are transformed. The transmission transformation matrix is ​​calculated from the given eight coordinate points and the transformation is applied to achieve image correction.

[0106] By mapping the switch identification results to the inspection switch image, this embodiment of the invention can ensure the comprehensiveness of the identification of the status of all rotary switches in the inspection switch image and improve the efficiency of rotary switch status identification in the inspection switch image.

[0107] This invention, by determining whether the quality of the inspection switch image meets a preset image quality standard, ensures the clarity of the images captured by the inspection robot, thereby guaranteeing the accuracy of knob switch status detection. Furthermore, introducing color entropy and spatial entropy into the image quality judgment ensures the spatial distribution of pixels in the inspection switch image and their distribution in the color image, guaranteeing the comprehensiveness of the captured inspection switch image. In addition, locating the knob switch's position information based on the switch coordinate frame and cropping the inspection switch image based on this position information ensures a one-to-one correspondence between the cropped knob switch images and the inspection switch images, guaranteeing the accuracy of the knob switch status detection results corresponding to the inspection switch images. Moreover, mapping the switch recognition results to the inspection switch image ensures the comprehensiveness of all knob switch status recognitions in the inspection switch image, improving the efficiency of knob switch status recognition in the inspection switch image.

[0108] like Figure 2 The image shown is an inspection switch image of a rotary switch state detection method provided in an embodiment of the present invention.

[0109] like Figure 3 The image shown is a target segmentation result image of a rotary switch state detection method provided in an embodiment of the present invention.

[0110] like Figure 4 The image shown is an original rotary switch image of a rotary switch state detection method provided in an embodiment of the present invention.

[0111] like Figure 5 The image shown is a rotary switch image after tilt correction processing of the original rotary switch image of the rotary switch state detection method provided in an embodiment of the present invention.

[0112] like Figure 6 The figure shows the state recognition result of a rotary switch using a rotary switch state detection method according to an embodiment of the present invention. The recognition result for the rotary switch in the figure is "auto," indicating an automatic state.

[0113] like Figure 7 The diagram shown is a functional block diagram of a rotary switch status detection device provided in an embodiment of the present invention.

[0114] The rotary switch state detection device 100 of the present invention can be installed in an electronic device. Depending on the function implemented, the rotary switch state detection device 100 may include an image acquisition module 101, an image cropping module 102, and a state recognition module 103.

[0115] The module described in this invention can also be called a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and are stored in the memory of the electronic device.

[0116] In this embodiment, the functions of each module / unit are as follows:

[0117] The image acquisition module 101 is used to acquire an image of the inspection switch and determine whether the quality of the inspection switch image meets a preset image quality standard.

[0118] In this embodiment of the invention, the inspection switch image refers to a switch image taken by the inspection robot when performing switch status inspection work.

[0119] As an embodiment of the present invention, determining whether the image quality of the inspection switch meets the preset image quality standard includes:

[0120] Calculate the information entropy of the inspection switch image in the RGB color components to obtain the color information entropy;

[0121] The inspection switch image is divided into multiple image blocks according to a preset segmentation size;

[0122] Calculate the information entropy of the spatial component of each of the multiple image blocks to obtain the spatial information entropy;

[0123] The color information entropy and the spatial information entropy are integrated to obtain the total information entropy. The total information entropy is used to calculate the evaluation score. The quality of the inspection switch image is judged to meet the image quality standard by determining whether the evaluation score meets the preset evaluation score threshold.

[0124] In this embodiment of the invention, the preset segmentation size refers to a pre-set aspect ratio in pixels.

[0125] In this embodiment of the invention, the color information entropy includes the information entropy of the RGB three channels.

[0126] This invention calculates the information entropy of the inspection switch image in the RGB color components using the following formula:

[0127]

[0128] Where, p R (r i ) represents the gray level r in the red channel. i The probability of occurrence, p G (g i ) represents the gray level g in the green channel. i The probability of occurrence, p B (b i ) represents the gray level b in the blue channel. i The probability of occurrence, r i For the grayscale value in the red channel, g i b represents the grayscale value in the green channel. i This represents the grayscale value in the blue channel.

[0129] Further, calculating the information entropy of the spatial components of each of the plurality of image blocks includes:

[0130] The information entropy of the spatial component of each of the plurality of image blocks is calculated using the following formula:

[0131]

[0132] Among them, H space Let H(I) be the spatial information entropy, N be the number of image patches, and H(I) be the spatial information entropy. j ) represents the color information entropy of the j-th image patch.

[0133] Furthermore, the total information entropy is obtained by integrating the color information entropy and the spatial information entropy using the following calculation formula:

[0134]

[0135] In the formula, a is the first weighting coefficient, β is the second weighting coefficient, and γ is the third weighting coefficient. It is the fourth weighting coefficient.

[0136] In this embodiment of the invention, the weighting coefficient is adjusted based on the image feature information within the scene, and must meet the following requirements: For example, α, β, γ, Set them to 0.3, 0.2, 0.2, and 0.3 respectively.

[0137] In this embodiment of the invention, the preset evaluation score threshold refers to a reference value for the amount of information in the image obtained by experimental statistics on multiple historical data images.

[0138] This invention, by determining whether the quality of the inspection switch image meets a preset image quality standard, can ensure the clarity of the images captured by the inspection robot, thereby ensuring the accuracy of the knob switch status detection. In addition, by introducing color information entropy and spatial information entropy into the image quality judgment, the spatial distribution of pixels in the inspection switch image and their distribution in the color image can be guaranteed, ensuring the comprehensiveness of the captured inspection switch image.

[0139] The image cropping module 102 is used to filter out the inspection switch image when the quality of the inspection switch image does not meet the preset image quality standard; when the quality of the inspection switch image meets the preset image quality standard, it uses a pre-built target detection model to detect the switch coordinate frame in the inspection switch image; it locates the position information of the knob switch according to the switch coordinate frame, and crops the inspection switch image according to the position information to obtain a cropped switch image.

[0140] When the image quality of the inspection switch does not meet the preset image quality standard, the inspection switch image is filtered out.

[0141] In this embodiment of the invention, the pre-built target detection model refers to a target detection model that can achieve the detection of a specific target after training, such as the YOLOv8 target detection model.

[0142] As an embodiment of the present invention, the step of detecting the switch coordinate frame in the inspection switch image using a pre-constructed target detection model includes:

[0143] The pre-built target detection model is used to detect the switch images in the inspection switch images, and the fixed-point coordinates of the switch images are recorded, wherein the fixed-point coordinates include the coordinates of the upper left corner and the lower right corner of the switch images;

[0144] The coordinate frame of the switch image is constructed based on the fixed point coordinates, thus obtaining the switch coordinate frame.

[0145] In this embodiment of the invention, the coordinate frame of the switch image is a coordinate array composed of fixed point coordinates. For example, (x1, y1) is the coordinate of the upper left corner of the coordinate frame, and (x2, y2) is the coordinate of the lower right corner of the coordinate frame. Then the switch coordinate frame is (x1, y1, x2, y2).

[0146] In this embodiment of the invention, the location information refers to the position of multiple rotary switches contained in an inspection switch image within the inspection switch image.

[0147] As an embodiment of the present invention, the step of locating the position information of the rotary switch according to the switch coordinate frame includes:

[0148] Calculate the center point coordinates based on the fixed point coordinates in the switch coordinate frame;

[0149] The position of the center point is located in the inspection switch image to obtain the position information of the rotary switch.

[0150] As an embodiment of the present invention, the step of cutting the inspection switch image according to the location information to obtain the cut switch image includes:

[0151] Based on the location information, the position of a single rotary switch is located, and the position and switch coordinate frame are combined to cut out the single rotary switch image from the inspection switch image.

[0152] In this embodiment of the invention, the position information of the rotary switch is located according to the switch coordinate frame, and the inspection switch image is cropped according to the position information to obtain a cropped switch image. This ensures that the cropped rotary switch image corresponds one-to-one with the inspection switch image and guarantees the accuracy of the rotary switch status detection result corresponding to the inspection switch image.

[0153] The state recognition module 103 is used to perform target segmentation on the cutter switch image using a pre-trained segmentation network, obtain switch state recognition results based on the target segmentation results, and map the switch recognition results to the inspection switch image to obtain the switch state of the rotary switch.

[0154] In this embodiment of the invention, the cut-out image of the cutter switch is tilted to a certain extent due to the influence of the shooting angle.

[0155] In this embodiment of the invention, the pre-trained segmentation network refers to a network used to identify the on / off state of an image. For example, the pre-trained segmentation network may be a Mask R-CNN network, where Mask R-CNN (Mask Region-based Convolutional Neural Network) is a deep learning model for image segmentation.

[0156] As an embodiment of the present invention, before performing switch state recognition on the cutter switch image using the pre-trained segmentation network, the cutter switch image is further subjected to tilt correction processing.

[0157] Furthermore, the tilt correction processing of the shear switch image includes:

[0158] Feature point information of the shear switch image is extracted using a feature detection algorithm;

[0159] The feature point information is clustered along the four corners of the shear switch image using a preset clustering algorithm to obtain four feature point information clusters;

[0160] The cluster center point of each of the four feature point information clusters is selected as the feature coordinate point to obtain four feature coordinate points, and the image tilt angle is calculated based on the four feature coordinate points.

[0161] The shear switch image is corrected using a transmission transformation algorithm based on the image tilt angle.

[0162] In this embodiment of the invention, the feature detection algorithm may be the SIFT feature detection algorithm, which is an algorithm used to describe local features of an image.

[0163] In this embodiment of the invention, the preset clustering algorithm refers to an algorithm that aggregates discrete feature points into multiple feature clusters. For example, the preset clustering algorithm may be the K-means clustering algorithm.

[0164] In this embodiment of the invention, the rotary switch states include manual state, stop state, and automatic state.

[0165] For example, in this embodiment of the invention, the shearing switch image is corrected using a transmission transformation algorithm based on the image tilt angle, including the following steps:

[0166] Step 1: After obtaining the four feature coordinate points, starting from the top left and proceeding clockwise, assume the four feature coordinate points are P1(x1, y1), P2(x2, y2), P3(x3, y3), and P4(x4, y4). Calculate the angle u1 between P1 and P2 in the horizontal direction and the angle u2 between P1 and P4 in the vertical direction. If both angles u1 and u2 are less than 10°, the image will not be corrected; otherwise, the image needs to be corrected.

[0167] Step 2: Image correction is achieved using the transmission transformation method. After processing in Step 1, the four coordinate points of the area to be corrected in the image are P1(x1, y1), P2(x2, y2), P3(x3, y3), and P4(x4, y4). The four coordinate points after correction are P1'(x1, y1), P2'(x3, y1), P3'(x3, y3), and P4'(x1, y3). That is, the coordinates of the upper left P1 and the lower right P3 remain unchanged, while P2 and P4 are transformed. The transmission transformation matrix is ​​calculated from the given eight coordinate points and the transformation is applied to achieve image correction.

[0168] By mapping the switch identification results to the inspection switch image, this embodiment of the invention can ensure the comprehensiveness of the identification of the status of all rotary switches in the inspection switch image and improve the efficiency of rotary switch status identification in the inspection switch image.

[0169] Reference Figure 8 The diagram shown is a structural schematic of an electronic device that implements the rotary switch state detection method according to an embodiment of the present invention.

[0170] The electronic device may include a processor 10, a memory 11, a communication bus 12, and a communication interface 13. It may also include a computer program stored in the memory 11 and capable of running on the processor 10, such as a rotary switch state detection method program.

[0171] In some embodiments, the processor 10 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 10 is the control unit of the electronic device, connecting various components of the entire electronic device through various interfaces and lines. It executes programs or modules stored in the memory 11 (e.g., executing a rotary switch state detection method program) and calls data stored in the memory 11 to perform various functions of the electronic device and process data.

[0172] The memory 11 includes at least one type of readable storage medium, including flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of an electronic device, such as a portable hard drive. In other embodiments, the memory 11 can be an external storage device of the electronic device, such as a plug-in portable hard drive, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc. Furthermore, the memory 11 can include both internal and external storage units of the electronic device. The memory 11 can be used not only to store application software and various types of data installed on the electronic device, such as the code of a rotary switch state detection method program, but also to temporarily store data that has been output or will be output.

[0173] The communication bus 12 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable communication between the memory 11 and at least one processor 10, etc.

[0174] The communication interface 13 is used for communication between the aforementioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, Bluetooth interface, etc.), typically used to establish communication connections between the electronic device and other electronic devices. The user interface may be a display, an input unit (such as a keyboard), or, optionally, a standard wired or wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the electronic device and to display a visual user interface.

[0175] Figure 8 Only electronic devices with components are shown; it will be understood by those skilled in the art that... Figure 8The structure shown does not constitute a limitation on the electronic device and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0176] For example, although not shown, the electronic device may also include a power supply (such as a battery) to power the various components. Preferably, the power supply can be logically connected to the at least one processor 10 through a power management device, thereby enabling functions such as charging management, discharging management, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0177] It should be understood that the embodiments described are for illustrative purposes only and are not limited to this structure in the scope of the patent application.

[0178] The program for detecting the state of a rotary switch stored in the memory 11 of the electronic device is a combination of multiple instructions. When run in the processor 10, it can achieve the following:

[0179] Acquire an image of the inspection switch and determine whether the quality of the inspection switch image meets the preset image quality standard;

[0180] When the image quality of the inspection switch does not meet the preset image quality standard, the inspection switch image is filtered out.

[0181] When the quality of the inspection switch image meets the preset image quality standard, the switch coordinate frame in the inspection switch image is detected using a pre-constructed target detection model;

[0182] The position information of the rotary switch is located according to the switch coordinate frame, and the inspection switch image is cropped according to the position information to obtain the cropped switch image;

[0183] The cutter switch image is segmented using a pre-trained segmentation network. The switch state recognition result is obtained based on the target segmentation result. The switch recognition result is then mapped to the inspection switch image to obtain the switch state of the rotary switch.

[0184] Specifically, the specific implementation method of the processor 10 for the above instructions can be referred to the description of the relevant steps in the corresponding embodiment of the accompanying drawings, and will not be repeated here.

[0185] Furthermore, if the modules / units integrated in the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0186] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor of an electronic device, can perform the following:

[0187] Acquire an image of the inspection switch and determine whether the quality of the inspection switch image meets the preset image quality standard;

[0188] When the image quality of the inspection switch does not meet the preset image quality standard, the inspection switch image is filtered out.

[0189] When the quality of the inspection switch image meets the preset image quality standard, the switch coordinate frame in the inspection switch image is detected using a pre-constructed target detection model;

[0190] The position information of the rotary switch is located according to the switch coordinate frame, and the inspection switch image is cropped according to the position information to obtain the cropped switch image;

[0191] The cutter switch image is segmented using a pre-trained segmentation network. The switch state recognition result is obtained based on the target segmentation result. The switch recognition result is then mapped to the inspection switch image to obtain the switch state of the rotary switch.

[0192] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0193] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0194] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0195] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0196] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0197] The blockchain referred to in this invention is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.

[0198] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0199] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a system claim may also be implemented by a single unit or device through software or hardware. The terms "first," "second," etc., are used to indicate names and do not indicate any specific order.

[0200] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for detecting the state of a rotary switch, characterized in that, The method includes: Acquire an image of the inspection switch and determine whether the image quality meets a preset image quality standard. The determination of whether the image quality meets the preset image quality standard includes: Calculate the information entropy of the inspection switch image in the three RGB color components to obtain the color information entropy; The inspection switch image is divided into multiple image blocks according to a preset segmentation size; Calculate the information entropy of the spatial component of each of the plurality of image blocks, and obtain the spatial information entropy using the following formula: ,in, For spatial information entropy, The number of image patches, For the first j Information entropy of an image patch; The color information entropy and the spatial information entropy are integrated to obtain the total information entropy. The total information entropy is used to calculate the evaluation score. The quality of the inspection switch image is judged to meet the image quality standard by determining whether the evaluation score meets the preset evaluation score threshold. When the image quality of the inspection switch does not meet the preset image quality standard, the inspection switch image is filtered out. When the quality of the inspection switch image meets the preset image quality standard, the switch coordinate frame in the inspection switch image is detected using a pre-constructed target detection model; The position information of the rotary switch is located according to the switch coordinate frame, and the inspection switch image is cropped according to the position information to obtain the cropped switch image; The cutter switch image is segmented using a pre-trained segmentation network. The switch state recognition result is obtained based on the target segmentation result. The switch state recognition result is then mapped to the inspection switch image to obtain the switch state of the rotary switch.

2. The rotary switch state detection method as described in claim 1, characterized in that, The step of detecting the switch coordinate frame in the inspection switch image using a pre-built target detection model includes: The pre-built target detection model is used to detect the switch images in the inspection switch images, and the fixed-point coordinates of the switch images are recorded, wherein the fixed-point coordinates include the coordinates of the upper left corner and the lower right corner of the switch images; The coordinate frame of the switch image is constructed based on the fixed point coordinates, thus obtaining the switch coordinate frame.

3. The rotary switch state detection method as described in claim 1, characterized in that, The step of locating the position information of the rotary switch based on the switch coordinate frame includes: Calculate the center point coordinates based on the fixed point coordinates in the switch coordinate frame; The position of the center point is located in the inspection switch image to obtain the position information of the rotary switch.

4. The rotary switch state detection method as described in claim 1, characterized in that, Before using the pre-trained segmentation network to identify the switch state of the shear switch image, the method further includes tilt correction processing of the shear switch image.

5. The rotary switch state detection method as described in claim 4, characterized in that, The tilt correction processing of the shear switch image includes: Feature point information of the shear switch image is extracted using a feature detection algorithm; The feature point information is clustered along the four corners of the shear switch image using a preset clustering algorithm to obtain four feature point information clusters; The cluster center point of each of the four feature point information clusters is selected as the feature coordinate point to obtain four feature coordinate points, and the image tilt angle is calculated based on the four feature coordinate points. The shear switch image is corrected using a transmission transformation algorithm based on the image tilt angle.

6. A rotary switch status detection device, characterized in that, The device can implement the rotary switch state detection method as described in any one of claims 1 to 5, and the device includes: An image acquisition module is used to acquire an image of an inspection switch and determine whether the quality of the inspection switch image meets a preset image quality standard. Determining whether the quality of the inspection switch image meets the preset image quality standard includes: calculating the information entropy of the inspection switch image on the RGB three color components to obtain color information entropy; segmenting the inspection switch image into multiple image blocks according to a preset segmentation size; calculating the information entropy of the spatial components of each image block to obtain spatial information entropy, using the following formula: ,in, For spatial information entropy, The number of image patches, For the first j The information entropy of each image block; the color information entropy and the spatial information entropy are integrated to obtain the total information entropy, and the evaluation score is calculated using the total information entropy. The quality of the inspection switch image is judged to meet the image quality standard by determining whether the evaluation score meets the preset evaluation score threshold. The image cropping module is used to filter out the inspection switch image when the quality of the inspection switch image does not meet the preset image quality standard; when the quality of the inspection switch image meets the preset image quality standard, it uses a pre-built target detection model to detect the switch coordinate frame in the inspection switch image; it locates the position information of the rotary switch according to the switch coordinate frame, and crops the inspection switch image according to the position information to obtain the cropped switch image. The state recognition module is used to perform target segmentation on the cutter switch image using a pre-trained segmentation network, obtain the switch state recognition result based on the target segmentation result, and map the switch state recognition result to the inspection switch image to obtain the switch state of the rotary switch.

7. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the rotary switch state detection method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the rotary switch state detection method as described in any one of claims 1 to 5.

Images