Circuit breaker plastic fixture quality inspection method and system based on vision assistance

Abstract

The present application relates to the field of image data processing technology, and specifically to a visually assisted circuit breaker plastic fixture quality detection method and system, including: obtaining a set of windows to be tested based on the average signal value of all pixels in a feature window area and the frequency value difference of pixels in a corresponding frequency domain image, obtaining a first feature window area and a second feature window area based on the position of the feature window area, obtaining the necessity of merging based on the peak value between the second feature window areas and the difference in selected evaluation parameters, merging the second feature window area, and further using a quality detection model to obtain a quality-unqualified area. The present application performs regional analysis and merging on the image, while reducing the amount of calculation, avoids the problem of decreased detection accuracy caused by over-segmentation of the quality-unqualified area by the feature window area, and improves the accuracy of quality detection.

Description

Technical Field

[0001] The present application relates to the technical field of image data processing, and in particular to a visually assisted quality detection method and system for a circuit breaker plastic fixture. Background Art

[0002] Circuit breakers are important control and protection devices in power systems. During normal operation, they connect or disconnect the no-load current and load current of the circuit to play a control role. When a fault occurs, the circuit breaker, protective device and automatic device work together to cut off the fault current and protect the safe operation of the fault-free part. Therefore, the quality of the circuit breaker is crucial to its stable operation.

[0003] Circuit breaker plastic fixtures are usually used in the production process of circuit breakers to support and fix the device. During the use of circuit breaker plastic fixtures, surface defects such as wear and cracks will occur, and a large number of unqualified quality areas will be formed on the fixture surface. If unqualified fixtures are used for fixing during the production process of circuit breakers, the production quality of the circuit breaker will not meet the standards.

[0004] At present, the method of using machine vision and deep learning for auxiliary detection has been popularized in many fields due to its advantages of fast detection speed, high precision, and lightweight deployment. The detection results of this type of quality detection method depend on the input data of the detection object. For tasks such as quality detection of circuit breaker fixtures, it is usually necessary to directly obtain the appearance image and train the detection model. However, since the manifestations of unqualified circuit breakers are not consistent, such as unqualified quality is caused by inconsistent surface defect shapes of circuit breaker fixtures, resulting in inconsistent feature window sizes. Therefore, spatial pyramid pooling is often used for processing, and some feature windows contain interference information, resulting in inaccurate quality detection results. Summary of the invention

[0005] The present application provides a visually assisted circuit breaker plastic fixture quality detection method and system to solve the existing problems.

[0006] In a first aspect, the present application provides a visually assisted circuit breaker plastic fixture quality detection method, the method comprising the following steps:

[0007] The surface image of the circuit breaker plastic fixture is collected, and the surface image of the circuit breaker plastic fixture is segmented and convolved to obtain a number of characteristic window areas, and the gray value of the pixel point in the characteristic window area is called the signal value;

[0008] A frequency domain image of the feature window region is obtained, wherein one pixel in the frequency domain image corresponds to one frequency value, and a selection evaluation parameter of the feature window region is obtained according to the difference between the average signal value of all pixels in the feature window region and the frequency value of the corresponding pixel in the frequency domain image, wherein the selection evaluation parameter is used to describe the probability that the feature window region is a quality-unqualified region; and a set of windows to be tested is obtained according to the difference in the selection evaluation parameter;

[0009] According to the positional relationship between the characteristic window areas in the window set to be tested, a first characteristic window area and a second characteristic window area are obtained, a peak value of any second characteristic window area is obtained, and the necessity of merging the second characteristic window areas is obtained according to the peak values ​​between the second characteristic window areas and the difference of the selected evaluation parameters, and the second characteristic window areas are merged according to the size of the necessity of merging to obtain the final characteristic window area;

[0010] Obtaining a quality inspection result of the circuit breaker plastic fixture to be inspected by using the unqualified quality areas in the first characteristic window area and the final characteristic window area;

[0011] The specific method of obtaining the selection evaluation parameter of the feature window area according to the average signal value of all pixels in the feature window area and the frequency value difference of the pixels in the corresponding frequency domain image includes:

[0012] The average signal value of all pixels in any feature window area is recorded as the signal parameter of the feature window area, and the selection evaluation parameter of any feature window area is obtained. The specific calculation method is:

[0013]

[0014] in, represents the selection evaluation parameter of the feature window area; S represents the normalized result of the signal parameter of the feature window area; Represents the information entropy of all signal values ​​in the feature window area; Indicates the number of pixels in the frequency domain image corresponding to the feature window area; Indicates the feature window area corresponding to the first The frequency value of each pixel; Indicates the minimum frequency value in the frequency domain image corresponding to the feature window area, Represents the standard deviation of the frequency values ​​of all pixels in the frequency domain image corresponding to the feature window area.

[0015] Furthermore, the specific method of segmenting and convolution processing the surface image of the circuit breaker plastic fixture to obtain a plurality of feature window areas includes:

[0016] Performing threshold segmentation on the circuit breaker plastic fixture surface image to obtain a foreground image of the circuit breaker plastic fixture surface image;

[0017] The foreground image of the surface image of the circuit breaker plastic fixture is convolved through the convolution layer of the neural network, and the obtained image is recorded as the signal image. The pixels in the signal image are clustered according to the grayscale value of the pixels in the signal image and the distance between the coordinates. Several clusters are obtained, and the connected domains of the clusters are extracted. Any connected domain formed by the pixels in any cluster is recorded as the feature window area.

[0018] Furthermore, the specific method of obtaining the frequency domain image of the feature window area includes:

[0019] Any characteristic window area in the signal image is used as an image template, and the corresponding area in the surface image of the circuit breaker plastic fixture is extracted using the image template to obtain the corresponding original image area. The original image area is Fourier transformed to obtain the frequency domain image of the characteristic window area.

[0020] Furthermore, the specific method of obtaining the set of windows to be tested according to the difference of the selected evaluation parameters is as follows:

[0021] First, the selected evaluation parameters of all feature window areas are arranged in descending order to obtain a parameter sequence. Denoted as parameter difference coefficient, where Indicates the first The selection evaluation parameters of the feature window area are Indicates the first Selection evaluation parameters of feature window areas;

[0022] Obtain two characteristic window areas corresponding to when the parameter difference coefficient is the largest, record them as the first area, and record the first area with the largest evaluation parameter as the second area;

[0023] Then, the selected evaluation parameter of the second area is recorded as the target parameter, and the set formed by all the corresponding characteristic window areas between the maximum value of the selected evaluation parameter and the target parameter in the parameter sequence is recorded as the window set to be tested.

[0024] Furthermore, the obtaining of the first characteristic window area and the second characteristic window area according to the positional relationship between the characteristic window areas in the window set to be tested includes the following specific methods:

[0025] Any feature window area in the window set to be tested is recorded as a target area. When the target area in the window set to be tested is not adjacent to all other feature window areas, the target area is recorded as the first feature window area; when the target area in the window set to be tested is adjacent to any other feature window area, the target area is recorded as the second feature window area.

[0026] Furthermore, the obtaining of the peak value of any second characteristic window area includes the following specific methods:

[0027] For any set of adjacent regions, an adaptive peak detection algorithm is used to obtain a number of peaks of a sequence composed of all signal values ​​in any second characteristic window region in the set of adjacent regions.

[0028] Furthermore, the necessity of merging the second feature window areas is obtained according to the peak values ​​between the second feature window areas and the difference in the selected evaluation parameters, including the specific method of:

[0029] First, the Euclidean distance between any two peaks in any second feature window region is obtained and recorded as a distance parameter, and the mean of all distance parameters in the second feature window region is recorded as a distance factor of the second feature window region;

[0030] Then, the necessity of merging two adjacent second feature window areas in any adjacent area set is obtained, and the specific calculation method is as follows:

[0031]

[0032] in, Indicates the second feature window area With the second feature window area Necessity of merger between Indicates the second feature window area The distance factor of Indicates the second feature window area The distance factor of Indicates the second feature window area Selection of evaluation parameters; Indicates the second feature window area Selection of evaluation parameters; Represents an exponential function with a natural constant as base.

[0033] Furthermore, the second feature window regions are merged according to the size of the merging necessity to obtain the final feature window region, including the specific method of:

[0034] All the merging necessity in the adjacent area set is greater than or equal to the threshold The two second feature window areas are merged to obtain a new feature window area, the second feature window area and the new feature window area in the adjacent area set are collectively referred to as element areas, the necessity of merging any two adjacent element areas is obtained, and the merging operation is repeated until the necessity of merging any two adjacent element areas in the adjacent area set is less than a threshold , the element area in the final set of adjacent areas is recorded as the final feature window area, where the threshold are the preset hyperparameters.

[0035] In the second aspect, an embodiment of the present application also provides a circuit breaker plastic clamp quality detection system based on visual assistance, including a memory, a processor, and a computer program stored in the memory and running on the processor, and when the processor executes the computer program, it implements the steps of any one of the above-mentioned circuit breaker plastic clamp quality detection methods based on visual assistance.

[0036] The beneficial effect of the technical solution of the present application is that by using the SPP layer for feature recognition, the original morphological features of the surface of the circuit breaker plastic fixture in the unqualified quality area in the image can be maintained as much as possible, and the significant features of the signal values ​​between the feature window areas are used to merge the areas based on the information frequency differences in the feature window areas. While reducing the amount of calculation, the problem of decreased detection accuracy caused by over-segmentation of the feature window area for the unqualified quality area is avoided, thereby improving the accuracy of quality detection of the circuit breaker plastic fixture. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0038] Figure 1 A flowchart of the steps of the visually assisted circuit breaker plastic fixture quality detection method provided in this application;

[0039] Figure 2 A flowchart for obtaining a set of windows to be tested provided by this application;

[0040] Figure 3 This is a flowchart for obtaining the final feature window area provided in this application. DETAILED DESCRIPTION

[0041] In order to further explain the technical means and effects adopted by the present application to achieve the predetermined invention purpose, the following, in combination with the accompanying drawings and preferred embodiments, describes in detail the specific implementation method, structure, features and effects of the visually assisted circuit breaker plastic fixture quality detection method and system proposed in the present application. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0042] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

[0043] The specific scheme of the visual-assisted circuit breaker plastic fixture quality detection method and system provided in the present application is described in detail below with reference to the accompanying drawings.

[0044] See also Figure 1 , which shows a flowchart of a visually assisted circuit breaker plastic fixture quality detection method provided by an embodiment of the present application, the method comprising the following steps:

[0045] Step S001: The surface image of the circuit breaker plastic fixture is collected by a camera, and the surface image of the circuit breaker plastic fixture is segmented and convolved to obtain a number of feature window areas.

[0046] It should be noted that the scenario targeted by this embodiment is: when using machine vision and neural network technology to assist in quality inspection of circuit breaker plastic fixtures, the feature window of the SPP (Spatial PyramidPooling) layer in the quality inspection model network structure can be used to obtain areas in the surface images of circuit breaker plastic fixtures of different sizes. However, when there are factors such as lighting, noise, and transmission distortion, the image information on the surface of the circuit breaker plastic fixture is easily disturbed, which further causes the pooling process of the quality inspection model network to be disturbed, thereby affecting the accuracy of quality inspection of the circuit breaker plastic fixture.

[0047] Specifically, in order to implement the visual-assisted circuit breaker plastic fixture quality detection method proposed in this embodiment, it is first necessary to collect the circuit breaker plastic fixture surface image, and the specific process is as follows:

[0048] Firstly, images of each surface of the circuit breaker plastic fixture are collected by a camera, which are all recorded as the surface images of the circuit breaker plastic fixture. The camera is an industrial camera, and the acquired images are RGB images. The higher the resolution of the camera, the higher the clarity of the acquired image, and the more accurate the final quality inspection result of the circuit breaker plastic fixture.

[0049] For any surface image of the circuit breaker plastic fixture, the circuit breaker plastic fixture surface image is segmented by Otsu threshold to obtain a foreground image in the circuit breaker plastic fixture surface image, which is recorded as the foreground image to be detected. Otsu threshold segmentation is a well-known technology and will not be described in detail in this embodiment.

[0050] It should be noted that since the plastic fixture of the circuit breaker is irregular, there will be certain differences in the geometric shapes and structures of each surface. Therefore, cameras arranged in multiple directions are often used to capture images of each surface. Since the SPP layer of the neural network has no requirements on the size of the feature window, there is no need to crop the image size.

[0051] Furthermore, when training the quality inspection model for quality inspection of circuit breaker plastic fixtures, the foreground image to be inspected will first pass through the convolution layer in the network structure of the quality inspection model for a convolution operation, and the image obtained by the convolution will be used as the signal image of the foreground image to be inspected. The grayscale value of the pixel in the signal image is recorded as the signal value, based on the signal value of the pixel in the signal image and the distance between the coordinates.

[0052] Specifically, the pixel points in the signal image are used as input, and the metric distance between the two pixel points in the clustering process is determined based on the difference in the signal value between the two pixel points and the Euclidean distance between the pixel point coordinates. The pixel points in the signal image are clustered using the DBSCAN clustering algorithm to obtain several clusters, and the connected domains of the clusters are extracted, and any connected domain formed by the pixel points in any cluster is recorded as the feature window area. Among them, the calculation formula of the metric distance D between two pixel points is:

[0053]

[0054] In the formula, is the Euclidean distance between two pixels, is the absolute value of the difference between the signal values ​​of two pixels, , are weight parameters, which are used to balance the influence of signal value and distance on clustering results. In this embodiment, , They are set to 0.5 and 0.5 respectively, and can be adjusted according to actual conditions. This application does not make any specific limitations on this.

[0055] It should be noted that, in this embodiment, the convolutional layer size of the preset neural network is ,in , and preset the number of convolutional layers of the neural network , can be adjusted according to actual conditions, and this application does not make specific limitations on this.

[0056] It should be noted that the Chinese name of the DBSCAN (Density-Based Spatial Clustering of Applications with Noise) clustering algorithm is density-based clustering method with noise. Since the DBSCAN clustering algorithm is an existing clustering algorithm, it is not described in detail in this embodiment.

[0057] Step S002: Obtain a frequency domain image of the feature window area, obtain selection evaluation parameters of the feature window area according to the average signal value of all pixels in the feature window area and the frequency value of the corresponding pixel in the frequency domain image; obtain a set of windows to be tested according to the difference in the selected evaluation parameters.

[0058] It should be noted that in this application, the signal value in the feature window area is used to distinguish the grayscale fluctuation caused by the unqualified quality area in the current original grayscale image, and different grayscale differences produce different signal distributions, and then the noise interference window is excluded by the distribution between the high and low frequencies of the signal, the noise interference is reduced, and the number of feature window areas is reduced. The specific process is as follows Figure 2 As shown, it includes step 201 and step 202.

[0059] 201 : Calculate the selection evaluation parameter of each feature window based on the frequency information of the signal value in each feature window.

[0060] Firstly, for any signal image, any characteristic window area in the signal image is used as an image template, and the coordinate information of the pixel points in the image template is used to extract the corresponding area from the surface image of the circuit breaker plastic fixture where the image template is located, so as to obtain the original image area corresponding to the image template, and the original image area is subjected to Fourier transform, and the Fourier transform result is recorded as the frequency domain image of the characteristic window area. One pixel point in the frequency domain image corresponds to one frequency value. Fourier transform is a commonly used technology in the field of image processing, and the specific process will not be repeated here.

[0061] It should be noted that, when analyzing the frequency domain image, the lower the frequency of the low-frequency information, the more frequent its occurrence, which means that there are fewer grayscale fluctuations in the frequency domain area and more normal background areas in the feature window area.

[0062] Then, the average signal value of all pixels in any feature window area is recorded as the signal parameter of the feature window area, the signal parameters of all signal parameters in the signal image are obtained, and the maximum-minimum normalization processing is performed to obtain the normalized result of each signal parameter. The maximum-minimum normalization processing is a commonly used technology in the field of data processing, and the specific process will not be repeated.

[0063] Get the selection evaluation parameters of any feature window area. The specific calculation method is:

[0064]

[0065] in, represents the selection evaluation parameter of the feature window area; S represents the normalized result of the signal parameter of the feature window area; Represents the information entropy of all signal values ​​in the feature window area; Indicates the number of pixels in the frequency domain image corresponding to the feature window area; Indicates the feature window area corresponding to the first The frequency value of each pixel; Indicates the minimum frequency value in the frequency domain image corresponding to the feature window area, Represents the standard deviation of the frequency values ​​of all pixels in the frequency domain image corresponding to the feature window area.

[0066] It should be noted that the selected evaluation parameter is used to describe the probability that the feature window area is a quality-unqualified area. The larger the selected evaluation parameter, the greater the probability that the feature window area is a quality-unqualified area, and vice versa.

[0067] It should be noted that the signal strength of pixels at different positions in the signal image reflects the difference between the frequencies corresponding to the high-frequency information under the brightness change. Therefore, the selection evaluation parameters of the feature window area are obtained by combining the spectrum distribution in the signal strength comprehensive image.

[0068] It should be noted that the larger the normalized value of the signal parameter in the feature window area, the The closer the value is to 1, the stronger the characteristics of the feature window area in the convolution process, and the more attention should be paid to the spectral characteristics in the feature window area.

[0069] It should be noted that It represents the difference between the frequency value of the data point in the frequency domain image corresponding to the feature window area and the minimum frequency value. The larger the difference, the less low-frequency information and the more high-frequency information in the feature window area, and the higher the possibility that the feature window area is a quality-unqualified area.

[0070] It should be noted that the greater the information entropy of all signal values ​​in the feature window area, the more frequently the signal values ​​of the pixels in the feature window area change, the more inconsistent the amplitude of the signal value changes of the pixels in the feature window area, and the greater the difference between the high-frequency information and the low-frequency information, which means that the edge information on the surface of the circuit breaker plastic fixture is less likely to be affected by light.

[0071] It should be noted that when the surface of the circuit breaker plastic fixture is relatively flat, fewer brightness edges are generated in the image, or when the edge texture of the circuit breaker plastic fixture surface is relatively regular, the edge information in the image is strong but the grayscale types are relatively small, which can easily cause high-frequency information to be masked by a large amount of low-frequency information, resulting in the convolution of the overall area. The signal value is less significant in the image, so the selection evaluation parameter of the corresponding area is smaller.

[0072] It should be noted that when the irregular characteristic edges are produced by wear or damage on the surface of the circuit breaker plastic fixture, there are many types of grayscale values ​​of the pixels around the edge pixels, which leads to a stronger frequency of high-frequency information while the frequency of low-frequency information is generally higher. In addition, more types of grayscale values ​​lead to a larger information entropy corresponding to the area, so the selection evaluation of the corresponding area is higher.

[0073] 202 , screening the characteristic window regions based on the selection evaluation parameters of all characteristic window regions in the signal image to obtain a set of windows to be tested.

[0074] First, the selected evaluation parameters of all feature window areas are arranged in descending order to obtain a parameter sequence. Denoted as parameter difference coefficient, where Indicates the first The selection evaluation parameters of the feature window area are Indicates the first The evaluation parameters of the selected feature window areas are obtained; the two feature window areas corresponding to the maximum parameter difference coefficient are obtained, the two feature window areas are recorded as the first area, and the first area with the largest selected evaluation parameter is recorded as the second area.

[0075] Then, the selected evaluation parameter of the second area is recorded as the target parameter, and the set formed by all the corresponding characteristic window areas between the maximum value of the selected evaluation parameter and the target parameter in the parameter sequence is recorded as the window set to be tested.

[0076] It should be noted that, for each signal image, if the difference in the possibility that the two feature window areas are unqualified quality areas is greater, then the corresponding parameter difference coefficient between the selected evaluation parameters of the two feature window areas is greater. When the unqualified quality situation in one feature window area is the most significant and the unqualified quality situation in another feature window area is the least significant, the parameter difference coefficient is the maximum value. Since the parameter sequence is arranged in descending order, in this embodiment, the parameter difference coefficient is the largest, and the corresponding two selected evaluation parameters are adjacent elements in the sequence. At this time, one should contain unqualified quality areas and the other should not contain unqualified quality areas, thereby determining two first areas; and the second area is the first area with the largest selected evaluation parameter, then the second area is the area with the least unqualified quality situation in all feature window areas containing unqualified quality areas in the signal image. At this time, the set of windows to be tested selected by taking the target parameter as the lower limit represents all feature windows containing unqualified quality areas in the signal image, and the screening of the feature window areas is completed.

[0077] Step S003: Obtain the first feature window area and the second feature window area according to the positional relationship between the feature window areas in the window set to be tested; obtain the necessity of merging the second feature window areas according to the peak values ​​between the second feature window areas and the difference in selected evaluation parameters, and obtain the final feature window area based on the necessity of merging.

[0078] It should be noted that in the set of windows to be tested, the abnormal quality failure caused by the difference in feature signals may cause signal concentration during the convolution process, resulting in the grayscale fluctuation position on the quality failure area in some feature areas being attracted by the high-frequency edge and the resulting high-frequency signal masking the low-frequency signal. Therefore, it is necessary to integrate the position distribution relationship of the window in the feature map to merge the feature areas to provide spatial feature information, and then integrate the signal strength difference in the feature window to determine whether the surrounding area has generated signal masking features, and obtain the necessity of merging between windows, that is, the acquisition process of the final feature window area is as follows: Figure 3 As shown, it includes step 301 and step 302.

[0079] 301 , dividing the window set to be tested, and obtaining a plurality of peak values ​​in a second characteristic window area.

[0080] First, any feature window area in the window set to be tested is recorded as a target area. When the target area in the window set to be tested is not adjacent to all other feature window areas, the target area is recorded as the first feature window area; when the target area in the window set to be tested is adjacent to any other feature window area, the target area is recorded as the second feature window area.

[0081] Then, a set formed by a plurality of adjacent second feature window regions is recorded as an adjacent region set, and a plurality of adjacent region sets are obtained.

[0082] Finally, for any set of adjacent regions, an adaptive peak detection algorithm is used to obtain several peaks of the sequence composed of signal values ​​in any second feature window area in the set of adjacent regions. The adaptive peak detection algorithm is a commonly used technology in the field of data processing, and the specific process will not be repeated here.

[0083] 302 , merging multiple second characteristic window areas to obtain a final characteristic window area.

[0084] First, the Euclidean distance between any two peaks in any second feature window region is obtained and recorded as a distance parameter, and the mean of all distance parameters in the second feature window region is recorded as a distance factor of the second feature window region.

[0085] Then, the necessity of merging two adjacent second feature window areas in any adjacent area set is obtained, and the specific calculation method is as follows:

[0086]

[0087] in, Indicates the second feature window area With the second feature window area Necessity of merger between Indicates the second feature window area The distance factor of Indicates the second feature window area The distance factor of Indicates the second feature window area Selection of evaluation parameters; Indicates the second feature window area Selection of evaluation parameters; Represents an exponential function with a natural constant as base.

[0088] It should be noted that when the distance factor difference between two adjacent second feature window areas The closer it is to 0, the closer the signal value distribution between two adjacent second feature window areas is. The closer it is to 0, that is, the closer the selection evaluation parameters between two adjacent second feature window areas are, the more similar it is, indicating that the signal value difference and the high- and low-frequency information content between the two adjacent second feature window areas are, and the more the two second feature window areas need to be merged.

[0089] Finally, all the merging necessity in the adjacent area set is greater than or equal to the threshold The two second feature window areas are merged to obtain a new feature window area. The second feature window area and the new feature window area in the adjacent area set are collectively referred to as element areas. This can prevent the same area from being merged multiple times, obtain the necessity of merging any two adjacent element areas, and repeat the merging operation until the necessity of merging any two adjacent element areas in the adjacent area set is less than the threshold , the element area in the final set of adjacent areas is recorded as the final feature window area, where the threshold are the preset hyperparameters.

[0090] It should be noted that in this embodiment, the threshold is preset based on experience. It is 0.75, which can be adjusted according to actual conditions, and this application does not make any specific limitation on this.

[0091] Step S004: Perform quality inspection using the first characteristic window area and the final characteristic window area in the signal image to obtain a quality inspection result of the circuit breaker plastic fixture.

[0092] It should be noted that in the present application, the network structure of the quality detection model is a convolutional neural network CNN. The SPP layer in the network structure processes feature window areas of different sizes through template sizes of the same size to obtain feature vectors of the same scale; since the SPP layer belongs to a layer in the CNN network, the classification result of the network is finally output through the fully connected layer of the network.

[0093] Specifically, first, obtain The first characteristic window area and the final characteristic window area of ​​all the circuit breaker plastic fixture surface images of different shapes and different lighting conditions are obtained according to the above steps S001-S003, wherein are the preset hyperparameters.

[0094] It should be noted that the hyperparameters are preset based on experience. It is 500, which can be adjusted according to actual conditions and is not specifically limited in this application.

[0095] Then, an image annotation tool is used to manually annotate all first feature window areas and final feature window areas of the surface image of the circuit breaker plastic fixture at the pixel level. The manual labels corresponding to the pixel-level annotations are 0, 1, 2, and 3, respectively, representing normal areas, slightly unqualified quality areas, moderately unqualified quality areas, and severely unqualified quality areas. The set formed by all first feature window areas and final feature window areas with manual labels is used as the input of the SPP layer of the CNN neural network, and the CNN neural network is trained. The loss function in the training process is a multi-classification cross entropy function, and the optimizer is an Adam (Adaptive Moment Estimation) optimizer. The model after training is used as the quality detection model of the circuit breaker plastic fixture. The neural network training is a well-known technology, and the specific process will not be repeated here.

[0096] It should be noted that the commonly used SPP layer template group is selected in this embodiment as , so the output feature vector size is The feature vector of the size is used as the feature vector output by the SPP layer. The SPP layer template group can be adjusted according to actual conditions, and this application does not make specific limitations.

[0097] Finally, the quality inspection model is used to perform quality inspection on the circuit breaker plastic fixture to be inspected, and the areas labeled 1, 2, and 3 in the output results of the quality inspection model are recorded as unqualified quality areas.

[0098] Based on the same inventive concept as the above method, an embodiment of the present invention also provides a circuit breaker plastic fixture quality detection system based on visual assistance, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, the steps of any one of the above-mentioned circuit breaker plastic fixture quality detection methods based on visual assistance are implemented.

[0099] It should be noted that the above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

[0100] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

Claims

1. A visually assisted quality inspection method for circuit breaker plastic fixtures, characterized in that: The method comprises the following steps: The surface image of the circuit breaker plastic fixture is collected, and the surface image of the circuit breaker plastic fixture is segmented and convolved to obtain a number of characteristic window areas, and the gray value of the pixel point in the characteristic window area is called the signal value; A frequency domain image of the feature window region is obtained, wherein one pixel in the frequency domain image corresponds to one frequency value, and a selection evaluation parameter of the feature window region is obtained according to the difference between the average signal value of all pixels in the feature window region and the frequency value of the corresponding pixel in the frequency domain image, wherein the selection evaluation parameter is used to describe the probability that the feature window region is a quality-unqualified region; and a set of windows to be tested is obtained according to the difference in the selection evaluation parameter; According to the positional relationship between the characteristic window areas in the window set to be tested, a first characteristic window area and a second characteristic window area are obtained, a peak value of any second characteristic window area is obtained, and the necessity of merging the second characteristic window areas is obtained according to the peak values ​​between the second characteristic window areas and the difference of the selected evaluation parameters, and the second characteristic window areas are merged according to the size of the necessity of merging to obtain the final characteristic window area; Obtaining a quality inspection result of the circuit breaker plastic fixture to be inspected by using the unqualified quality areas in the first characteristic window area and the final characteristic window area; The specific method of obtaining the selection evaluation parameter of the feature window area according to the average signal value of all pixels in the feature window area and the frequency value difference of the pixels in the corresponding frequency domain image includes: The average signal value of all pixels in any feature window area is recorded as the signal parameter of the feature window area, and the selection evaluation parameter of any feature window area is obtained. The specific calculation method is: in, represents the selection evaluation parameter of the feature window area; S represents the normalized result of the signal parameter of the feature window area; Represents the information entropy of all signal values ​​in the feature window area; Indicates the number of pixels in the frequency domain image corresponding to the feature window area; Indicates the feature window area corresponding to the first The frequency value of each pixel; Indicates the minimum frequency value in the frequency domain image corresponding to the feature window area, Represents the standard deviation of the frequency values ​​of all pixels in the frequency domain image corresponding to the feature window area.

2. The visually assisted circuit breaker plastic fixture quality inspection method according to claim 1 is characterized in that: The specific method of segmenting and convolution processing the surface image of the circuit breaker plastic fixture to obtain a number of feature window areas includes: Performing threshold segmentation on the circuit breaker plastic fixture surface image to obtain a foreground image of the circuit breaker plastic fixture surface image; The foreground image of the surface image of the circuit breaker plastic fixture is convolved through the convolution layer of the neural network, and the obtained image is recorded as the signal image. The pixels in the signal image are clustered according to the grayscale value of the pixels in the signal image and the distance between the coordinates. Several clusters are obtained, and the connected domains of the clusters are extracted. Any connected domain formed by the pixels in any cluster is recorded as the feature window area.

3. The visually assisted circuit breaker plastic fixture quality inspection method according to claim 1 is characterized in that: The specific method of obtaining the frequency domain image of the characteristic window area includes: Any characteristic window area in the signal image is used as an image template, and the corresponding area in the surface image of the circuit breaker plastic fixture is extracted using the image template to obtain the corresponding original image area. The original image area is Fourier transformed to obtain the frequency domain image of the characteristic window area.

4. The visually assisted circuit breaker plastic fixture quality inspection method according to claim 1 is characterized in that: The specific method of obtaining the set of windows to be tested according to the difference of the selected evaluation parameters is as follows: First, the selected evaluation parameters of all feature window areas are arranged in descending order to obtain a parameter sequence. Denoted as parameter difference coefficient, where Indicates the first The selection evaluation parameters of the feature window area are Indicates the first Selection evaluation parameters of feature window areas; Obtain two characteristic window areas corresponding to when the parameter difference coefficient is the largest, record them as the first area, and record the first area with the largest evaluation parameter as the second area; Then, the selected evaluation parameter of the second area is recorded as the target parameter, and the set formed by all the corresponding characteristic window areas between the maximum value of the selected evaluation parameter and the target parameter in the parameter sequence is recorded as the window set to be tested.

5. The visually assisted circuit breaker plastic fixture quality inspection method according to claim 1 is characterized in that: The method of obtaining the first characteristic window area and the second characteristic window area according to the positional relationship between the characteristic window areas in the window set to be tested includes: Record any feature window region in the test window set as a target region, and when the target region in the test window set is not adjacent to all other feature window regions, record the target region as the first feature window region; When the target area in the window set to be tested is adjacent to any other characteristic window area, the target area is recorded as the second characteristic window area.

6. The visually assisted circuit breaker plastic fixture quality inspection method according to claim 1 is characterized in that: The specific method of obtaining the peak value of any second characteristic window area includes: For any set of adjacent regions, an adaptive peak detection algorithm is used to obtain a number of peaks of a sequence composed of all signal values ​​in any second characteristic window region in the set of adjacent regions.

7. The visually assisted circuit breaker plastic fixture quality inspection method according to claim 1 is characterized in that: The necessity of merging the second feature window regions according to the peak values ​​between the second feature window regions and the difference in the selected evaluation parameters is obtained, and the specific method includes: First, the Euclidean distance between any two peaks in any second feature window region is obtained and recorded as a distance parameter, and the mean of all distance parameters in the second feature window region is recorded as a distance factor of the second feature window region; Then, the necessity of merging two adjacent second feature window areas in any adjacent area set is obtained, and the specific calculation method is as follows: in, Indicates the second feature window area With the second feature window area Necessity of merger between Indicates the second feature window area The distance factor of Indicates the second feature window area The distance factor of Indicates the second feature window area Selection of evaluation parameters; Indicates the second feature window area Selection of evaluation parameters; Represents an exponential function with a natural constant as base.

8. The visually assisted circuit breaker plastic fixture quality inspection method according to claim 1 is characterized in that: The method of merging the second feature window areas according to the size of the merging necessity to obtain the final feature window area includes: All the merging necessity in the adjacent area set is greater than or equal to the threshold The two second feature window areas are merged to obtain a new feature window area, the second feature window area and the new feature window area in the adjacent area set are collectively referred to as element areas, the necessity of merging any two adjacent element areas is obtained, and the merging operation is repeated until the necessity of merging any two adjacent element areas in the adjacent area set is less than a threshold , the element area in the final set of adjacent areas is recorded as the final feature window area, where the threshold are the preset hyperparameters.

9. A circuit breaker plastic fixture quality inspection system based on visual assistance, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of the visual-assisted circuit breaker plastic fixture quality detection method as described in any one of claims 1 to 8 are implemented.

Images