Intelligent visual inspection and weighing device and method
By collecting and processing images of transparent plastic parts, combined with the Faster R-CNN method, identifying and marking defect types, the problem of insufficient accuracy in defect recognition in the prior art is solved, and more efficient and accurate defect detection is achieved.
Patent Information
- Application Number
- CN202510215914.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-02-25
- Publication Date
- 2025-06-13
AI Technical Summary
In the machine vision defect monitoring of transparent plastic parts, the image acquisition, processing and recognition processes are complex, resulting in the impact of recognition accuracy and insufficient defect recognition accuracy.
By collecting images of transparent parts, forming data sets, and verifying them on the training network model, the accurate target image is obtained. Then the image is grayed out, Gaussian filtering, gradient value and direction calculation, non-maximum suppression and double-threshold detection to obtain an edge binary map. Finally, the Faster R-CNN method is used to identify and mark defect types in the image.
It improves the accuracy of image data and the accuracy of defect recognition, can capture tiny details, significantly shorten the detection cycle, improve production efficiency, and realize automated production processes.
Smart Images
Figure CN120147252A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of defect recognition, and particularly relates to an intelligent vision detection and weighing device and method. Background Art
[0002] An intelligent vision detection and weighing device is an advanced device integrating computer vision technology and weighing sensors. It can collect image information of commodities through a camera, use deep learning algorithms to identify and process the images, and combine the data of the weighing sensors to achieve automatic identification. The intelligent vision detection and weighing device is widely used in retail places such as supermarkets, convenience stores, and farmers' markets, and is particularly suitable for weighing and settlement of fresh commodities. In addition, the device can also be applied to the field of industrial automation, such as material identification and counting on production lines.
[0003] The region detection method based on Faster R-CNN is an efficient and accurate object detection algorithm, which is of great significance in the field of object detection. By introducing RPN, the algorithm can quickly generate accurate candidate regions, reduce the amount of calculation, improve the running speed, and achieve its efficiency. The algorithm can process input images and candidate regions of any size, and is applicable to different application scenarios, highlighting its flexibility. The region detection method based on Faster R-CNN is an efficient and accurate object detection algorithm, which has application prospects in multiple fields, especially in the fields of industrial automation and quality control.
[0004] In the prior art, for the defect monitoring method of machine vision for transparent plastic parts, it generally includes image acquisition and image processing. The image processing process includes processes such as feature extraction and segmentation, then determines whether there are defects, and finally identifies and classifies the defects, etc. However, in the process of image acquisition, processing, and recognition, the process of layer-by-layer processing affects the accuracy of the final recognition. The image accuracy of the vision detection process can be further improved to improve the accuracy of the final defect recognition. Summary of the Invention
[0005] Aiming at the problems existing in the prior art, the present invention provides an intelligent vision detection and weighing device and method. By forming a data set through image acquisition and verifying it on a training network model, the accuracy of the image data is improved, then the image is processed to divide the defect area, and then the Faster R-CNN method is combined to complete the identification and marking of the types of image defects.
[0006] The technical solution of the present invention is realized as follows:
[0007] An intelligent vision detection method includes the following steps:
[0008] S1. Collect images of the transparent parts. The pixel values of the images of each transparent part are the same, and a dataset of all transparent part images is obtained;
[0009] S2. Divide the dataset of transparent part images and verify it on the training network model to obtain the target image;
[0010] S3. Grayscale the target image, filter the grayscale image using the Gaussian filter function, obtain the gradient value and direction based on the resulting image, then find the maximum value in the local area of the pixel points by comparing the gradient values before and after the gradient direction, select double-threshold detection and connect the edges to obtain the edge binary image, and obtain the defect area based on the edge binary image;
[0011] S4. Identify the defect area. Based on the Faster R-CNN method, identify and mark the types of defects in the image. Extract the primary features in the sparse filtering layer as the input layer of the convolutional part, preliminarily extract the regions of interest in the RPN layer, and then perform position marking and defect type identification.
[0012] Further, in S1, the specific method for collecting images of the transparent parts includes:
[0013] Set a light source on the back of the transparent part. The light shines on the transparent part and projects into the camera. The camera records the image of the transparent part and transmits it to the computer for storage, forming a dataset of transparent part images; The methods for the camera to collect transparent part images include rotation, longitudinal movement, horizontal movement, zooming, horizontal flipping, and vertical flipping. The image data sets of different angles of the transparent part obtained form a dataset of transparent part images.
[0014] Further, in S2, divide the dataset S of transparent part images into X non-overlapping data subsets. Set the training samples as P. According to the training samples and the data subsets, the number of samples in the data subsets is P / X. Each data subset is S = {S 1 , S 2 ... S x}. Extract any one data subset, use the remaining data subsets as the training network model, verify the extracted data subset on the training network model, and obtain the verification result of the extracted data subset;
[0015] Verify all the data subsets on the corresponding training network models to obtain the verification results of all the data subsets, which is the training set. According to the verification results, obtain the accuracy rate of the training set, and select the verification result closest to the accuracy rate. The corresponding data subset is the target image.
[0016] Further, the specific method for obtaining the gradient value and direction in S3 is to calculate the gradient value of each pixel using the Sobel function based on the X and Y directions, and then obtain the direction of the gradient value of each pixel; that is, Gx and Gy, where Gx represents the horizontal gradient component of the pixel point, and Gy represents the vertical gradient component of the pixel point.
[0017] The non-maximum suppression method is used to detect edges: compare the gradient value of the pixel point with the gradient values of the horizontally and vertically adjacent pixel points to suppress non-maximum values.
[0018] The double thresholds are selected by the gray gradient value histogram to determine the optimal values of the double thresholds TH1 and TH2. The specific method is to take the midpoint of the rising branch of the histogram as the strong edge threshold TH1; starting from the strong edge threshold TH1, take the midpoint of the falling branch as the weak edge threshold TH2.
[0019] According to the determined double thresholds TH1 and TH2, and according to the neighborhood pixel response, determine whether the pixel point is an edge or not.
[0020] If the gradient value of the pixel point in the image is greater than the strong edge threshold TH1, the pixel point is considered a strong edge point; if the gradient value of the pixel point is less than the weak edge threshold TH2, the pixel point is considered a non-edge point; if the gradient value of the pixel point is less than TH1 and greater than TH2, its neighborhood is judged.
[0021] When the gradient value of the pixel point in the image is less than TH1 and greater than TH2, if the pixel points in its neighborhood are strong edge points, the pixel point is finally considered a strong edge point; if the pixel points in its neighborhood are non-edge points, the pixel point is finally considered a non-edge point; judge each pixel point in turn to obtain the edge binary image.
[0022] Specifically, the purpose of Gaussian filtering is to smooth the image; the Sobel operator is used to calculate the gradient value and direction to enhance the image and highlight the points with significant changes in the field.
[0023] Further, the method for obtaining the defect area according to the edge binary image is as follows:
[0024] Using the dilation algorithm, dilate the edge binary image with a 16-pixel butterfly structuring element so that the thin edges do not disappear after filling and form thick edges; count the connected components of each image and calculate the contour length, and select the contour of the connected component with the longest contour length as the defect area.
[0025] Further, in the edge binary image, according to the horizontal and vertical offsets of the structuring element, each structuring element is overlapped with the image to judge whether there are edge points in the structuring element. If there are, fill the edge points in the structuring element, and finally form a new connected component with all the filled points.
[0026] Specifically, according to the area of each connected component, calculate the contour of the connected component corresponding to the largest area, calculate the length of the contour, count the length of each connected component contour, select the longest contour, and calculate the inscribed rectangle of the longest contour; Take the centroid of the largest connected component contour as the origin of the original image, take the long side of the largest contour as the major axis, and the short side as the minor axis, calculate the rotation matrix, rotate the original image, and after rotation, the defect is located within the rectangle to obtain a new image;
[0027] Further, in S4, based on the Faster R-CNN method, identify and mark the types of defects in the image. The defect detection model includes a feature extraction layer, an RPN layer, and a Faster R-CNN layer; The sparse filtering layer in the feature extraction layer extracts primary features as the input layer of the convolutional part, and the Faster R-CNN layer performs position marking and defect type identification, specifically including:
[0028] Use the sparse filtering method to extract primary features. Obtain the gray value ranges of the defect area and the background area through the histogram of the image. In the mask corresponding to the image, the gray value of the defect area is 1, and the gray value of the background area is 0. The product of all pixels in the image and the mask forms the window of the sparse filtering function, and the primary features of the sample image are obtained through convolution;
[0029] Perform multiple convolutions on the primary features, connect the convolutional feature maps, and perform region of interest screening through the RPN layer to obtain a feature map with the same size as the image.
[0030] Specifically, the implementation process of performing multiple convolutions on the primary features includes:
[0031] Use a first convolutional kernel of size 5×5 to extract features and connect them, use a second convolutional kernel of size 3×3 to extract features and connect them, and use a third convolutional kernel of size 3×3 to extract features and connect them;
[0032] Obtain the connected primary features; On the basis of the primary features, continue to use a first convolutional kernel of size 5×5 to extract features and connect them, use a second convolutional kernel of size 3×3 to extract features and connect them, and use a third convolutional kernel of size 3×3 to extract features and connect them to obtain the connected features; The size of the primary features is 256×56;
[0033] Further, the process of screening the primary features through the RPN layer to obtain the feature map includes: Obtain the coordinates of each pixel of the feature as the input of the RPN layer;
[0034] Set the threshold of the sliding window, set the threshold in multiple intervals, calculate the arithmetic mean value of the elements in each sliding window of the feature map; Select the sliding window with a high arithmetic mean value as the region of interest, output the corresponding coordinates, and form a feature map with the same size as the input image.
[0035] Specifically, set the threshold in multiple intervals, select some elements within the sliding window for each interval, calculate the arithmetic mean of the remaining elements in the current interval, and for some elements, use the pixel corresponding to the element with the largest value among the elements in the threshold interval; filter to obtain a feature map with the same size as the input image in the feature map according to the arithmetic mean;
[0036] The RPN layer includes 12 convolutional kernels;
[0037] Input the feature map into the Faster R-CNN layer, obtain the initial offset at each position, and use the initial offset as the position label;
[0038] Obtain the initial candidate box offset, which is used to represent the center coordinates of the defect feature region in the initial candidate box; adjust the size of the initial candidate box;
[0039] Use the soft-max classifier to classify the initial candidate box into foreground and background, where the foreground is the region with defects and the background is the non-defect region;
[0040] Use a regressor to identify the defect type of the foreground. The defect types include oil stains, black spots, and bright spots, and obtain a regression label, which contains the position of the initially to-be-identified target defect in the initial candidate box;
[0041] In the Faster R-CNN layer, extract defect features from the feature map.
[0042] A weighing device is applied to an intelligent vision detection method described in any one of the above.
[0043] Compared with the prior art, the present invention has the following beneficial effects:
[0044] The present invention provides an intelligent vision detection and weighing device and method, mainly including collecting images of transparent parts from different angles to form a data set, dividing and training and validating the data set to obtain a better and accurate target image; processing the image to extract a binary image of the image edge to obtain edge position and direction information, including grayscale conversion, Gaussian filtering, calculating gradient values and directions, suppressing non-maxima, selecting double thresholds, and detecting edges, etc.; then using the Faster R-CNN method to identify and mark the types of defects in the image.
[0045] By further processing the initial data of the image to obtain an accurate target image, the subsequent processes of image segmentation, edge extraction, and defect identification are more accurate, improving the accuracy and being able to capture minute detail changes.
[0046] Using advanced image - processing algorithms, it is possible to complete the detection of a large number of samples in an extremely short time, significantly shortening the detection cycle and improving production efficiency. Combining with the Faster R - CNN method can accurately identify and locate various surface defects, which has powerful feature extraction capabilities and precise target - positioning capabilities. By analyzing and learning a large amount of detection data, the algorithm and parameters are continuously optimized to improve the accuracy and stability of detection.
[0047] The intelligent vision detection and weighing device can be linked and cooperate with other devices on the production line to achieve an automated production process. Brief Description of the Drawings
[0048] Figure 1 It is a flowchart of an intelligent vision detection method provided by an embodiment of the present invention. Detailed Embodiments
[0049] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without making creative efforts fall within the scope of protection of the present invention.
[0050] Embodiment
[0051] Such as Figure 1 , an intelligent vision detection method, comprising the following steps:
[0052] S1. Collect images of transparent parts. The pixel values of the images of each transparent part are the same, and a data set of all transparent - part images is obtained;
[0053] In the above - mentioned S1, the specific method for collecting images of transparent parts includes:
[0054] Set a light source on the back of the transparent part. The light shines on the transparent part and projects into the camera. The camera records the image of the transparent part and transmits it to the computer for storage, forming a data set of transparent - part images. The ways for the camera to collect images of transparent parts include rotation, longitudinal movement, horizontal movement, zooming, horizontal flipping, and vertical flipping, and the image - data sets of different angles of the transparent part obtained form a data set of transparent - part images.
[0055] S2. Divide the data set of transparent - part images and verify it on the training network model to obtain the target image;
[0056] In S2, the dataset S of the transparent part images is divided into X non - overlapping data subsets. Set the training samples as P. According to the training samples and the data subsets, the number of samples in each data subset is P / X. Each data subset is S = {S 1 , S 2 ……S x}. Extract any one data subset, use the remaining data subsets to train the network model, and verify the extracted data subset on the trained network model to obtain the verification result of the extracted data subset;
[0057] Verify all data subsets on their corresponding trained network models to obtain the verification results of all data subsets, which is the training set. According to the verification results, obtain the accuracy rate of the training set, and select the verification result closest to the accuracy rate. The corresponding data subset is the target image.
[0058] S3. Grayscale the target image, filter the grayscale image using the Gaussian filter function, obtain the gradient value and direction from the resulting image, then compare the gradient values before and after the gradient direction to find the maximum value in the local area of the pixel points, select double - threshold detection and connect the edges to obtain the edge binary image, and obtain the defect area according to the edge binary image;
[0059] The specific method for obtaining the gradient value and direction in S3 is to calculate the gradient value of each pixel using the Sobel function based on the X and Y directions, and then obtain the direction of each pixel's gradient value; that is, Gx and Gy, where Gx represents the horizontal gradient component of the pixel point and Gy represents the vertical gradient component of the pixel point;
[0060] Non - maximum suppression method is used to detect edges: compare the gradient value of the pixel point with the gradient values of the horizontally and vertically adjacent pixel points to suppress non - maximum values;
[0061] The double - threshold values are selected by the gray - scale gradient value histogram to determine the optimal values of the double - threshold values TH1 and TH2. The specific method is to take the mid - point of the rising branch of the histogram as the strong - edge threshold TH1; starting from the strong - edge threshold TH1, take the mid - point of the falling branch as the weak - edge threshold TH2;
[0062] According to the determined double - threshold values TH1 and TH2, and according to the neighborhood pixel response, determine whether the pixel point is an edge or non - edge;
[0063] If the gradient value of the pixel point in the image is greater than the strong - edge threshold TH1, then the pixel point is considered a strong - edge point; if the gradient value of the pixel point is less than the weak - edge threshold TH2, then the pixel point is considered a non - edge point; if the gradient value of the pixel point is less than TH1 and greater than TH2, then judge its neighborhood;
[0064] When the gradient value of a pixel point in the image is less than TH1 and greater than TH2, and the pixel points in its neighborhood are strong edge points, the pixel point is finally considered a strong edge point; if the pixel points in its neighborhood are non-edge points, the pixel point is finally considered a non-edge point; each pixel point is judged in turn to obtain a binary edge image.
[0065] The purpose of Gaussian filtering is to smooth the image; the Sobel operator is used to calculate the gradient value and direction to enhance the image and highlight the points with significant changes in the neighborhood;
[0066] The method for obtaining the defect area based on the binary edge image is as follows:
[0067] Using the dilation algorithm, dilate the binary edge image with a butterfly structuring element of 16 pixels, so that the thin edges do not disappear after filling, forming thick edges; count the connected components of each image and calculate the contour length, and select the contour of the connected component with the longest contour length, which is the defect area.
[0068] In the binary edge image, according to the horizontal and vertical offsets of the structuring element, each structuring element is overlapped with the image, and it is judged whether there are edge points in the structuring element. If there are, fill the edge points in the structuring element, and finally form a new connected component with all the filled points.
[0069] According to the area of each connected component, calculate the contour of the connected component corresponding to the largest area, calculate the length of the contour, count the contour lengths of each connected component, select the longest contour, and calculate the inscribed rectangle of the longest contour; take the centroid of the largest connected component contour of the original image as the origin, take the long side of the largest contour as the major axis, and the short side as the minor axis, calculate the rotation matrix, rotate the original image, and the defect is located within the rectangle after rotation to obtain a new image;
[0070] In this embodiment, some methods in S3 can refer to the Canny operator of the edge detection operator;
[0071] S4. Identify the defect area. Based on the Faster R-CNN method, identify and mark the types of defects in the image. Extract primary features in the sparse filtering layer as the input layer of the convolutional part. Initially extract the region of interest in the RPN layer, and then perform position marking and defect type identification.
[0072] In S4, based on the Faster R-CNN method, identify and mark the types of defects in the image. The defect detection model includes a feature extraction layer, an RPN layer, and a Faster R-CNN layer; the sparse filtering layer in the feature extraction layer extracts primary features as the input layer of the convolutional part, and the Faster R-CNN layer performs position marking and defect type identification, specifically including:
[0073] Extract the primary features using the sparse filtering method, obtain the grayscale value ranges of the defect area and the background area through the histogram of the image. In the corresponding mask of the image, the grayscale value of the defect area is 1, and the grayscale value of the background area is 0. The product of all pixels in the image and the mask forms the window of the sparse filtering function, and the primary features of the sample image are obtained through convolution.
[0074] Perform multiple convolutions on the primary features, connect the convolutional feature maps, and screen the regions of interest through the RPN layer to obtain a feature map with the same size as the image.
[0075] The implementation process of performing multiple convolutions on the primary features includes:
[0076] Extract and connect features using the first 5×5 convolutional kernel, extract and connect features using the second 3×3 convolutional kernel, and extract and connect features using the third 3×3 convolutional kernel;
[0077] Obtain the connected primary features; continue to extract and connect features using the first 5×5 convolutional kernel on the basis of the primary features, extract and connect features using the second 3×3 convolutional kernel, and extract and connect features using the third 3×3 convolutional kernel to obtain the connected features; the size of the primary features is 256×56;
[0078] The process of screening the primary features through the RPN layer to obtain the feature map includes: obtaining the coordinates of each pixel of the feature as the input of the RPN layer;
[0079] Set the threshold of the sliding window, set the threshold in multiple intervals, calculate the arithmetic mean value of the elements in each sliding window of the feature map; select the sliding window with a high arithmetic mean value as the region of interest, output the corresponding coordinates, and form a feature map with the same size as the input image.
[0080] Set the threshold in multiple intervals, select some elements in the sliding window in each interval, calculate the arithmetic mean value of the remaining elements in the current interval, and use the pixel corresponding to the element with the largest value among the elements in the threshold interval for some elements; screen according to the arithmetic mean value to obtain a feature map with the same size as the input image in the feature map;
[0081] The RPN layer includes 12 convolutional kernels;
[0082] Input the feature map into the Faster R-CNN layer to obtain the initial offset at each position, and use the initial offset as the position label;
[0083] Obtain the initial candidate box offset, and the initial candidate box offset is used to represent the center coordinates of the defect feature area in the initial candidate box; adjust the size of the initial candidate box;
[0084] Use a soft-max classifier to classify the initial candidate bounding boxes into foreground and background, where the foreground is the area with defects and the background is the area without defects;
[0085] Use a regressor to identify the defect types of the foreground. The defect types include oil stains, black spots, and bright spots, and obtain a regression label, which contains the position of the initially to-be-identified target defect in the initial candidate bounding box;
[0086] In the Faster R-CNN layer, extract defect features from the feature map.
[0087] A weighing device is applied to an intelligent vision detection method as described above.
[0088] In this embodiment, Gaussian filtering, Sobel operator, Canny operator, non-maximum suppression method, Faster R-CNN method, soft-max classifier, etc. are mentioned, and the relevant algorithm technologies of some existing designs can be referred to.
[0089] This embodiment provides an intelligent vision detection and weighing device and method, including collecting data of a transparent part to form a data set, dividing the data set and validating it through a trained network model to obtain an accurate target image, processing the image and using the Faster R-CNN method to identify and mark the defect types in the image; by further processing the initial data of the image, an accurate target image is obtained to improve the accuracy of defect recognition.
[0090] According to the disclosure and teaching of the above specification, those skilled in the art to which the present invention pertains can also make changes and modifications to the above embodiments. Therefore, the present invention is not limited to the specific embodiments disclosed and described above, and some modifications and changes to the present invention should also fall within the protection scope of the claims of the present invention. In addition, although some specific terms are used in this specification, these terms are only for convenience of description and do not constitute any limitation to the present invention.
Claims
1. An intelligent visual detection method, characterized in that: The following steps are involved: S1, collecting images of transparent parts, the pixel values of each transparent part image are the same, and obtaining a data set of all transparent part images; S2, dividing the data set of transparent part images and verifying it on the trained network model to obtain the target image; S3, graying the target image, filtering the grayed image using a Gaussian filter function, obtaining the gradient value and direction according to the obtained image, and then finding the maximum value of the local area of the pixel by comparing the gradient values before and after the gradient direction, selecting double threshold detection and connecting edges, obtaining an edge binary map, and obtaining the defect area according to the edge binary map; S4. Identify defective areas. Based on the Faster R-CNN method, identify and mark the types of defects in the image. Extract primary features in the sparse filter layer as the input layer of the convolution part. Preliminarily extract the region of interest in the RPN layer, and then perform position marking and defect type identification.
2. An intelligent visual detection method according to claim 1, characterized in that: In S1, the specific method of collecting the image of the transparent part includes: A light source is set on the back of the transparent part, and the light shines on the transparent part and is projected into the camera. The camera records the image of the transparent part and transmits it to a computer for storage, forming a data set of the transparent part image. The camera collects the image of the transparent part in the following ways: rotation, longitudinal movement, horizontal movement, scaling, horizontal flipping, and vertical flipping. The image data sets of the transparent part at different angles are obtained to form a data set of the transparent part image.
3. An intelligent visual detection method according to claim 2, characterized in that: In S2, the transparent image data set S is divided into X non-overlapping data subsets, and the training sample is set to P. The number of samples of the data subset obtained according to the training sample and the data subset is P / X. Each data subset is S = {S1, S2...S x }, extract any data subset, use the remaining data subset as a training network model, verify the extracted data subset on the training network model, and obtain the verification result of the extracted data subset; All data subsets are verified on the corresponding training network model to obtain the verification results of all data subsets, which is the training set. The accuracy of the training set is obtained based on the verification results, and the verification result closest to the accuracy is selected, and its corresponding data subset is the target image.
4. The intelligent visual detection method according to claim 1, characterized in that: The specific method of obtaining the gradient value and direction in S3 is to calculate the gradient value of each pixel using the Sobel function based on the X and Y directions, and then obtain the direction of the gradient value of each pixel; that is, Gx and Gy, Gx represents the horizontal gradient component of the pixel point, and Gy represents the vertical gradient component of the pixel point; The non-maximum suppression method is used to detect edges: the gradient value of a pixel is compared with the gradient values of horizontally and vertically adjacent pixels, and non-maximum values are suppressed; The dual threshold is selected by determining the optimal values of the dual thresholds TH1 and TH2 through the grayscale gradient value histogram. The specific method is to take the midpoint of the rising branch of the histogram as the strong edge threshold TH1; starting from the strong edge threshold TH1, take the midpoint of the descending branch as the weak edge threshold TH2; According to the determined double thresholds TH1 and TH2, and according to the response of the neighboring pixels, it is determined whether the pixel is an edge or a non-edge; If the gradient value of a pixel in the image is greater than the strong edge threshold TH1, the pixel is considered a strong edge point; if the gradient value of a pixel is less than the weak edge threshold TH2, the pixel is considered a non-edge point; if the gradient value of a pixel is less than TH1 and greater than TH2, its neighborhood is determined.
5. The intelligent visual detection method according to claim 4, characterized in that: If there is a weak edge in its neighborhood, the pixel is ultimately considered a weak edge point; if there is a non-edge in its neighborhood, the pixel is ultimately considered a non-edge point; each pixel is judged in turn to obtain an edge binary map.
6. The intelligent visual detection method according to claim 5, characterized in that: The method for obtaining the defect area according to the edge binary image is: The edge binary image and the 16-pixel butterfly structure element are expanded using the dilation algorithm so that the thin edge does not disappear after filling and a thick edge is formed; the connected domains of each image are counted and the contour length is calculated, and the connected domain contour with the longest contour length is selected as the defect area.
7. An intelligent visual detection method according to claim 6, characterized in that: In the edge binary image, each structural element is overlapped with the image according to the horizontal and vertical offsets of the structural element, and it is determined whether there are edge points in the structural element. If so, the edge points in the structural element are filled, and finally all the filled points are formed into a new connected domain.
8. The intelligent visual detection method according to claim 1, characterized in that: In S4, based on the Faster R-CNN method, the defect types in the image are identified and marked. The defect detection model includes a feature extraction layer, an RPN layer, and a Faster R-CNN layer. The sparse filter layer in the feature extraction layer extracts primary features as the input layer of the convolution part, and the Faster R-CNN layer performs position marking and defect type identification, specifically including: Use sparse filtering to extract primary features. Obtain the grayscale value range of the defect area and the background area through the image histogram. In the mask corresponding to the image, the grayscale value of the defect area is 1, and the grayscale value of the background area is 0. The product of all pixels in the image and the mask constitutes the window of the sparse filtering function. The primary features of the sample image are obtained through convolution. The primary features are convolved multiple times, the convolved feature maps are connected and passed through the RPN layer to filter the region of interest to obtain a feature map of the same size as the image.
9. An intelligent visual detection method according to claim 8, characterized in that: The primary features are filtered through the RPN layer, and the process of obtaining the feature map includes: obtaining the coordinates of each pixel of the feature as the input of the RPN layer; Set the threshold of the sliding window, set the threshold in multiple intervals, calculate the arithmetic mean of the elements in each sliding window in the feature map; select the sliding window with a high arithmetic mean as the region of interest, output the corresponding coordinates, and form a feature map with the same size as the input image.
10. A weighing device, characterized in that: An intelligent visual detection method applied to any one of claims 1 to 9.
Citation Information
Cited By
Electric power line image defect area segmentation method, system, equipment and medium
CN120451189A