Cabinet body surface rust detection method and system based on image recognition
By installing an image camera on the cabinet detection station, image recognition and pre-processing, and combining with the rust recognition model, automatic detection of rust on the cabinet surface is realized, solving the problems of high error detection rate and high cost of traditional manual detection.
Patent Information
- Application Number
- CN202411991135.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-31
- Publication Date
- 2025-04-29
AI Technical Summary
In the prior art, the detection of rust on the surface of the cabinet mainly relies on manual observation, and there are problems such as high error detection rate, missed detection rate and high labor cost.
Using an image recognition-based detection method, by installing an image camera on the detection station, images of the corresponding surfaces of the cabinet are taken, image preprocessing, reflectivity detection and elimination are performed, and finally rust recognition is used to use the pre-trained rust recognition model.
Automatic detection of rust on the surface of the cabinet is realized, labor costs are reduced, and inspection accuracy and efficiency are improved.
Smart Images
Figure CN119904440A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer vision technology, and in particular to a cabinet surface rust detection method and system based on image recognition. Background Art
[0002] Cabinets are common facilities in production and life. There are many types and uses of cabinets, such as storage cabinets, vending cabinets, control cabinets, insulation cabinets, etc.
[0003] In the process of cabinet production and quality inspection, the detection of surface rust is a very important link. Only products with qualified quality, no rust on the surface and good appearance can meet the quality inspection requirements. At present, some manufacturers mainly rely on traditional manual inspection methods for cabinet surface rust detection, that is, workers observe each side of the cabinet with their naked eyes. This quality inspection method of manual naked eye observation has obvious defects: first, it increases labor costs; second, it is more dependent on the workers' sense of responsibility; third, it is prone to high false detection rate and missed detection rate. The above factors affect the efficiency and accuracy of rust detection and need to be improved. Summary of the invention
[0004] Based on the above description, the present invention provides a cabinet surface rust detection method based on image recognition, which can realize automatic detection of cabinet surface rust instead of manual detection.
[0005] The technical solution of the present invention to solve the above technical problems is as follows:
[0006] A method for detecting rust on the surface of a cabinet based on image recognition is as follows: a detection station for placing the cabinet is set, and two sets of camera devices are installed on each surface of the detection station; after the cabinet is placed at the detection station, the two sets of camera devices respectively shoot the corresponding surface of the cabinet, i.e., the target surface, from two fixed positions, so as to obtain two sets of images of the target surface; the cabinet surface area image is identified and segmented and extracted from the two sets of images, and the two sets of cabinet surface area images are corrected into rectangles of the same length and width, i.e., a first image and a second image are obtained; the first image and the second image are divided into the same grid array, and each grid in the first image and the second image is respectively detected. The reflectivity of the counter surface area is detected and marked; the reflectivity parameters of each grid in the first image and the second image are traversed, the reflectivity parameters of the grids corresponding to each group of positions in the first image and the second image are compared, and grids with lower reflectivity values are selected as sample grids, all sample grids are spliced into an overall image and the overall image is merged with the standard image of the counter surface to obtain a third image; the corresponding grid areas in the third image are de-reflected according to the reflectivity parameters of each sample grid to obtain a fourth image; rust is identified in the fourth image, and an abnormal alarm message is issued if rust is identified.
[0007] As a preferred solution: when the overall image is fused with the standard image, the weight of the sample grid and the weight of the standard image during fusion are determined according to the reflectivity parameters of each sample grid, and the higher the reflectivity of the sample grid, the smaller its weight.
[0008] As a preferred solution: build a rust recognition model, collect a large number of image samples, train the rust recognition model, and use the rust recognition model to identify rust.
[0009] As a preferred solution: the rust recognition model includes classification model A, classification model B and classification model C; wherein classification model A encodes the fourth image into a feature vector f, calculates the similarity between the feature vector f and the feature vectors {f1, f2, f3....fm} of images with different rust degrees in the database, finds the image with the closest similarity, and the category of the most similar image is the category of the fourth image; classification model B uses the CNN deep learning model to directly classify the fourth image and simulates the probability of rust output; classification model C uses the U-Net semantic segmentation model to perform pixel segmentation on the fourth image The rust area is separated from the background by level segmentation, and an accurate rust area map is generated while the probability of the existence of the rust area is output; finally, the formula Score = Wa*Pa+Wb*Pb+Wc*Pc is used, where Score is the rust confidence score of the fourth image, Pa is the highest similarity of the fourth image, Pb is the probability of rust in the fourth image, Pc is the probability of the existence of the rust area, Wa, Wb, and Wc are respectively the preset weight coefficients, Wa+Wb+Wc=1; finally, the confidence score is compared with the preset score, and when the former is greater than the latter, it is determined that the target counter surface is rusted.
[0010] As a preferred solution: when detecting the reflectivity, the color image is first converted into a grayscale image, followed by noise reduction processing on the grayscale image, and then a brightness threshold is set in the image after noise reduction processing, the brightness of the image of each grid area is obtained and the brightness threshold is compared, and the reflectivity of each grid area is determined based on the comparison result.
[0011] As a preferred solution: when detecting reflection, the gradient of the image is calculated, and the change of the gradient value of each grid area and the reflectivity of each grid are combined to determine whether each grid has reflection.
[0012] The cabinet surface rust detection system based on image recognition comprises a first camera device, a second camera device, an image acquisition unit, an image storage unit, an image processing unit, a reflection detection unit, a reflection elimination unit, a rust detection unit and an abnormal alarm unit; wherein the first camera device and the second camera device are installed on one side of the cabinet detection station, and the two groups of camera devices respectively shoot the target surface of the cabinet from two fixed positions, so as to obtain two groups of images of the target surface; the image processing unit recognizes and extracts the cabinet surface area image from the two groups of images by segmentation, and corrects the two groups of cabinet surface area images into rectangles of the same length and width, that is, obtains the first image and the second image; the reflection detection unit divides the first image and the second image into the same grid array, and the reflection detection unit respectively detects the first image and the second image; The reflectivity of the counter surface area in each grid in the first image and the second image is detected and marked; the image processing unit traverses the reflectivity parameters of each grid in the first image and the second image, compares the reflectivity parameters of the grids corresponding to each group of positions in the first image and the second image, and selects the grids with lower reflectivity values as sample grids. The image processing unit splices all the sample grids into an overall image and merges the overall image with the standard image of the counter surface to obtain a third image; the reflection elimination unit eliminates the reflection of each corresponding grid area in the third image according to the reflectivity parameters of each sample grid to obtain a fourth image; the rust detection unit identifies rust on the fourth image, and if rust is identified, the abnormal alarm unit issues an abnormal alarm message.
[0013] As a preferred solution: the rust detection unit is built with a pre-trained rust recognition model, and the rust recognition model includes a classification model A, a classification model B and a classification model C; the classification model A encodes the fourth image into a feature vector f, calculates the similarity between the feature vector f and the feature vectors of images with different rust degrees in the database, finds the image with the closest similarity, and the category of the most similar image is the category of the fourth image; the classification model B uses the CNN deep learning model to directly classify the fourth image and simulates the output of the probability of rust; the classification model C uses the U-Net semantic segmentation model to perform pixel-level segmentation on the fourth image, separates the rust area from the background, generates an accurate rust area map and outputs the probability of the existence of the rust area; finally, the calculation formula is used to calculate the probability of the existence of the rust area.
[0014] Score=Wa*Pa+Wb*Pb+Wc*Pc, where Score is the rust confidence score of the fourth image, Pa is the highest similarity of the fourth image, Pb is the probability of rust in the fourth image, Pc is the probability of the existence of a rusty area, Wa, Wb, and Wc are preset weight coefficients respectively, Wa+Wb+Wc=1; finally, the confidence score is compared with the preset score, and when the former is greater than the latter, it is judged that the target counter surface is rusted.
[0015] Compared with the prior art, the technical solution of the present application has the following beneficial technical effects: the present application uses a multi-angle camera to capture images on each opposite side, pre-processes the images, and then uses reflection detection technology and reflection elimination technology to process them to obtain images with low reflectivity and retaining a large number of image features and details, and then inputs the image into a pre-trained rust recognition model, and uses the recognition model to automatically identify whether the cabinet surface is rusted, thereby realizing automatic detection of the cabinet body, which can replace the traditional manual detection method, not only reducing labor costs, but also ensuring the accuracy and efficiency of detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] Figure 1 It is a schematic diagram of the arrangement of the camera device in the first embodiment;
[0017] Figure 2 This is a schematic diagram of image processing in Example 1;
[0018] Figure 3 This is a schematic diagram of the system in Example 2.
[0019] In the accompanying drawings, the components represented by the reference numerals are listed as follows:
[0020] 1. Cabinet; 2. First camera device; 3. Second camera device; 4. Inspection station. DETAILED DESCRIPTION
[0021] Embodiment 1:
[0022] A method for detecting rust on the surface of a cabinet 1 based on image recognition:
[0023] like Figure 1 As shown, a detection station 4 for placing a cabinet is set, and the cabinet to be detected is placed on the detection station 4. When the cabinet is placed, each surface of the cabinet faces a predetermined direction. Two sets of camera devices are installed on each surface of the detection station 4; after the cabinet is placed in the detection station 4, the two sets of camera devices respectively shoot the corresponding surface of the cabinet, i.e., the target surface, from two fixed positions, so that two sets of images of the target surface can be obtained; the cabinet surface area image is identified and segmented and extracted from the two sets of images, and the two sets of cabinet surface area images are corrected into rectangles of the same length and width, so as to obtain the first image and the second image.
[0024] like Figure 2 As shown, the first image and the second image are divided into the same grid array, Figure 2 (I) is the first image, Figure 2 (II) is the second image.
[0025] The reflectivity of the counter surface area in each grid in the first image and the second image is detected and marked respectively.
[0026] The process of detecting reflectivity is as follows: first, the color image is converted into a grayscale image, because the grayscale image can effectively reflect the brightness difference. Then, the grayscale image is subjected to noise reduction processing, and then a brightness threshold is set in the image after noise reduction processing. In the grayscale image, the brightness of the reflective area will be much higher than the background area, so by setting a brightness threshold, the reflective area and the non-reflective area can be separated. The brightness of the image of each grid area is obtained and the brightness threshold is compared, and the reflectivity of each grid area is determined according to the comparison result. In this embodiment, the definition of reflectivity is: the result obtained by dividing the image brightness of the grid area by the brightness threshold is the reflectivity. If the image brightness of the grid area is less than the brightness threshold, the reflectivity of the grid area is less than 1, and if the image brightness of the grid area is greater than the brightness threshold, the reflectivity of the grid area is greater than 1.
[0027] Considering that judging the reflectivity by brightness may be affected by some interference factors, thus affecting the accuracy of detection, the present embodiment adopts a method of comprehensive judgment of image gradient and image brightness to detect the reflective area.
[0028] That is, when detecting reflection, the gradient of the image is calculated, and the change of the gradient value of each grid area and the reflectivity of each grid are combined to determine whether each grid has reflection. If the image gradient of the grid area changes significantly and the reflectivity of the grid area is greater than a certain value, it is considered that the grid area has reflection.
[0029] Next, the reflectivity parameters of each grid in the first image and the second image are traversed, the reflectivity parameters of the grids corresponding to each group of positions in the first image and the second image are compared, and the grids with lower reflectivity values are selected as sample grids, and all the sample grids are spliced into the overall image (such as Figure 2 Ⅲ). For example, if the A1 grid in the first image and the B1 grid in the second image are two groups of corresponding grids, and the reflectivity of the A1 grid is less than the reflectivity of the second B1 grid, then the A1 grid is selected as the sample grid; if the A2 grid in the first image and the B2 grid in the second image are two groups of corresponding grids, and the reflectivity of the A2 grid is greater than the reflectivity of the second B2 grid, then the A2 grid is selected as the sample grid. The above measures can be used to pre-select grids with relatively low reflectivity to stitch images, reduce the difficulty and workload of subsequent reflection elimination, and improve efficiency.
[0030] The spliced overall image is fused with the standard image of the counter surface to obtain the third image. This step requires the pre-collection of a non-reflective front view image of the target counter surface, i.e., the standard image. The fusion of the overall image and the standard image is also a block fusion. During the fusion, each grid in the overall image and its corresponding area in the standard image are assigned a fusion weight value. The fusion weight value is related to the reflectivity of the sample grid area. The higher the reflectivity of the sample grid, the smaller its weight. Through the above scheme, the overall reflectivity of the third image can be pre-weakened to a certain extent, while retaining the features and details in the image as much as possible.
[0031] The fourth image is obtained by performing a de-reflection process on each corresponding grid area in the third image according to the reflectivity parameters of each sample grid, that is, the higher the reflectivity of the sample grid, the greater the de-reflection strength of the corresponding grid area (image de-reflection is a prior art and will not be described in detail here).
[0032] The fourth image is subjected to rust recognition, and if rust is recognized, an abnormal alarm message is issued.
[0033] In this embodiment, the method for identifying rust is as follows: building a rust recognition model, collecting a large number of image samples with different degrees of rust, inputting the image samples into the rust recognition model, training the rust recognition model, and then using the trained rust recognition model to identify rust.
[0034] Specifically, the rust recognition model in this embodiment includes classification model A, classification model B and classification model C; wherein classification model A encodes the fourth image into a feature vector f, calculates the similarity between the feature vector f and the feature vectors {f1, f2, f3....fm} of images with different rust degrees in the database, finds the image with the closest similarity, and the category of the most similar image is the category of the fourth image; classification model B uses the CNN deep learning model to directly classify the fourth image, and simulates and outputs the probability of rust; classification model C uses the U-Net semantic segmentation model to perform pixel-level segmentation on the fourth image, separates the rust area from the background, generates an accurate rust area map, and outputs the probability of the existence of the rust area.
[0035] Finally, the formula Score=Wa*Pa+Wb*Pb+Wc*Pc is used, where Score is the rust confidence score of the fourth image, Pa is the highest similarity of the fourth image, Pb is the probability of rust in the fourth image, Pc is the probability of the existence of a rust area, Wa, Wb, and Wc are preset weight coefficients, and Wa+Wb+Wc=1.
[0036] Finally, the confidence score Score is compared with the preset score. When the former is greater than the latter, it is determined that the target counter surface is rusted, and the abnormal alarm unit issues a rust prompt.
[0037] Embodiment 2:
[0038] Reference Figure 3 A cabinet 1 surface rust detection system based on image recognition comprises a first camera device 2, a second camera device 3, an image acquisition unit, an image storage unit, an image processing unit, a reflection detection unit, a reflection elimination unit, a rust detection unit and an abnormal alarm unit; wherein the first camera device and the second camera device are installed on one side of a cabinet detection station 4, and the two camera devices respectively shoot the target surface of the cabinet from two fixed positions, so as to obtain two groups of images of the target surface; the image processing unit recognizes and extracts the cabinet surface area image from the two groups of images by segmentation, and corrects the two groups of cabinet surface area images into rectangles of the same length and width, that is, obtaining the first image and the second image; the reflection detection unit divides the first image and the second image into the same grid array, and the reflection detection unit respectively The reflectivity of the counter surface area in each grid in the first image and the second image is detected and marked; the image processing unit traverses the reflectivity parameters of each grid in the first image and the second image, compares the reflectivity parameters of the grids corresponding to each group of positions in the first image and the second image, and selects the grids with lower reflectivity values as sample grids, the image processing unit splices all the sample grids into an overall image and merges the overall image with the standard image of the counter surface to obtain a third image; the reflection elimination unit eliminates the reflection of each corresponding grid area in the third image according to the reflectivity parameters of each sample grid to obtain a fourth image; the rust detection unit identifies rust on the fourth image, and if rust is identified, the abnormal alarm unit issues an abnormal alarm message.
[0039] In this embodiment: the rust detection unit is built with a pre-trained rust recognition model, and the rust recognition model includes a classification model A, a classification model B and a classification model C; the classification model A encodes the fourth image into a feature vector f, calculates the similarity between the feature vector f and the feature vectors of images with different rust degrees in the database, finds the image with the closest similarity, and the category of the most similar image is the category of the fourth image; the classification model B uses the CNN deep learning model to directly classify the fourth image, and simulates and outputs the probability of rust; the classification model C uses the U-Net semantic segmentation model to perform pixel-level segmentation on the fourth image, separates the rust area from the background, generates an accurate rust area map, and outputs the probability of the existence of the rust area; finally, the calculation formula Score=Wa*Pa+Wb*Pb+Wc*Pc is used, where Score is the rust confidence score of the fourth image, Pa is the highest similarity of the fourth image, Pb is the probability of the fourth image being rusted, Pc is the probability of the existence of the rust area, Wa, Wb, and Wc are respectively preset weight coefficients, and Wa+Wb+Wc=1.
[0040] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A cabinet surface rust detection method based on image recognition, characterized by: A detection station for placing a cabinet is set, and two sets of camera devices are installed on each surface of the detection station; after the cabinet is placed at the detection station, the two sets of camera devices respectively shoot the corresponding surface of the cabinet, i.e., the target surface, from two fixed positions, so as to obtain two sets of images of the target surface; the cabinet surface area image is identified and segmented and extracted from the two sets of images, and the two sets of cabinet surface area images are corrected into rectangles of the same length and width, i.e., a first image and a second image are obtained; the first image and the second image are divided into the same grid array, and the reflectivity of the cabinet surface area in each grid in the first image and the second image is respectively detected and marked; Traverse the reflectivity parameters of each grid in the first image and the second image, compare the reflectivity parameters of the grids corresponding to each group of positions in the first image and the second image, and select grids with lower reflectivity values as sample grids, splice all the sample grids into an overall image, and merge the overall image with the standard image of the counter surface to obtain a third image; according to the reflectivity parameters of each sample grid, eliminate the reflection of each corresponding grid area in the third image to obtain a fourth image; perform rust recognition on the fourth image, and issue an abnormal alarm message if rust is recognized.
2. The cabinet surface rust detection method based on image recognition according to claim 1 is characterized in that: When the overall image is fused with the standard image, the weight of the sample grid and the weight of the standard image during fusion are determined according to the reflectivity parameters of each sample grid. The higher the reflectivity of the sample grid, the smaller its weight.
3. The cabinet surface rust detection method based on image recognition according to claim 1 is characterized in that: Build a rust recognition model, collect a large number of image samples, train the rust recognition model, and use the rust recognition model to identify rust.
4. The cabinet surface rust detection method based on image recognition according to claim 3 is characterized in that: The rust recognition model includes classification model A, classification model B and classification model C; wherein classification model A encodes the fourth image into a feature vector f, calculates the similarity between the feature vector f and the feature vectors {f1, f2, f3....fm} of images with different rust degrees in the database, finds the image with the closest similarity, and the category of the most similar image is the category of the fourth image; classification model B uses the CNN deep learning model to directly classify the fourth image and simulates the probability of rust output; classification model C uses the U-Net semantic segmentation model to perform pixel-level segmentation on the fourth image , separate the rust area from the background, generate an accurate rust area map and output the probability of the existence of the rust area; finally, use the formula Score = Wa*Pa+Wb*Pb+Wc*Pc, where Score is the rust confidence score of the fourth image, Pa is the highest similarity of the fourth image, Pb is the probability of rust in the fourth image, Pc is the probability of the existence of the rust area, Wa, Wb, Wc are the preset weight coefficients, Wa+Wb+Wc=1; finally, compare the confidence score with the preset score, and when the former is greater than the latter, it is judged that the target counter surface is rusted.
5. The cabinet surface rust detection method based on image recognition according to claim 1 is characterized in that: When detecting the reflectivity, the color image is first converted into a grayscale image, followed by noise reduction processing on the grayscale image. Then, a brightness threshold is set in the image after noise reduction processing, the brightness of the image of each grid area is obtained and the brightness threshold is compared, and the reflectivity of each grid area is determined according to the comparison result.
6. The cabinet surface rust detection method based on image recognition according to claim 2 is characterized in that: When detecting reflections, the gradient of the image is calculated, and the change in the gradient value of each grid area and the reflectivity of each grid are combined to determine whether each grid has reflections.
7. The cabinet surface rust detection system based on image recognition is characterized by: The system comprises a first camera device, a second camera device, an image acquisition unit, an image storage unit, an image processing unit, a reflection detection unit, a reflection elimination unit, a rust detection unit and an abnormal alarm unit; wherein the first camera device and the second camera device are installed on one side of the cabinet detection station, and the two groups of camera devices respectively shoot the target surface of the cabinet from two fixed positions, so as to obtain two groups of images of the target surface; the image processing unit identifies and extracts the cabinet surface area image from the two groups of images by segmentation, and corrects the two groups of cabinet surface area images into rectangles of the same length and width, that is, obtains the first image and the second image; the reflection detection unit divides the first image and the second image into the same grid array, and the reflection detection unit respectively detects and marks the reflectivity of the cabinet surface area in each grid in the first image and the second image; The image processing unit traverses the reflectivity parameters of each grid in the first image and the second image, compares the reflectivity parameters of the grids corresponding to each group of positions in the first image and the second image, and selects grids with lower reflectivity values as sample grids. The image processing unit splices all the sample grids into an overall image and merges the overall image with the standard image of the cabinet surface to obtain a third image; the reflection elimination unit eliminates the reflection of each corresponding grid area in the third image according to the reflectivity parameters of each sample grid to obtain a fourth image; the rust detection unit identifies rust on the fourth image, and if rust is identified, the abnormal alarm unit issues an abnormal alarm message.
8. The cabinet surface rust detection system based on image recognition according to claim 7 is characterized in that: The rust detection unit is equipped with a pre-trained rust recognition model, which includes a classification model A, a classification model B and a classification model C; the classification model A encodes the fourth image into a feature vector f, calculates the similarity between the feature vector f and the feature vectors of images with different rust degrees in the database, finds the image with the closest similarity, and the category of the most similar image is the category of the fourth image; the classification model B uses the CNN deep learning model to directly classify the fourth image and simulates the probability of rust output; the classification model C uses the U-Net semantic segmentation model to perform pixel segmentation on the fourth image The rust area is separated from the background by level segmentation, and an accurate rust area map is generated while the probability of the existence of the rust area is outputted. Finally, the calculation formula Score=Wa*Pa+Wb*Pb+Wc*Pc is used, where Score is the rust confidence score of the fourth image, Pa is the highest similarity of the fourth image, Pb is the probability of rust in the fourth image, Pc is the probability of the existence of the rust area, Wa, Wb, and Wc are respectively the preset weight coefficients, Wa+Wb+Wc=1. Finally, the confidence score is compared with the preset score. When the former is greater than the latter, it is determined that the target counter surface is rusted.