Wallboard defect intelligent detection system and method based on image recognition

Through an intelligent detection system based on image recognition, the convolutional neural network model and feature extraction technology are used to solve the problems of low efficiency and low accuracy of traditional wall panel defect detection, and the rapid and accurate detection of multiple defects is achieved, improving the efficiency and accuracy of various defects. Test efficiency and accuracy.

CN119067960AActive Publication Date: 2024-12-03GUANGDONG SOBEN GREEN NEW MATERIAL CO LTD
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202411260794.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-10
Publication Date
2024-12-03
Estimated Expiration
2044-09-10

AI Technical Summary

Technical Problem

Traditional wall panel defect detection relies on manual visual inspection, which is inefficient and has low accuracy, and cannot detect multiple types of defects at the same time.

Method used

Using an intelligent detection system based on image recognition, wall panel images are acquired through the image acquisition module and pre-processed. The pre-trained convolutional neural network model is used to detect the damage and color difference defects of wall panels, combined with feature extraction networks, regional suggestions networks RPN and The subpixel grayscale matrix is ​​detected, and the results are fused and the output module are analyzed for the final results.

Benefits of technology

It realizes rapid and accurate detection of wall panel defects, improves detection efficiency and accuracy, reduces missed and missed inspections, and can identify multiple types of defects at the same time, such as holes, cracks, blocks and color aberrations, ensuring wall panels High standards of product quality.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119067960A_ABST
    Figure CN119067960A_ABST
Patent Text Reader

Abstract

The invention discloses an intelligent wallboard defect detection system and method based on image recognition, and relates to the technical field of image processing, and the system comprises an image acquisition module, an image detection module and a result fusion and output module. Data quality is ensured through the image acquisition and preprocessing module, two convolutional neural network models are utilized to quickly and accurately detect damage and chromatic aberration defects respectively, and then comprehensive analysis and false detection elimination are further performed through an intelligent algorithm through the result fusion and output module, so that comprehensiveness and accuracy of a detection result are ensured, and the accuracy of the detection result is improved. According to the method, the detection efficiency and accuracy are improved, the labor cost is reduced, the method is suitable for detecting various types of defects, and the quality control standard of wallboard products is remarkably improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an intelligent wallboard defect detection system and method based on image recognition. Background Art

[0002] As people's living standards improve, more and more people are pursuing high-quality life and have higher and higher requirements for home environment. When decorating their houses, they will buy some high-end decorative materials to enhance the overall texture of the room. For example, wall panels have good sound insulation effect, which can reduce noise pollution, and their thermal insulation performance is also very good, which can reduce heat loss. In addition, wall panels have good fire retardant function and higher safety factor. However, in the actual production process, due to process problems or human factors, various defects often occur, which affect the aesthetics of the product and reduce the product quality, so quality control is needed.

[0003] At present, traditional wallboard defect detection mainly relies on manual visual inspection, but this inspection method is inefficient, inaccurate, and prone to missed detection. In order to improve this situation, some researchers have tried to use computer vision technology to detect wallboard defects in recent years. The advantages of this method are fast speed and high accuracy, but it can only detect specific types of products and cannot detect multiple types of defects at the same time. Summary of the invention

[0004] The present invention provides a wallboard defect intelligent detection system and method based on image recognition to solve the above problems existing in the prior art.

[0005] According to one aspect of the present invention, there is provided a wallboard defect intelligent detection system based on image recognition, comprising an image acquisition module, an image detection module and a result fusion and output module.

[0006] The image acquisition module is used to obtain the wallboard image to be detected, preprocess the wallboard image data to be detected, and obtain the preprocessed image data. The image preprocessing operation includes image graying, filtering and size standardization.

[0007] The image detection module detects the damage and color difference defects on the wallboard surface through the pre-trained first convolutional neural network model and the second convolutional neural network model; the image detection module includes a damage detection unit and a color difference defect detection unit,

[0008] The damage detection unit: performs wallboard damage detection on the preprocessed image data through a pre-trained first convolutional neural network model to obtain a first detection result;

[0009] The color difference defect detection unit: performs wallboard color difference defect detection on the preprocessed image data through a pre-trained second convolutional neural network model to obtain a second detection result;

[0010] The result fusion and output module is used to fuse the first detection result and the second detection result, and output the final detection result.

[0011] Optionally, the image acquisition module is further used to set the position and angle of the camera, placing the camera at a predetermined height from the wall panel to be measured; adjusting the angle of the camera so that the center of the camera lens forms a predetermined angle with the horizontal direction of the wall panel to be measured.

[0012] Optionally, the damage detection unit includes: using the first convolutional neural network model to perform forward propagation on the preprocessed image data to obtain a first prediction box and a first confidence score; if the first confidence score is greater than or equal to a predetermined threshold, determining the first prediction box as a first detection result;

[0013] The color difference defect detection unit includes: using the second convolutional neural network model to forward propagate the preprocessed image data to obtain a second prediction box and a second confidence score; if the second confidence score is greater than or equal to a predetermined threshold, determining the second prediction box as a second detection result.

[0014] Optionally, the preprocessed image data is forward propagated using the first convolutional neural network model, which also includes a feature extraction network, a region proposal network RPN, and a ROI pooling layer.

[0015] Wherein, the feature extraction network is used to extract a set of feature maps from the preprocessed image data;

[0016] The region proposal network RPN is used to predict multiple region proposal boxes according to the feature map;

[0017] The ROI pooling layer is used to compress the information in the region proposal box into a feature map of a fixed size after performing convolution operations on each region proposal box respectively.

[0018] Optionally, forward propagation of the preprocessed image data using the second convolutional neural network model also includes sub-pixel grayscale matrix construction, subordinate main pixel determination, grayscale weight calculation, sub-pixel grayscale value determination, and sub-pixel image construction.

[0019] The sub-pixel grayscale matrix is ​​constructed to convert the grayscale image of the pre-processed image data into a sub-pixel grayscale matrix, and the sub-pixel grayscale matrix includes main pixel points and sub-pixel points;

[0020] The subordinate main pixel point determination is used to calculate the grayscale difference and distance difference between the sub-pixel point and the candidate main pixel point, and determine the candidate main pixel point with the highest degree of membership corresponding to the sub-pixel point according to a preset threshold, that is, the main pixel point to which the sub-pixel point belongs;

[0021] The grayscale weight calculation is used to calculate the grayscale weight of the sub-pixel point according to the grayscale value and position of the main pixel point to which the sub-pixel point belongs;

[0022] The sub-pixel grayscale value determination is used to determine the grayscale value of the sub-pixel according to the grayscale value of the main pixel to which the sub-pixel belongs, the grayscale weight of the sub-pixel and the grayscale value of the main pixel;

[0023] The sub-pixel image construction is used to construct a sub-pixel image according to the grayscale value of the sub-pixel point.

[0024] Optionally, the subordinate main pixel point determination is specifically used to calculate the grayscale difference and the distance difference between the sub-pixel point and the candidate main pixel point, and determine the main pixel point to which the sub-pixel point belongs according to a preset threshold, including:

[0025] If |GG″|≤ε1 and d≤ε2, the sub-pixel point belongs to the candidate main pixel point, wherein G represents the grayscale value of the sub-pixel point, G″ represents the grayscale value of the candidate main pixel point, d represents the distance between the sub-pixel point and the candidate main pixel point, ε1 represents the grayscale difference threshold, and ε2 represents the distance difference threshold.

[0026] Optionally, the grayscale weight calculation is specifically used to calculate the grayscale weight of the sub-pixel point according to the grayscale value and position of the main pixel point to which the sub-pixel point belongs, including:

[0027] If the sub-pixel point belongs to the main pixel point in the i-th row and j-th column, the grayscale weight of the sub-pixel point is w=(i-α)·(β-j), wherein α represents the ratio of the number of rows where the main pixel point to which the sub-pixel point belongs to the total number of rows, and β represents the ratio of the number of columns where the main pixel point to which the sub-pixel point belongs to the total number of columns.

[0028] Optionally, the sub-pixel grayscale value determination is specifically used to determine the grayscale value of the sub-pixel according to the grayscale value of the main pixel to which the sub-pixel belongs, the grayscale weight of the sub-pixel and the grayscale value of the main pixel, including:

[0029] If the sub-pixel point belongs to the main pixel point in the i-th row and j-th column, the grayscale value of the sub-pixel point is g=w*gi+(1-w)*gj, wherein g represents the grayscale value of the sub-pixel point, w represents the grayscale weight of the sub-pixel point, gi represents the grayscale value of the main pixel point in the i-th row and j-th column, and gj represents the grayscale value of the main pixel point in the j-th column.

[0030] Optionally, the result fusion and output module is used to fuse the first detection result and the second detection result and output the final detection result, including:

[0031] Obtaining location information of a first type of defect in the wallboard image according to the first detection result; obtaining location information of a second type of defect according to the second detection result; the first type of defect is a hole, a crack or a missing block, and the second type of defect is a color difference defect;

[0032] Among them, when the position area of ​​the first detection result and the second detection result overlap, the overlapping area is analyzed again. When the overlapping result is a false detection due to color difference, the position of the marked area is damaged; when the first detection result and the second detection result have no position area overlap, it is determined that both results are valid defect detection results, and the results are recorded separately to obtain the final detection result.

[0033] According to another aspect of the present invention, there is provided a wallboard defect intelligent detection method based on image recognition, comprising the following steps:

[0034] Acquire the wall panel image to be inspected through the camera;

[0035] Preprocessing the wallboard image data to be detected to obtain preprocessed image data;

[0036] Performing wall panel damage detection on the preprocessed image data by using a pre-trained first convolutional neural network model to obtain a first detection result;

[0037] Performing wallboard color difference defect detection on the preprocessed image data through a pre-trained second convolutional neural network model to obtain a second detection result;

[0038] Fusion of the first detection result and the second detection result, outputting a final detection result;

[0039] wherein the preprocessed image data is forward propagated using the first convolutional neural network model to obtain a first prediction box and a first confidence score; if the first confidence score is greater than or equal to a predetermined threshold, the first prediction box is determined as a first detection result;

[0040] Performing forward propagation on the preprocessed image data using the second convolutional neural network model to obtain a second prediction box and a second confidence score; if the second confidence score is greater than or equal to a predetermined threshold, determining the second prediction box as a second detection result;

[0041] The fusing the first detection result and the second detection result to output a final detection result includes:

[0042] Obtaining location information of a first type of defect in the wallboard image according to the first detection result; obtaining location information of a second type of defect according to the second detection result; the first type of defect is a hole, a crack or a missing block, and the second type of defect is a color difference defect;

[0043] Among them, when the position area of ​​the first detection result and the second detection result overlap, the overlapping area is analyzed again. When the overlapping result is a false detection due to color difference, the position of the marked area is damaged; when the first detection result and the second detection result have no position area overlap, it is determined that both results are valid defect detection results, and the results are recorded separately to obtain the final detection result.

[0044] The beneficial effects of the present invention are:

[0045] The automated image acquisition module efficiently acquires wallboard images and performs necessary preprocessing, including grayscale, filtering, and size standardization. The preprocessing operation not only ensures the quality of image data, but also lays a solid foundation for subsequent defect detection, reduces manual operations, and significantly improves the efficiency of the detection process. At the same time, the pre-trained convolutional neural network model is used to achieve rapid detection of wallboard image damage and color difference defects.

[0046] The present invention uses two specialized convolutional neural network models to accurately identify different types of defects. The damage detection model uses feature extraction network, region proposal network RPN and ROI pooling layer technology to optimize the location process of the damaged area. The color difference defect model realizes sub-pixel level detection of color difference by constructing sub-pixel gray matrix and accurate calculation of gray weight. The fusion of these two models not only improves the accuracy and reliability of detection, but also reduces the possibility of missed detection and false detection.

[0047] Through the result fusion and output module, an intelligent algorithm is used to comprehensively analyze the damage and color difference defect detection results, and the defect areas that may overlap are re-analyzed to eliminate false detections, ensuring the comprehensiveness and accuracy of the detection results. It can not only detect a single type of defect, but also simultaneously identify multiple defects including holes, cracks, missing pieces and color differences, greatly expanding the application scope of detection and ensuring the high quality standards of wall panel products. This comprehensive method realizes the automation, precision and intelligence of wall panel defect detection, effectively improving the detection efficiency and accuracy, while reducing labor costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] For better understanding and implementation, the technical solution of the present application is described in detail below with reference to the accompanying drawings.

[0049] Figure 1 A schematic diagram of the structure of an intelligent wallboard defect detection system based on image recognition provided in Example 1 of the present application;

[0050] Figure 2 A schematic diagram of a process of performing forward propagation of preprocessed image data by using the first convolutional neural network model in a wallboard defect intelligent detection system based on image recognition provided in Embodiment 1 of the present application;

[0051] Figure 3 A schematic diagram of a process of using the second convolutional neural network model to forward propagate the preprocessed image data in a wallboard defect intelligent detection system based on image recognition provided in Embodiment 1 of the present application;

[0052] Figure 4 A schematic flow chart of an intelligent wall panel defect detection method based on image recognition provided in Example 2 of the present application. DETAILED DESCRIPTION

[0053] In order to further explain the technical means and effects taken by the present invention to achieve the predetermined invention purpose, exemplary embodiments will be described in detail here, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are only examples of methods and systems consistent with some aspects of the present application as detailed in the attached claims.

[0054] The terms used in this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The singular forms of "a", "the" and "" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in this article refers to and includes any or all possible combinations of one or more associated listed items.

[0055] The specific implementation methods, features and effects of the present invention are described in detail below in conjunction with the accompanying drawings and preferred embodiments.

[0056] Embodiment 1

[0057] Reference Figure 1 An embodiment of the present invention provides an intelligent wallboard defect detection system based on image recognition, comprising an image acquisition module, an image detection module and a result fusion and output module, wherein:

[0058] Image acquisition module: used to obtain the wallboard image to be detected, preprocess the wallboard image data to be detected, and obtain the preprocessed image data. The image preprocessing operation includes image graying, filtering and size standardization, etc.;

[0059] Image detection module: detects wallboard surface damage and color difference defects through the pre-trained first convolutional neural network model and the second convolutional neural network model; the image detection module includes a damage detection unit and a color difference defect detection unit,

[0060] The damage detection unit: performs wallboard damage detection on the preprocessed image data through a pre-trained first convolutional neural network model to obtain a first detection result;

[0061] The color difference defect detection unit: performs wallboard color difference defect detection on the preprocessed image data through a pre-trained second convolutional neural network model to obtain a second detection result;

[0062] Result fusion and output module: used to fuse the first detection result and the second detection result and output the final detection result.

[0063] The intelligent wall panel defect detection system based on image recognition provided in this embodiment analyzes the image of the wall panel and automatically identifies the defects therein without human intervention, thereby greatly improving the detection efficiency and reducing labor costs. It can also be applied to other types of panel defect detection and has a very wide range of applications.

[0064] Of course, as a preferred embodiment, the image acquisition module is also used to set the position and angle of the camera, placing the camera at a predetermined height from the wall panel to be measured; adjusting the angle of the camera so that the center of the camera lens and the horizontal direction of the wall panel to be measured form a predetermined angle. The advantage of doing this is that it can ensure that the captured pictures are clear and complete, which is conducive to subsequent image processing and analysis.

[0065] Preferably, the damage detection unit comprises: using the first convolutional neural network model to perform forward propagation on the preprocessed image data to obtain a first prediction box and a first confidence score; if the first confidence score is greater than or equal to a predetermined threshold, determining the first prediction box as a first detection result;

[0066] The color difference defect detection unit includes: using the second convolutional neural network model to forward propagate the preprocessed image data to obtain a second prediction box and a second confidence score; if the second confidence score is greater than or equal to a predetermined threshold, determining the second prediction box as a second detection result.

[0067] As a preferred example, Figure 2 As shown, using the first convolutional neural network model to forward propagate the preprocessed image data also includes:

[0068] S11, feature extraction network: used to extract a set of feature maps from the preprocessed image data;

[0069] S12, region proposal network RPN: used to predict multiple region proposal boxes according to the feature map;

[0070] S13, ROI pooling layer: used to compress the information in the region proposal box into a feature map of a fixed size after performing convolution operations on each region proposal box.

[0071] The image recognition-based intelligent wall panel defect detection system provided in this embodiment extracts a set of feature maps from the input image through a feature extraction network, and then predicts multiple region proposal boxes that may have problems through a region proposal network RPN. Finally, the information in these region proposal boxes is compressed into a feature map of a fixed size through a ROI pooling layer, thereby realizing rapid positioning and identification of wall panel defects.

[0072] As a preferred example, Figure 3 As shown, using the second convolutional neural network model to forward propagate the preprocessed image data also includes:

[0073] S21, constructing a sub-pixel grayscale matrix: for converting the grayscale image of the pre-processed image data into a sub-pixel grayscale matrix, wherein the sub-pixel grayscale matrix includes main pixel points and sub-pixel points;

[0074] S22, determining the subordinate main pixel point: for calculating the grayscale difference and distance difference between the sub-pixel point and the candidate main pixel point, and determining the main pixel point to which the sub-pixel point belongs according to a preset threshold;

[0075] S23, grayscale weight calculation: used to calculate the grayscale weight of the sub-pixel point according to the grayscale value and position of the main pixel point to which the sub-pixel point belongs;

[0076] S24, sub-pixel grayscale value determination: used to determine the grayscale value of the sub-pixel according to the grayscale value of the main pixel to which the sub-pixel belongs, the grayscale weight of the sub-pixel and the grayscale value of the main pixel;

[0077] S25, sub-pixel image construction: used to construct a sub-pixel image according to the grayscale value of the sub-pixel point.

[0078] The image recognition-based intelligent wall panel defect detection system provided in this embodiment converts the grayscale image of the preprocessed image data into a sub-pixel grayscale matrix, and then determines the main pixel point to which the sub-pixel point belongs based on the grayscale difference and distance difference between the sub-pixel point and the candidate main pixel point, thereby realizing accurate identification of color difference defects of the wall panel.

[0079] Furthermore, the subordinate main pixel point determination is specifically used to calculate the grayscale difference and distance difference between the sub-pixel point and the candidate main pixel point, and determine the main pixel point to which the sub-pixel point belongs according to a preset threshold, including:

[0080] If |GG″|≤ε1 and d≤ε2, the sub-pixel point belongs to the candidate main pixel point, wherein G represents the grayscale value of the sub-pixel point, G″ represents the grayscale value of the candidate main pixel point, d represents the distance between the sub-pixel point and the candidate main pixel point, ε1 represents the grayscale difference threshold, and ε2 represents the distance difference threshold.

[0081] Furthermore, the grayscale weight calculation is specifically used to calculate the grayscale weight of the sub-pixel point according to the grayscale value and position of the main pixel point to which the sub-pixel point belongs, including:

[0082] If the sub-pixel point belongs to the main pixel point in the i-th row and j-th column, the grayscale weight of the sub-pixel point is w=(i-α)·(β-j), wherein α represents the ratio of the number of rows where the main pixel point to which the sub-pixel point belongs to the total number of rows, and β represents the ratio of the number of columns where the main pixel point to which the sub-pixel point belongs to the total number of columns.

[0083] Further, the sub-pixel grayscale value determination is specifically used to determine the grayscale value of the sub-pixel according to the grayscale value of the main pixel to which the sub-pixel belongs, the grayscale weight of the sub-pixel and the grayscale value of the main pixel, including:

[0084] If the sub-pixel point belongs to the main pixel point in the i-th row and j-th column, the grayscale value of the sub-pixel point is g=w*gi+(1-w)*gj, wherein g represents the grayscale value of the sub-pixel point, w represents the grayscale weight of the sub-pixel point, gi represents the grayscale value of the main pixel point in the i-th row and j-th column, and gj represents the grayscale value of the main pixel point in the j-th column.

[0085] It should be noted that the advanced damage detection unit and color difference defect detection unit can realize the rapid and accurate identification of wallboard defects. The damage detection unit uses the feature extraction network, the region proposal network RPN and the ROI pooling layer to compress the information in these boxes into a fixed-size feature map to efficiently locate and identify the potential damaged area from the preprocessed image. The color difference defect detection unit realizes the accurate identification of wallboard color difference defects by constructing a sub-pixel grayscale matrix, accurately calculating the grayscale weight and the sub-pixel grayscale value, and ensures the accurate measurement and analysis of the sub-pixel level color difference. The system further adopts the subordinate main pixel determination and grayscale weight calculation to enhance the recognition ability of sub-pixel level defects. Finally, the result fusion and output module integrates the results of the two detection units, intelligently eliminates false detections and accurately integrates the effective defect information, and outputs an accurate and reliable final inspection report, which provides strong technical support for the quality control of wallboards. An intelligent wallboard defect detection system based on image recognition significantly improves the automation level and accuracy of defect detection, ensuring the high standard quality of wallboard products.

[0086] Preferably, the result fusion and output module is used to fuse the first detection result and the second detection result and output the final detection result, including:

[0087] Obtaining location information of a first type of defect in the wallboard image according to the first detection result; obtaining location information of a second type of defect according to the second detection result; the first type of defect is a hole, a crack or a missing block, and the second type of defect is a color difference defect;

[0088] Among them, when the position area of ​​the first detection result and the second detection result overlap, the overlapping area is analyzed again. When the overlapping result is a false detection due to color difference, the position of the marked area is damaged; when the first detection result and the second detection result have no position area overlap, it is determined that both results are valid defect detection results, and the results are recorded separately to obtain the final detection result.

[0089] In summary, the method of the present invention realizes the automated detection of wall panels, improves detection efficiency, and reduces labor costs; the method of the present invention adopts a deep learning algorithm to improve detection accuracy; the method of the present invention can simultaneously detect multiple types of defects and has a wide range of applications; the method of the present invention takes into account the correlation between different defects and avoids the occurrence of misjudgment.

[0090] Embodiment 2

[0091] Reference Figure 4 , an intelligent wallboard defect detection method based on image recognition according to an embodiment of the present invention comprises the following steps:

[0092] S100: Acquire an image of a wall panel to be detected through a camera;

[0093] S200: preprocessing the wallboard image data to be detected to obtain preprocessed image data, where the image preprocessing operations include image grayscale, filtering, size standardization, etc.;

[0094] S300: performing wall panel damage detection on the preprocessed image data by using a pre-trained first convolutional neural network model to obtain a first detection result;

[0095] S400: performing wallboard color difference defect detection on the preprocessed image data by using a pre-trained second convolutional neural network model to obtain a second detection result;

[0096] S500: Fusing the first detection result and the second detection result, and outputting a final detection result.

[0097] Specifically, the intelligent detection method for wallboard defects based on image recognition provided in this embodiment obtains the wallboard image to be detected through a camera, preprocesses the wallboard image data to be detected, obtains the preprocessed image data, performs wallboard damage detection on the preprocessed image data through a pre-trained first convolutional neural network model, obtains a first detection result, performs wallboard color difference defect detection on the preprocessed image data through a pre-trained second convolutional neural network model, obtains a second detection result, fuses the first detection result and the second detection result, and outputs the final detection result. The method improves detection efficiency and reduces labor costs; adopts a deep learning algorithm to improve detection accuracy; can detect multiple types of defects at the same time, and has a wide range of applications; takes into account the correlation between different defects, and avoids the occurrence of misjudgment.

[0098] Preferably, the method further comprises:

[0099] Set the position and angle of the camera, and place the camera at a predetermined height from the wall plate to be measured; adjust the angle of the camera so that the center of the camera lens and the horizontal direction of the wall plate to be measured form a predetermined angle.

[0100] Preferably, according to step S300, the method further includes: using the first convolutional neural network model to forward propagate the preprocessed image data to obtain a first prediction box and a first confidence score; if the first confidence score is greater than or equal to a predetermined threshold, determining the first prediction box as a first detection result;

[0101] According to step S400, it also includes: using the second convolutional neural network model to forward propagate the preprocessed image data to obtain a second prediction box and a second confidence score; if the second confidence score is greater than or equal to a predetermined threshold, determining the second prediction box as a second detection result.

[0102] Preferably, forward propagating the preprocessed image data using the first convolutional neural network model includes:

[0103] Extracting a set of feature maps from the preprocessed image data;

[0104] Predicting multiple region proposal boxes according to the feature map;

[0105] After performing convolution operations on each region proposal box, the information in the region proposal box is compressed into a feature map of a fixed size.

[0106] Preferably, forward propagating the preprocessed image data using the second convolutional neural network model includes:

[0107] Converting the grayscale image of the preprocessed image data into a sub-pixel grayscale matrix, wherein the sub-pixel grayscale matrix includes main pixel points and sub-pixel points;

[0108] Calculating the grayscale difference and distance difference between the sub-pixel point and the candidate main pixel point, and determining the main pixel point to which the sub-pixel point belongs according to a preset threshold;

[0109] Calculate the grayscale weight of the sub-pixel point according to the grayscale value and position of the main pixel point to which the sub-pixel point belongs;

[0110] Determine the grayscale value of the sub-pixel point according to the grayscale value of the main pixel point to which the sub-pixel point belongs, the grayscale weight of the sub-pixel point, and the grayscale value of the main pixel point;

[0111] A sub-pixel image is constructed according to the grayscale values ​​of the sub-pixel points.

[0112] Preferably, the calculating the grayscale difference and the distance difference between the sub-pixel point and the candidate main pixel point, and determining the main pixel point to which the sub-pixel point belongs according to a preset threshold, includes:

[0113] If |GG″|≤ε1 and d≤ε2, the sub-pixel point belongs to the candidate main pixel point, wherein G represents the grayscale value of the sub-pixel point, G″ represents the grayscale value of the candidate main pixel point, d represents the distance between the sub-pixel point and the candidate main pixel point, ε1 represents the grayscale difference threshold, and ε2 represents the distance difference threshold.

[0114] Preferably, calculating the grayscale weight of the sub-pixel point according to the grayscale value and position of the main pixel point to which the sub-pixel point belongs includes:

[0115] If the sub-pixel point belongs to the main pixel point in the i-th row and j-th column, the grayscale weight of the sub-pixel point is w=(i-α)·(β-j), wherein α represents the ratio of the number of rows where the main pixel point to which the sub-pixel point belongs to the total number of rows, and β represents the ratio of the number of columns where the main pixel point to which the sub-pixel point belongs to the total number of columns.

[0116] Preferably, determining the grayscale value of the sub-pixel point according to the grayscale value of the main pixel point to which the sub-pixel point belongs, the grayscale weight of the sub-pixel point, and the grayscale value of the main pixel point includes:

[0117] If the sub-pixel point belongs to the main pixel point in the i-th row and j-th column, the grayscale value of the sub-pixel point is g=w*gi+(1-w)*gj, wherein g represents the grayscale value of the sub-pixel point, w represents the grayscale weight of the sub-pixel point, gi represents the grayscale value of the main pixel point in the i-th row and j-th column, and gj represents the grayscale value of the main pixel point in the j-th column.

[0118] Preferably, the fusing the first detection result and the second detection result to output the final detection result includes:

[0119] Obtaining location information of a first type of defect in the wallboard image according to the first detection result; obtaining location information of a second type of defect according to the second detection result; the first type of defect is a hole, a crack or a missing block, and the second type of defect is a color difference defect;

[0120] Among them, when the position area of ​​the first detection result and the second detection result overlap, the overlapping area is analyzed again. When the overlapping result is a false detection due to color difference, the position of the marked area is damaged; when the first detection result and the second detection result have no position area overlap, it is determined that both results are valid defect detection results, and the results are recorded separately to obtain the final detection result.

[0121] Embodiment 3

[0122] This embodiment, after determining the grayscale value of the sub-pixel point, further includes: removing outliers; performing interpolation operation on the remaining sub-pixel points to obtain the final grayscale value of the sub-pixel point, specifically including:

[0123] Determine the grayscale value and position: Determine the grayscale values ​​of pixel A and pixel B as g A and g B , and their positions in the image;

[0124] Calculate distance: Calculate the distance d between pixel points A and B AB , if A and B are adjacent pixels, this distance is usually 1;

[0125] Determine the specific position of point P relative to A and calculate the distance d from point P to A P , if P is located exactly between A and B, then d P will be d AB / 2;

[0126] Apply the linear interpolation formula: Use the linear interpolation formula to calculate the gray value g of point P P ,

[0127]

[0128] If point P has a grayscale weight w, it is calculated based on the grayscale value and position of the main pixel to which point P belongs, so the final grayscale value of point P is g′ P It is expressed as:

[0129] g′ P =w×g P +(1-w)×(g A +g B ) / 2

[0130] During the calculation process, if point P is an outlier, its influence is removed or corrected according to the method of determining the gray value and position;

[0131] Repeat calculation: Repeat the above steps for all sub-pixel points that need to be interpolated;

[0132] Construct the final sub-pixel image: The grayscale values ​​g′ of all calculated sub-pixel points are P Put them back to their correct positions in the image and construct the final sub-pixel image.

[0133] Through the above steps, the sub-pixel points in the wallboard image can be effectively interpolated, thereby improving the accuracy and resolution of color difference detection, and in the wallboard defect detection, very small defects can be identified.

[0134] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technical personnel in this field can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any brief modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. An intelligent wallboard defect detection system based on image recognition, characterized in that: It includes image acquisition module, image detection module and result fusion and output module. The image acquisition module is used to acquire the wallboard image to be detected, preprocess the wallboard image data to be detected, and obtain the preprocessed image data; The image detection module detects the damage and color difference defects on the wallboard surface through the pre-trained first convolutional neural network model and the second convolutional neural network model; the image detection module includes a damage detection unit and a color difference defect detection unit, The damage detection unit performs wallboard damage detection on the preprocessed image data by using a pre-trained first convolutional neural network model to obtain a first detection result; The color difference defect detection unit: performs wallboard color difference defect detection on the preprocessed image data through a pre-trained second convolutional neural network model to obtain a second detection result; The result fusion and output module is used to fuse the first detection result and the second detection result, and output the final detection result.

2. According to claim 1, the wallboard defect intelligent detection system based on image recognition is characterized in that: The image acquisition module is also used to set the position and angle of the camera, placing the camera at a predetermined height from the wall plate to be measured; adjusting the angle of the camera so that the center of the camera lens and the horizontal direction of the wall plate to be measured form a predetermined angle.

3. According to claim 1, the wallboard defect intelligent detection system based on image recognition is characterized in that: The damage detection unit includes: using the first convolutional neural network model to perform forward propagation on the preprocessed image data to obtain a first prediction box and a first confidence score; if the first confidence score is greater than or equal to a predetermined threshold, determining the first prediction box as a first detection result; The color difference defect detection unit includes: using the second convolutional neural network model to forward propagate the preprocessed image data to obtain a second prediction box and a second confidence score; if the second confidence score is greater than or equal to a predetermined threshold, determining the second prediction box as a second detection result.

4. The wallboard defect intelligent detection system based on image recognition according to claim 3 is characterized by: The preprocessed image data is forward propagated using the first convolutional neural network model, which also includes a feature extraction network, a region proposal network RPN, and a ROI pooling layer. Wherein, the feature extraction network is used to extract a set of feature maps from the preprocessed image data; The region proposal network RPN is used to predict multiple region proposal boxes according to the feature map; The ROI pooling layer is used to compress the information in the region proposal box into a feature map of a fixed size after performing convolution operations on each region proposal box respectively.

5. The wallboard defect intelligent detection system based on image recognition according to claim 3 is characterized by: The preprocessed image data is forward propagated using the second convolutional neural network model, which also includes sub-pixel grayscale matrix construction, subordinate main pixel determination, grayscale weight calculation, sub-pixel grayscale value determination, and sub-pixel image construction. The sub-pixel grayscale matrix is ​​constructed to convert the grayscale image of the pre-processed image data into a sub-pixel grayscale matrix, and the sub-pixel grayscale matrix includes main pixel points and sub-pixel points; The subordinate main pixel point determination is used to calculate the grayscale difference and distance difference between the sub-pixel point and the candidate main pixel point, and determine the candidate main pixel point with the highest degree of membership corresponding to the sub-pixel point according to a preset threshold, that is, the main pixel point to which the sub-pixel point belongs; The grayscale weight calculation is used to calculate the grayscale weight of the sub-pixel point according to the grayscale value and position of the main pixel point to which the sub-pixel point belongs; The sub-pixel grayscale value determination is used to determine the grayscale value of the sub-pixel according to the grayscale value of the main pixel to which the sub-pixel belongs, the grayscale weight of the sub-pixel and the grayscale value of the main pixel; The sub-pixel image construction is used to construct a sub-pixel image according to the grayscale value of the sub-pixel point.

6. The wallboard defect intelligent detection system based on image recognition according to claim 5 is characterized in that: The subordinate main pixel point determination is specifically used to calculate the grayscale difference and distance difference between the sub-pixel point and the candidate main pixel point, and determine the main pixel point to which the sub-pixel point belongs according to a preset threshold, including: If |GG″|≤ε1 and d≤ε2, the sub-pixel point belongs to the candidate main pixel point, wherein G represents the grayscale value of the sub-pixel point, G″ represents the grayscale value of the candidate main pixel point, d represents the distance between the sub-pixel point and the candidate main pixel point, ε1 represents the grayscale difference threshold, and ε2 represents the distance difference threshold.

7. The wallboard defect intelligent detection system based on image recognition according to claim 5 is characterized by: The grayscale weight calculation is specifically used to calculate the grayscale weight of the sub-pixel point according to the grayscale value and position of the main pixel point to which the sub-pixel point belongs, including: If the sub-pixel point belongs to the main pixel point in the i-th row and j-th column, the grayscale weight of the sub-pixel point is w=(i-α)·(β-j), wherein α represents the ratio of the number of rows where the main pixel point to which the sub-pixel point belongs to the total number of rows, and β represents the ratio of the number of columns where the main pixel point to which the sub-pixel point belongs to the total number of columns.

8. The wallboard defect intelligent detection system based on image recognition according to claim 5 is characterized by: The sub-pixel grayscale value determination is specifically used to determine the grayscale value of the sub-pixel according to the grayscale value of the main pixel to which the sub-pixel belongs, the grayscale weight of the sub-pixel, and the grayscale value of the main pixel, including: If the sub-pixel point belongs to the main pixel point in the i-th row and j-th column, the grayscale value of the sub-pixel point is g=w*gi+(1-w)*gj, wherein g represents the grayscale value of the sub-pixel point, w represents the grayscale weight of the sub-pixel point, gi represents the grayscale value of the main pixel point in the i-th row and j-th column, and gj represents the grayscale value of the main pixel point in the j-th column.

9. The wallboard defect intelligent detection system based on image recognition according to claim 1 is characterized by: The result fusion and output module is used to fuse the first detection result and the second detection result and output the final detection result, including: Obtaining location information of a first type of defect in the wallboard image according to the first detection result; obtaining location information of a second type of defect according to the second detection result; the first type of defect is a hole, a crack or a missing block, and the second type of defect is a color difference defect; Among them, when the position area of ​​the first detection result and the second detection result overlap, the overlapping area is analyzed again. When the overlapping result is a false detection due to color difference, the position of the marked area is damaged; when the first detection result and the second detection result have no position area overlap, it is determined that both results are valid defect detection results, and the results are recorded separately to obtain the final detection result.

10. An intelligent detection method for wallboard defects based on image recognition, applied to an intelligent detection system for wallboard defects based on image recognition as claimed in any one of claims 1 to 9, characterized in that: The following steps are involved: Acquire the wall panel image to be inspected through the camera; Preprocessing the wallboard image data to be detected to obtain preprocessed image data; Performing wall panel damage detection on the preprocessed image data by using a pre-trained first convolutional neural network model to obtain a first detection result; Performing wallboard color difference defect detection on the preprocessed image data through a pre-trained second convolutional neural network model to obtain a second detection result; Fusion of the first detection result and the second detection result, outputting a final detection result; wherein the preprocessed image data is forward propagated using the first convolutional neural network model to obtain a first prediction box and a first confidence score; if the first confidence score is greater than or equal to a predetermined threshold, the first prediction box is determined as a first detection result; Performing forward propagation on the preprocessed image data using the second convolutional neural network model to obtain a second prediction box and a second confidence score; if the second confidence score is greater than or equal to a predetermined threshold, determining the second prediction box as a second detection result; The fusing the first detection result and the second detection result to output a final detection result includes: Obtaining location information of a first type of defect in the wallboard image according to the first detection result; obtaining location information of a second type of defect according to the second detection result; the first type of defect is a hole, a crack or a missing block, and the second type of defect is a color difference defect; Among them, when the position area of ​​the first detection result and the second detection result overlap, the overlapping area is analyzed again. When the overlapping result is a false detection due to color difference, the position of the marked area is damaged; when the first detection result and the second detection result have no position area overlap, it is determined that both results are valid defect detection results, and the results are recorded separately to obtain the final detection result.

Citation Information

Patent Citations

  • Building wall defect measurement method based on image and system thereof

    CN104568981A

  • Defect detection method and device

    CN114820543A

  • Steel surface defect detection system based on deep learning

    CN114926407A

  • Tile finished product defect automatic identification method and device, intelligent terminal and storage medium

    CN115496984A

  • Wallboard surface defect rapid detection method based on artificial intelligence

    CN116542972A