A prefabricated bay window defect detection system based on improved Faster R-CNN

Abstract

The present invention discloses a prefabricated bay window defect detection system based on improved Faster R-CNN, including an image collection device and a background PC, the image collection device including a screw rod, the left and right ends of the screw rod surface are threadedly connected with sliding nuts, the left and right sides of the sliding nut are fixedly connected with extension rods, the surface of the extension rod is respectively sleeved with a first fixed sleeve, a second fixed sleeve and a third fixed sleeve from the direction close to the screw rod to the direction away from the screw rod, and a spring is fixedly connected between the first fixed sleeve and the second fixed sleeve and between the second fixed sleeve and the third fixed sleeve and on the surface of the extension rod, and the present invention relates to the technical field. The prefabricated bay window defect detection system based on improved Faster R-CNN can realize image acquisition of prefabricated bay windows of different specifications by providing a screw rod, cooperating with the extension rod, the first fixed sleeve, the second fixed sleeve, the third fixed sleeve, the spring, and the industrial camera, and the position of the industrial camera in each collection group can be adaptively adjusted.

Description

Technical Field

[0001] The present invention relates to the technical field, and in particular to a prefabricated bay window defect detection system based on an improved Faster R-CNN. Background Art

[0002] With the development of computer technology and image processing technology, machine vision detection technology has been widely used in industrial defect detection. For example, Celik et al. combined image algorithms based on double threshold binarization, wavelet transform and morphological operations with feedforward neural networks to achieve defect detection; Afshar et al. used statistical methods to detect defect edges, and then used multi-strategy support vector machines as classifiers to identify defects; Guo Hui et al. proposed a method based on gray-level co-occurrence matrix and hierarchical clustering algorithm to achieve defect detection.

[0003] In recent years, deep learning technology, as an important branch of machine learning, has become increasingly mature in the industrial sector. YJCha et al. used convolutional neural networks to identify concrete crack defects; FC Chen et al. proposed a deep learning framework based on naive Bayes and convolutional neural network data fusion for detecting crack defects; Yu Yongwei et al. used convolutional neural networks to extract the essential features of suspicious defect areas, and then used radial basis networks to identify defects.

[0004] Existing quality is the life of the company. Product quality and production efficiency have become the top priorities of the company's development. Therefore, quality inspection of prefabricated bay windows is crucial to ensuring the quality of prefabricated bay windows. In the daily production process of prefabricated bay windows, there may be defects on the product surface. The traditional surface inspection method is mainly manual visual inspection. Long-term visual inspection is prone to fatigue, and the false detection rate also increases, while reducing work efficiency. The use of manual visual inspection methods has problems such as large workload, low detection efficiency, and high false detection rate. Traditional machine vision inspection methods usually use industrial cameras to capture images, and then use digital image technology to preprocess the captured images, and then set a classifier to classify and detect defective images. This type of detection model is relatively simple and requires manual feature engineering or manual selection of certain key parameters. It is greatly affected by human subjectivity, resulting in low generalization ability of the detection method and inability to cope with uncontrollable factors such as the many types of defects in prefabricated bay windows and complex production environment. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention provides a prefabricated bay window defect detection system based on an improved Faster R-CNN, which solves the problems of large workload, low detection efficiency, and high false detection rate of manual visual inspection methods. Traditional machine vision inspection methods usually use industrial cameras to collect images. Such detection models are relatively simple and require manual feature engineering or manual selection of certain key parameters. They are greatly affected by human subjectivity, resulting in low generalization ability of the detection method and inability to cope with uncontrollable factors such as the large variety of prefabricated bay window defects and the complex production environment.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: a prefabricated bay window defect detection system based on improved Faster R-CNN, including an image collection device and a background PC, the image collection device including a screw rod, the left and right ends of the screw rod surface are threadedly connected with sliding nuts, the left and right sides of the sliding nut are fixedly connected with an extension rod, the surface of the extension rod is respectively sleeved with a first fixed sleeve, a second fixed sleeve and a third fixed sleeve from the direction close to the screw rod to the direction away from the screw rod, a spring is fixedly connected between the first fixed sleeve and the second fixed sleeve and between the second fixed sleeve and the third fixed sleeve and on the surface of the extension rod, and the first fixed sleeve, the second fixed sleeve and the third fixed sleeve are fixedly connected with an industrial camera.

[0007] Preferably, threads of different directions are provided on the left and right ends of the screw rod, a limit rod is fixedly connected in the middle of the screw rod surface, and connecting rods are rotatably connected on the left and right sides of the limit rod, wherein the bottom end of one of the connecting rods is rotatably connected to the top of the third fixed sleeve, and the bottom end of the other connecting rod is rotatably connected to the top of the first fixed sleeve.

[0008] Preferably, an extension rod, a first fixed sleeve, a second fixed sleeve, a third fixed sleeve, two springs, three industrial cameras, a limit rod and two connecting rods form a collection group, and the image collection device is provided with four collection groups, and two adjacent collection groups are perpendicular to each other.

[0009] Preferably, a base plate is provided on the front side of the screw rod, a servo motor is fixedly connected to the rear side of the base plate, and an output end of the servo motor is fixedly connected to the screw rod.

[0010] Preferably, a support arm is fixedly connected to the front side of the base plate, a bottom end of the support arm is fixedly connected to a base, and a pulley is fixedly connected to the bottom of the base.

[0011] Preferably, the background PC is fixed on the top of the base, and the background PC includes an analysis system.

[0012] Preferably, the analysis system includes image preprocessing, improved Faster R-CNN, prefabricated bay window defect detection and prefabricated bay window defect judgment.

[0013] Preferably, the output end of the image preprocessing is coupled to the input end of the improved Faster R-CNN, the output end of the improved Faster R-CNN is coupled to the input end of the prefabricated convex window defect detection, and the output end of the prefabricated convex window defect detection is coupled to the input end of the prefabricated convex window defect judgment.

[0014] Beneficial Effects

[0015] The present invention provides a prefabricated bay window defect detection system based on improved Faster R-CNN. Compared with the prior art, it has the following beneficial effects:

[0016] (1) The prefabricated bay window defect detection system based on the improved Faster R-CNN comprises an image collection device and a background PC, wherein the image collection device comprises a screw rod, the left and right ends of the screw rod surface are threadedly connected with sliding nuts, the left and right sides of the sliding nut are fixedly connected with an extension rod, the surface of the extension rod is respectively sleeved with a first fixing sleeve, a second fixing sleeve and a third fixing sleeve from the direction close to the screw rod to the direction away from the screw rod, a spring is fixedly connected between the first fixing sleeve and the second fixing sleeve and between the second fixing sleeve and the third fixing sleeve and on the surface of the extension rod, the first fixing sleeve, the second fixing sleeve and the third fixing sleeve are fixedly connected with an industrial camera, the left and right ends of the screw rod are provided with threads in different directions, the middle of the screw rod surface is fixedly connected with a limit rod, and the limit rod is fixedly connected with the limit rod. The left and right sides of the positioning rod are rotatably connected with connecting rods, the bottom end of one connecting rod is rotatably connected to the top of the third fixed sleeve, and the bottom end of the other connecting rod is rotatably connected to the top of the first fixed sleeve. An extension rod, a first fixed sleeve, a second fixed sleeve, a third fixed sleeve, two springs, three industrial cameras, a limit rod and two connecting rods form a collection group. The image collection device is provided with four collection groups, and two adjacent collection groups are perpendicular to each other. By providing a screw rod, in conjunction with the extension rod, the first fixed sleeve, the second fixed sleeve, the third fixed sleeve, the spring, and the industrial camera, image acquisition can be realized for prefabricated bay windows of different specifications, and the position of the industrial camera in each collection group can be adaptively adjusted.

[0017] (2) The prefabricated bay window defect detection system based on the improved Faster R-CNN is provided with a base plate on the front side of the screw rod, a servo motor is fixedly connected to the rear side of the base plate, the output end of the servo motor is fixedly connected to the screw rod, a support arm is fixedly connected to the front side of the base plate, the bottom end of the support arm is fixedly connected to the base, and the bottom of the base is fixedly connected to a pulley. The pulley can be used to quickly and conveniently move the device, and the base plate and the servo motor are used to make the adjustment more convenient, thereby speeding up the work efficiency and increasing the practicality.

[0018] (3) The prefabricated bay window defect detection system based on the improved Faster R-CNN is fixed on the top of the base by the background PC, the background PC includes an analysis system, the analysis system includes image preprocessing, improved Faster R-CNN, prefabricated bay window defect detection and prefabricated bay window defect judgment, the output end of the image preprocessing is coupled and connected with the input end of the improved Faster R-CNN, the output end of the improved Faster R-CNN is coupled and connected with the input end of the prefabricated bay window defect detection, the output end of the prefabricated bay window defect detection is coupled and connected with the input end of the prefabricated bay window defect judgment, the improved Faster R-CNN network is used to realize fast and real-time prefabricated bay window defect detection, solves the problem that manual visual inspection, long-term visual inspection is prone to fatigue and high false detection rate, improves product quality inspection efficiency, realizes feature extraction through automatic learning, solves the problem that general feature extraction cannot realize target detection in a complex and changeable production environment, and the improved Faster R-CNN network is compared with the original network, so that small target defects are not easily lost during the detection process, and improves the accuracy of prefabricated bay window quality inspection. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a front view of the present invention;

[0020] Figure 2 is a side view of the substrate in the present invention;

[0021] Figure 3 It is a partial side view of the image collection device in the present invention;

[0022] Figure 4 For the present invention Figure 3 A partial enlarged view of the middle A;

[0023] Figure 5 is a logic flow chart of the present invention;

[0024] Figure 6 This is a schematic diagram of the improved Faster R-CNN network in the present invention;

[0025] Figure 7 It is a schematic diagram of the defect prediction network of prefabricated convex window integrated with feature pyramid in the present invention;

[0026] Figure 8 Schematic diagram of the basic structure of the residual network in the present invention.

[0027] In the figure: 1. Image collection device; 2. Background PC; 101. Screw; 102. Sliding nut; 103. Extension rod; 104. First fixed sleeve; 105. Second fixed sleeve; 106. Third fixed sleeve; 107. Spring; 108. Industrial camera; 109. Limit rod; 110. Connecting rod; 111. Base plate; 112. Servo motor; 113. Support arm; 114. Base; 115. Pulley; 201. Analysis system; 202. Image preprocessing; 203. Improved Faster R-CNN; 204. Prefabricated bay window defect detection; 205. Prefabricated bay window defect judgment. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] See also Figure 1-7The present invention provides a technical solution: a prefabricated bay window defect detection 204 system based on improved Faster R-CNN, comprising an image collection device 1 and a background PC 2, the image collection device 1 comprising a screw rod 101, both left and right ends of the surface of the screw rod 101 are threadedly connected with sliding nuts 102, both left and right sides of the sliding nut 102 are fixedly connected with an extension rod 103, the surface of the extension rod 103 is respectively sleeved with a first fixing sleeve 104, a second fixing sleeve 105 and a third fixing sleeve 106 from the direction close to the screw rod 101 to the direction away from the screw rod 101, and the surface of the extension rod 103 is located between the first fixing sleeve 104 and the second fixing sleeve 105 and between the second fixing sleeve 105 and the third fixing sleeve 106. The surfaces are all fixedly connected with springs 107, the first fixed sleeve 104, the second fixed sleeve 105 and the third fixed surface are all fixedly connected with industrial cameras 108, the left and right ends of the screw rod 101 are provided with threads in different directions, the middle of the surface of the screw rod 101 is fixedly connected with a limit rod 109, the left and right sides of the limit rod 109 are rotatably connected with connecting rods 110, the bottom end of one connecting rod 110 is rotatably connected to the top of the third fixed sleeve 106, the bottom end of the other connecting rod 110 is rotatably connected to the top of the first fixed sleeve 104, an extension rod 103, a first fixed sleeve 104, a second fixed sleeve 105, a third A fixed sleeve 106, two springs 107, three industrial cameras 108, a limit rod 109 and two connecting rods 110 form a collection group. The image collection device 1 is provided with four collection groups. Two adjacent collection groups are perpendicular to each other. By providing a screw rod 101, in conjunction with an extension rod 103, a first fixed sleeve 104, a second fixed sleeve 105, a third fixed sleeve 106, a spring 107, and an industrial camera 108, image collection can be achieved for prefabricated bay windows of different specifications, and the position of the industrial camera 108 in each collection group can be adjusted adaptively. A substrate 111 is provided on the front side of the screw rod 101, and a rear side of the substrate 111 is provided. A servo motor 112 is fixedly connected to the side, and the output end of the servo motor 112 is fixedly connected to the screw rod 101. A support arm 113 is fixedly connected to the front side of the substrate 111. The bottom end of the support arm 113 is fixedly connected to a base 114. A pulley 115 is fixedly connected to the bottom of the base 114. The pulley 115 can be used to quickly and conveniently move the device. The substrate 111 and the servo motor 112 make the adjustment more convenient, speed up the work efficiency, and increase the practicality. The background PC2 is fixed on the top of the base 114. The background PC2 includes an analysis system 201. The analysis system 201 includes an image preprocessing 202, an improved Faster R-CNN203, a prefabricated bay window defect detection 204 and a prefabricated bay window defect judgment 205. The output end of the image preprocessing 202 is coupled to the input end of the improved Faster R-CNN203, and the output end of the improved Faster R-CNN203 is coupled to the input end of the prefabricated bay window defect detection 204.The output end of the prefabricated bay window defect detection 204 is coupled to the input end of the prefabricated bay window defect judgment 205, and the improved Faster R-CNN203 network is used to realize fast and real-time prefabricated bay window defect detection 204, which solves the problem of fatigue and high false detection rate caused by manual visual inspection for a long time, improves the efficiency of product quality inspection, and realizes feature extraction through automatic learning, which solves the problem that general feature extraction cannot realize target detection in a complex and changeable production environment. Compared with the original network, the improved Faster R-CNN203 network makes it difficult for small target defects to be lost during the detection process, thereby improving the accuracy of prefabricated bay window quality inspection.

[0030] See also Figure 8 The specific technical steps are as follows:

[0031] Among them, C1, C2, and C3 are the initial feature maps, P3, P2, and P1 are the feature maps after the horizontal connection of 1*1 convolution, and then the 3*3 convolution is used to eliminate the aliasing effect caused by the upsampling process, and the possible areas of defects in the image are calibrated.

[0032] Where x represents the input value, Fx is the output value of x after two layers of convolution and one layer of Relu activation function. In ResNet, the input and output are added together using skip layer connection. At this time, the unit training target becomes Fx=Hx-x, that is, the residual of the training unit. The comprehensive output y of the residual unit is as follows:

[0033]

[0034] Where is the Relu activation function, W1 and W2 are the weights of the first and second layer networks respectively.

[0035] Meanwhile, the contents not described in detail in this specification belong to the prior art known to those skilled in the art, and the model parameters of each electrical appliance are not specifically limited, and conventional equipment can be used.

[0036] When in use, the pulley 115 is pushed to drive the entire device to move. After reaching the appropriate position, the servo motor 112 is turned on, and then the screw rod 101 is driven to rotate, driving the sliding nut 102 to move on the surface of the screw rod 101. When the sliding nut 102 drives the extension rod 103 to move, the limit rod 109 pulls the connecting rod 110, so that the first fixed sleeve 104 and the third fixed sleeve 106 are closer to the second fixed sleeve 105, and then the industrial camera 108 is turned on to collect images, and the image data is transmitted to the analysis system 201.

[0037] Step 1: Prefabricate the Bay Window Image Collection

[0038] An industrial camera 108 is deployed at the front end, and a long connection is established with the back end PC2, and the regularly collected image data is transmitted to the back end PC2 through Socket.

[0039] Step 2: Image Preprocessing 202

[0040] Image preprocessing 202 maximizes the restoration of the original features of the image and minimizes useless information such as noise that is irrelevant to detection.

[0041] 2.1 Image Enhancement

[0042] 2.1.1 Image Noise Reduction Processing

[0043] This paper uses median filtering to process image noise. The grayscale value of the target point is set to the median of all points in a certain area around it, so that the pixel value in its area is closer to the true value, thereby eliminating independent noise signals.

[0044] 2.1.2 Image sharpening

[0045] After image filtering, the clarity of image contours and edges will be reduced. The Laplace operator is used for sharpening to enhance the difference between the grayscale values ​​of objects and background in the image.

[0046] 2.2 De-meaning and normalization

[0047] The function library is called to remove the mean and normalize the image data, and the data is mapped to between -1 and 1, so that the data can converge quickly and find the optimal solution when the neural network model is trained.

[0048] Step 3: Defect Detection

[0049] According to the characteristics of target detection technology and the surface defect images of prefabricated bay windows, the improved Faster R-CNN203 network is adopted, and FPN is used to fuse multi-scale feature maps to generate feature maps with high-level and low-level semantics. RPN then provides a series of possible defect candidate areas in the image, and ResNet101 residual network is used to add the input and output together through "skip connection" to extract image features. Finally, Faster R-CNN is used to classify the feature maps that fuse all the information to complete the target classification and regression tasks.

[0050] The feature pyramid network fuses multi-scale feature maps, reduces the loss of small target defects in training, and improves the generalization ability of defect detection. It mainly consists of three parts. The first part is upsampling through the convolutional network, the second part is downsampling from the feature map, and the third part is horizontally connected to fuse the feature maps of the same size in the first two parts to obtain multi-scale feature maps.

[0051] The traditional VGG16 network as the basic feature extraction network is prone to gradient diffusion or disappearance during the back propagation of the deep network, and small target defects are easily lost during the detection process. Therefore, this paper uses the deep residual network ResNet to extract image features. ResNet adds the input and output of the unit together and activates it. On the one hand, it reduces the speed of gradient disappearance, and on the other hand, it allows small target defects to survive longer in the deep network.

[0052] 3.1 Dataset Annotation

[0053] Prefabricated bay windows mainly have seven types of defects: chipped corners, rough surfaces, honeycombs, cracks, pores, slurry leakage, and rough surface differences. The collected sample data sets are labeled, and the labeling tool LabelImg is used to label the defect types and corresponding bounding box coordinates in the image data set to generate an xml file. A Python script is written to batch process the xml file to extract the labeling box coordinates and target defect categories to generate a txt file and a training set path file.

[0054] 3.2 Generating Dataset

[0055] To generate the test set, 20% of the images were selected from the annotated image dataset as the test set, where the number of samples of each defect type was roughly the same, and the rest were used to create the training set and validation set.

[0056] 3.3 Setting anchor points of different sizes

[0057] In order for the model to find suitable hyperparameters to improve detection accuracy, anchor points of different scales and aspect ratios are set.

[0058] 3.4 Building Faster R-CNN Network Model

[0059] Use the Pytorch framework to provide related convolution, pooling, upsampling functions, downsampling functions, etc. to build the Faster R-CNN convolutional neural network and initialize the model.

[0060] 3.5 Faster R-CNN Model Training

[0061] Faster RCNN model training, the labeled prefabricated convex window samples and their corresponding labels are input into the model training, and the stochastic gradient descent method is used to iteratively update. The hyperparameters are compared and selected until the loss function tends to be stable. The training is completed and the model parameters are saved.

[0062] 3.6 Faster R-CNN Model Testing

[0063] Load the prefabricated bay window test set as input into the trained Faster R-CNN model, record the test results, and analyze the optimized model.

[0064] Step 4: Defect determination of prefabricated bay windows 205

[0065] The optimized Faster R-CNN model is applied to the detection and judgment of visual defects of prefabricated bay windows.

[0066] Step 5: Processing of test results

[0067] Prefabricated bay windows with defects in the inspection results will enter the defect repair process, and prefabricated bay windows without defects will enter the next production process.

[0068] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0069] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A prefabricated bay window defect detection system based on improved Faster R-CNN, comprising an image collection device (1) and a background PC (2), characterized in that: The image collection device (1) comprises a screw rod (101), the left and right ends of the surface of the screw rod (101) are both threadedly connected to sliding nuts (102), the left and right sides of the sliding nut (102) are both fixedly connected to an extension rod (103), the surface of the extension rod (103) is respectively sleeved with a first fixing sleeve (104), a second fixing sleeve (105) and a third fixing sleeve (106) from the direction close to the screw rod (101) to the direction away from the screw rod (101), the surface between the first fixing sleeve (104) and the second fixing sleeve (105) and between the second fixing sleeve (105) and the third fixing sleeve (106) and located on the surface of the extension rod (103) are all fixedly connected to a spring (107), and the first fixing sleeve (104), the second fixing sleeve (105) and the third fixing sleeve are all fixedly connected to the surface of the industrial camera (108); The left and right ends of the screw rod (101) are provided with threads in different directions, the middle of the surface of the screw rod (101) is fixedly connected to a limit rod (109), and the left and right sides of the limit rod (109) are rotatably connected to connecting rods (110), wherein the bottom end of one of the connecting rods (110) is rotatably connected to the top of the third fixing sleeve (106), and the bottom end of the other connecting rod (110) is rotatably connected to the top of the first fixing sleeve (104); An extension rod (103), a first fixing sleeve (104), a second fixing sleeve (105), a third fixing sleeve (106), two springs (107), three industrial cameras (108), a limiting rod (109) and two connecting rods (110) form a collection group, and the image collection device (1) is provided with four collection groups, and two adjacent collection groups are perpendicular to each other; A base plate (111) is provided on the front side of the lead screw (101), a servo motor (112) is fixedly connected to the rear side of the base plate (111), and an output end of the servo motor (112) is fixedly connected to the lead screw (101).

2. The prefabricated bay window defect detection system based on improved Faster R-CNN according to claim 1, characterized in that: The front side of the base plate (111) is fixedly connected to a support arm (113), the bottom end of the support arm (113) is fixedly connected to a base (114), and the bottom of the base (114) is fixedly connected to a pulley (115).

3. The prefabricated bay window defect detection system based on improved Faster R-CNN according to claim 1, characterized in that: The background PC (2) is fixed on the top of the base (114), and the background PC (2) includes an analysis system (201).

4. The prefabricated bay window defect detection system based on improved Faster R-CNN according to claim 3, characterized in that: The analysis system (201) includes image preprocessing (202), an improved Faster R-CNN (203), prefabricated bay window defect detection (204) and prefabricated bay window defect judgment (205).

5. The prefabricated bay window defect detection system based on improved Faster R-CNN according to claim 4, characterized in that: The output end of the image preprocessing (202) is coupled to the input end of the improved Faster R-CNN (203), the output end of the improved Faster R-CNN (203) is coupled to the input end of the prefabricated bay window defect detection (204), and the output end of the prefabricated bay window defect detection (204) is coupled to the input end of the prefabricated bay window defect judgment (205).

Images