A high-precision vision-based method for detecting defects on the inner wall of automobile engine cylinder bores

Abstract

A high-precision vision-based method for detecting defects on the inner wall of a cylinder bore of an automobile engine. A preliminary feature extractor is used to extract the features of the preprocessed image. The extracted preliminary features are encoded using Hy‑FEN to obtain four encoded features of different sizes. The four features of different sizes are input into Ra‑FPN for feature encoding, compression, and fusion. The features encoded by Ra‑FPN and the anchor frame ratios obtained by the dual multi-head clustering module are input into the region generation network RPN to generate candidate defect regions, thereby obtaining preliminary defect regions. The encoded fusion features obtained from Ra‑FPN and the preliminary defect regions obtained from the region generation network RPN are input into the position correction module for position refinement. Finally, the defect detection results of the image to be detected are output, thereby achieving the defect detection task for the current image to be detected. The present invention overcomes the inadaptability of conventional target detection algorithms to the task of detecting defects on the inner wall of a cylinder bore of an automobile engine.

Description

Technical Field

[0001] The present invention belongs to the field of digital image processing, and in particular relates to a high-precision automobile engine cylinder bore inner wall defect detection method based on vision. Background Art

[0002] Defect detection is an essential and crucial step in the industrial manufacturing process, and its inspection process is currently mostly performed by professionally trained quality inspectors. Manual quality inspection is susceptible to subjective factors, such as the high labor intensity that can lead to poor inspector performance, resulting in poor inspection stability and consistency. Furthermore, manual quality inspection has a low degree of automation, low production efficiency, and difficulty in achieving lean production. Furthermore, rising human resource costs are driving the requirements for quality inspectors to a higher standard, requiring companies and factories to invest significant manpower, material, and financial resources to train large numbers of quality inspectors.

[0003] With the rapid development of machine vision and industrial automation, the concept of Industry 4.0 has become increasingly prominent. As an inevitable trend in the advancement of human science and technology, traditional defect detection tasks requiring manual operation will gradually be replaced by industrial robots, with industrial machine vision replacing manual vision for product defect detection. Therefore, research on defect detection technologies that utilize visual image-based defect detection algorithms to analyze each workpiece, improving detection accuracy while reducing the cost of manual inspection, is particularly important.

[0004] The engine is the "heart" of the car and continuously provides power to the entire vehicle. Its quality directly determines the power output, smoothness and driving safety of the car. Therefore, for common defects in the engine, such as the inner wall of the engine cylinder bore (see Figure 1 ) defects, which may affect the normal operation of the surrounding lubrication system, intake system, combustion system and exhaust system at the least, or even cause driving safety problems at the most serious. Based on this, it is urgent to detect automobile engine defects.

[0005] Existing defect detection algorithms primarily consist of traditional methods and deep learning-based methods. Traditional methods require manual design of different defect features depending on the task. Examples include texture-based, shape-based, and color-based methods. With the successful introduction of deep learning methods in various fields of machine vision in recent years, deep learning-based defect detection algorithms have gradually emerged. Currently, deep learning-based defect detection algorithms can be categorized by learning method into fully supervised algorithms, such as those based on representation learning and metric learning; semi-supervised algorithms; and unsupervised algorithms based on image and feature spaces. However, due to the unique characteristics of automotive engine cylinder bore inner wall defects, such as scratches of various shapes, tiny pinholes, and bumps, current defect detection methods are not fully suitable for detecting engine cylinder bore inner wall defects. Summary of the Invention

[0006] The present invention provides a high-precision vision-based automobile engine cylinder bore inner wall defect detection method to solve the inapplicability of existing defect detection algorithms for engine inner wall defect detection and provide a new algorithm option for engine defect detection and metal surface defect detection.

[0007] The present invention is achieved through the following technical solutions:

[0008] A high-precision vision-based method for detecting defects on the inner wall of a cylinder bore of an automobile engine, the method comprising:

[0009] S1: Preprocess the current input image to be detected;

[0010] S2: Input the preprocessed image into the preliminary feature extractor for preliminary feature extraction to obtain the extracted features F p ;

[0011] S3: Preliminary extracted features F based on S2 p Input into the first stage encoder E1 in the hybrid feature extraction network Hy-FEN to encode the image features and obtain the feature F encoded by E1 E1 ;

[0012] S4: Encoding feature F obtained based on S3 E1 Input into the second stage encoder E2 in the hybrid feature extraction network Hy-FEN to obtain the feature F encoded by E2 E2 ;

[0013] S5: Encoding feature F obtained based on S4 E2Input into the third stage encoder E3 in the hybrid feature extraction network Hy-FEN to obtain the feature F encoded by E3 E3 ;

[0014] S6: Encoding feature F obtained based on S5 E3 Input into the fourth stage encoder E4 in the hybrid feature extraction network Hy-FEN to obtain the feature F encoded by E4 E4 ;

[0015] S7: The encoding feature F obtained based on S2, S4, S5 and S6 p 、F E2 、F E3 、F E4 Input into the receptive field perception feature pyramid network Ra-FPN for further feature encoding and fusion, and output the encoded fusion feature Ψ;

[0016] S8: Based on the encoding fusion feature Ψ obtained in S7, it is input into the region generation network RPN to generate defect candidate regions and obtain the preliminary defect region

[0017] S9: Encoded fusion feature Ψ obtained based on S7 and preliminary defect area obtained based on S8 The image is input into the position correction module for further refinement of the position, and finally the defect detection results of the image to be inspected are output.

[0018] Furthermore, the pre-processing of the input image in S1 specifically includes: normalization, image rotation, image translation, image shearing, horizontal flipping and color conversion;

[0019] The specific process of the normalization process is:

[0020]

[0021] Among them, I is the input image matrix, I min , I max is the minimum and maximum value of the pixels in the image matrix. At this time, the normalized image pixel value range is [0, 1];

[0022] The specific process of image rotation is as follows: for any original pixel point (x1, y1), when it rotates clockwise from the starting angle α1 to β through Δα, the trigonometric formula is:

[0023]

[0024] We can get:

[0025]

[0026] The specific process of image translation is as follows: for any original pixel point (x1, y1), it is translated by (Δx, Δy) from the starting point (x1, y1) along its respective component axes to the point (x2, y2), that is:

[0027]

[0028] The specific process of image shearing is as follows: In this embodiment, only the image to be detected is sheared along the y-axis. For any original pixel point (x1, y1), the corresponding point (x2, y2) is obtained by shearing along the y-axis according to the shearing angle Δη, as follows:

[0029]

[0030] The specific process of horizontal flipping is as follows: for any image The horizontal flipping process is as follows:

[0031]

[0032] The specific process of the color conversion is as follows: first, the image to be detected I in the RGB image space is gamma corrected to obtain the corrected image I1, and then the corrected image I1 is converted into the HSV image space as follows:

[0033]

[0034] Among them, I R , I G , I B are the R, G, and B channels of the input image.

[0035] The image I2 in the HSV image space is obtained, where the H dimension conversion method is:

[0036]

[0037] The conversion method between S dimension and V dimension is:

[0038]

[0039] H dimension I2 2-H Do the following processing:

[0040]

[0041] S-dimensional I2 2-S Do the following processing:

[0042] I′ 2-S =I 2-S ×μ (12)

[0043] Finally, the processed HSV image space image I3 is converted back to the RGB image space to obtain I4, and I4 is gamma-corrected again to obtain the final color-transformed output image I5. The gamma correction mentioned above is specifically as follows:

[0044] I5=I4 ν (13)

[0045] The parameters μ and ν used above are randomly obtained in the interval [0, 1.2] according to uniform distribution, that is, μ, ν ~ U (0, 1.2);

[0046] Except for the normalization operation, the other preprocessing operations are also performed randomly according to the uniform distribution, that is:

[0047] ζ~U(0,1),

[0048]

[0049] Furthermore, the features input in S2-S6 are first encoded and downsampled in the straight main path through the cascade (convolution-normalization-activation) -> (convolution-normalization-activation) -> (convolution-normalization) method. At the same time, the input features are also encoded and downsampled once in the bypass path, thereby overcoming the problem of gradient disappearance caused by the network being too deep. Finally, the features obtained in the main path and the bypass path are fused in a channel-by-channel and point-by-point summation manner as the output of the current residual block.

[0050] Furthermore, it is composed of two feature extraction blocks shaped like residual blocks, (layer normalization -> multi-dimensional multi-head mutual attention module -> dropout layer) -> (layer normalization -> multi-layer perceptron module MLP -> dropout layer);

[0051] Among them, the multi-dimensional multi-head mutual attention module considers both position and channel information, uses average pooling to obtain smoother features, and adopts maximum pooling to select more recognizable features, thereby making a higher response to specific areas and more suitable for defect detection tasks. This is called multi-dimensional;

[0052] For the input feature F Ei , it is encoded in the manner of a multi-dimensional multi-head mutual attention module to obtain the corresponding query matrix Q, key matrix K and value matrix V. The obtained Q, K, V matrices are projected using h different linear transformations through the mutual attention module. Finally, the mutual attention results of h different linear transformations are spliced ​​and fused, so it is called a multi-head mutual attention module, specifically:

[0053]

[0054] in,

[0055] Among them, q, k, v are coding functions, i = {2, 3} represents the coding layer index, W C is the learnable fusion matrix, CAT is the feature concatenation function, h is the number of linearly changing heads, x={x1,…,x N}.

[0056] Furthermore, the preliminary feature extractor in S2 is structurally constructed by stacking a set of convolution, activation, and batch normalization layers to obtain a preliminary 2x downsampled feature map F p ;

[0057] The first stage encoder E1 in the hybrid feature extraction network Hy-FEN in S3 is structurally composed of stacking n1 residual blocks, and then performing a preliminary 2x downsampling of the input feature F p Perform 2x downsampling encoding to obtain 4x downsampling encoding feature F E1 ;

[0058] The second stage encoder E2 in the hybrid feature extraction network Hy-FEN in S4 is structurally composed of stacking n2 residual blocks, which is used to downsample the input image feature F by 4 times. E1 Further 2 times downsampling encoding is performed to obtain 8 times downsampling encoding feature F E2 .

[0059] Furthermore, the third stage encoder E3 in the hybrid feature extraction network Hy-FEN in S5 is structurally composed of a pre-coding layer PM composed of convolution, n3 pyramid transformation layers and a normalization layer, which is used to extract the 8-fold down-sampled image features F E2 Perform 2x downsampling encoding again to obtain 16x downsampling encoding feature F E3 ;

[0060] The fourth stage encoder E4 in the hybrid feature extraction network Hy-FEN in S6 is composed of a pre-coding layer PM composed of convolution, n4 pyramid transformation layers and a normalization layer, which is used to extract the 16-fold down-sampled image features F E3 Perform the last 2x downsampling encoding to obtain the 32x downsampling encoding feature F E4 .

[0061] Furthermore, the receptive field perception feature pyramid network Ra-FPN in S7 is structurally composed of 4 feature mapping layers and 1 multi-layer perceptron module MLP2; the input feature F p 、F E2 、F E3 、F E4Input into 4 feature mapping layers respectively for compression and mapping of feature channels;

[0062] At the same time, F E4 The output feature F after the corresponding feature mapping layer sa4 The result is further input into the remaining multi-layer perceptron MLP2 module to further downsample the features and obtain a 64-fold downsampled feature map F. sa5 , used to capture large targets;

[0063] The receptive field perception feature pyramid network Ra-FPN in S7 also fuses feature maps of adjacent sizes, i.e., 32x and 16x fusion, 16x and 8x fusion, and 8x and 2x feature map fusion to obtain fused and compressed features, and splices 5 groups of compressed, mapped and fused features Ψ and outputs them.

[0064] Furthermore, the region generation network RPN in S8 is composed of a multi-layer perceptron. The multi-layer perceptron is structured by stacking convolutional layers to regress five groups of feature maps of different sizes in the input feature Ψ.

[0065] All defect sizes in the defect library are clustered by the dual multi-head clustering module and input into the RPN module to obtain a preliminary capture of the defect area location. The optimal m areas are selected from all captured areas as the final output of the RPN defect preliminary area location.

[0066] Furthermore, the position correction module in S9 is composed of a region size adjustment module and a regression branch; wherein the region size adjustment module is to capture the m defect preliminary position regions Further adjustment to the same size is beneficial for subsequent regression and classification operations; all defect areas adjusted by the area size adjustment module are The matrix is ​​input into the regression branch and the classification branch to further refine all the initially captured defect locations and classify the defect categories, and output the final defect location coordinates. And the corresponding category information

[0067] The beneficial effects of the present invention are:

[0068] In the framework of FasterRCNN, the present invention introduces the hybrid feature extraction network Hy-FEN and the receptive field perception feature pyramid network Ra-FPN to form a high-precision vision-based automobile engine cylinder bore inner wall defect detection method, thereby improving the detection accuracy of automobile engine cylinder bore inner wall surface defects. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 It is a physical picture of the engine cylinder hole of the present invention.

[0070] Figure 2 It is a flow chart of the method of the present invention.

[0071] Figure 3 It is a residual block structure diagram of the present invention.

[0072] Figure 4 It is a diagram of the pyramid transformation layer structure of the present invention.

[0073] Figure 5 This is a structural diagram of the receptive field perception feature pyramid network Ra-FPN of the present invention.

[0074] Figure 6 This is the structure diagram of the regional generation network RPN of the present invention.

[0075] Figure 7 It is a structural diagram of the position correction module of the present invention. DETAILED DESCRIPTION

[0076] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0077] The present invention is based on FasterRCNN. For the current image I to be detected, it is first input into the preprocessing module for random preprocessing of the image, and the preprocessed image I is p Input into the preliminary feature extractor to obtain the preliminary extracted features F with 2 times downsampling p , the initial feature F is extracted p Then it is input into the first stage encoding layer E1 of the hybrid feature extraction network Hy-FEN to obtain the 4-fold downsampled encoding feature F E1 , the 4-fold downsampled encoding feature F E1 Input to the second stage encoding layer E2 of the hybrid feature extraction network Hy-FEN to obtain the 8-fold downsampled encoding feature F E2 , the 8-fold downsampled encoding feature F E2 Input to the third stage encoding layer E3 of the hybrid feature extraction network Hy-FEN to obtain the 16-fold downsampled encoding feature F E3 , the 16-fold downsampled encoding feature F E3 Input to the fourth stage encoding layer E4 of the hybrid feature extraction network Hy-FEN to obtain the 32-fold downsampled encoding feature F E4Then the obtained 2x, 8x, 16x and 32x downsampled encoding features F p 、F E2 、F E3 and F E4 The input is input into the receptive field perception feature pyramid network Ra-FPN to map, compress and fuse the input downsampled coding features. At the same time, the 32-fold downsampled features are further downsampled by 2 times using a set of multi-layer perceptron modules MLP2 to obtain 64-fold downsampled features. Finally, the 5-layer feature Ψ compressed and fused by the receptive field perception feature pyramid network Ra-FPN is obtained. The feature Ψ compressed and fused by the receptive field perception feature pyramid network Ra-FPN is further input into the region generation network RPN to obtain the preliminary defect area. Finally, the preliminary defect area The features Ψ compressed and fused by the receptive field perception feature pyramid network Ra-FPN are input into the final position correction module to correct the initial defect area. Further refinement, output the final defect location after refinement and defect categories

[0078] A high-precision vision-based method for detecting defects on the inner wall of a cylinder bore of an automobile engine, the method comprising:

[0079] S1: Preprocess the current input image to be detected;

[0080] S2: Input the preprocessed image into the preliminary feature extractor for preliminary feature extraction to obtain the extracted features F p ;

[0081] S3: Preliminary extracted features F based on S2 p Input into the first stage encoder E1 in the hybrid feature extraction network Hy-FEN to encode the image features and obtain the feature F encoded by E1 E1 ;

[0082] S4: Encoding feature F obtained based on S3 E1 Input into the second stage encoder E2 in the hybrid feature extraction network Hy-FEN to obtain the feature F encoded by E2 E2 ;

[0083] S5: Encoding feature F obtained based on S4 E2 Input into the third stage encoder E3 in the hybrid feature extraction network Hy-FEN to obtain the feature F encoded by E3 E3 ;

[0084] S6: Encoding feature F obtained based on S5E3 Input into the fourth stage encoder E4 in the hybrid feature extraction network Hy-FEN to obtain the feature F encoded by E4 E4 ;

[0085] S7: The encoding feature F obtained based on S2, S4, S5 and S6 p 、F E2 、F E3 、F E4 Input into the receptive field perception feature pyramid network Ra-FPN for further feature encoding and fusion, and output the encoded fusion feature Ψ;

[0086] S8: Based on the encoding fusion feature Ψ obtained in S7, it is input into the region generation network RPN to generate defect candidate regions and obtain the preliminary defect region

[0087] S9: Encoded fusion feature Ψ obtained based on S7 and preliminary defect area obtained based on S8 The image is input into the position correction module for further refinement of the position, and finally the defect detection results of the image to be inspected are output.

[0088] Furthermore, the random pre-processing operations to be performed on the image to be detected I in S1 specifically include normalization, image rotation, image translation, image shearing, horizontal flipping and color conversion;

[0089] The specific process of the normalization process is:

[0090]

[0091] Among them, I is the input image matrix, I min , I max is the minimum and maximum value of the pixels in the image matrix. At this time, the normalized image pixel value range is [0, 1];

[0092] The specific process of image rotation is as follows: for any original pixel point (x1, y1), when it rotates clockwise from the starting angle α1 to β through Δα, the trigonometric formula is:

[0093]

[0094] We can get:

[0095]

[0096] In this embodiment, the rotation angle range is [-360°, 360°].

[0097] The specific process of image translation is as follows: for any original pixel point (x1, y1), it is translated by (Δx, Δy) from the starting point (x1, y1) along its respective component axes to the point (x2, y2), that is:

[0098]

[0099] In this implementation, the range of the translation variable is [0, 10%].

[0100] The specific process of image shearing is as follows: In this embodiment, only the image to be detected is sheared along the y-axis. For any original pixel point (x1, y1), the corresponding point (x2, y2) is obtained by shearing along the y-axis according to the shearing angle Δη, as follows:

[0101]

[0102] In this embodiment, the range of the shear angle is [0, 40°].

[0103] The specific process of horizontal flipping is as follows: for any image The horizontal flipping process is as follows:

[0104]

[0105] The specific process of the color conversion is as follows: first, the image to be detected I in the RGB image space is gamma corrected to obtain the corrected image I1, and then the corrected image I1 is converted into the HSV image space as follows:

[0106]

[0107] Among them, I R , I G , I B are the R, G, and B channels of the input image.

[0108] The image I2 in the HSV image space is obtained, where the H dimension conversion method is:

[0109]

[0110] The conversion method between S dimension and V dimension is:

[0111]

[0112] H dimension I2 2-H Do the following processing:

[0113]

[0114] S-dimensional I2 2-SDo the following processing:

[0115] I′ 2-S =I 2-S ×μ (12)

[0116] Finally, the processed HSV image space image I3 is converted back to the RGB image space to obtain I4, and I4 is gamma-corrected again to obtain the final color-transformed output image I5. The gamma correction mentioned above is specifically as follows:

[0117] I5=I4 ν (13)

[0118] The parameters μ and ν used above are randomly obtained in the interval [0, 1.2] according to uniform distribution, that is, μ, ν ~ U (0, 1.2);

[0119] Except for the normalization operation, the other preprocessing operations are also performed randomly according to the uniform distribution, that is:

[0120] ζ~U(0,1),

[0121]

[0122] Further, the preliminary feature extractor of this embodiment refers to Figure 2 , its specific structure is shown in Table 1. Through cascade convolution, activation and regularization, the preprocessed image I p Perform preliminary feature encoding and implement 2x downsampling.

[0123] Table 1

[0124]

[0125] Further, the hybrid feature extraction network Hy-FEN structure can be found in Figure 2 The whole network consists of 4 stages of coding layers, of which the first two stages are composed of the residual blocks in ResNet in a cascaded manner, and the last two stages are composed of the pre-coding layer, the pyramid transformation layer and the normalization layer. Figure 3 and Figure 4 , its structural parameters are shown in Table 2. For the initial extraction feature F p First, it is input into two coding layers E1 and E2 composed of staggered blocks to encode the features and perform downsampling at the same time, and the encoded and downsampled features F are obtained. E1 and F E2 .

[0126] The specific process is shown in Figure 3For the input features, the features are first encoded and downsampled in the straight main path through the cascade (convolution-normalization-activation) -> (convolution-normalization-activation) -> (convolution-normalization) method. At the same time, the input features are also encoded and downsampled once in the bypass path, thereby overcoming the problem of gradient vanishing caused by the network being too deep. Finally, the features obtained in the main path and the bypass path are fused in a channel-by-channel and point-by-point summation manner as the output of the current residual block. This embodiment also uses the method of E1 being cascaded by 3 residual blocks and E2 being cascaded by 4 residual blocks. For the coding layer E3 and the coding layer E4, this embodiment no longer uses the same convolutional neural network method as the coding layer E1 and the coding layer E2 to extract features, but adopts the Transformer structure, that is, refer to Figure 4 The pyramid transformation layer shown in FIG2 is a layer that further improves the structure of PVT2 by introducing position information. The overall structure of the pyramid transformation layer is shown in FIG2. Figure 4 As shown in the upper middle layer, it consists of two feature extraction blocks shaped like residual blocks, (layer normalization -> multi-dimensional multi-head mutual attention module -> Dropout layer) -> (layer normalization -> multi-layer perceptron module MLP -> Dropout layer). The multi-dimensional multi-head mutual attention module considers both position and channel information, uses average pooling to obtain smoother features, and uses maximum pooling to select features with higher recognition, thereby making a higher response to specific areas and more suitable for defect detection tasks. It is called multi-dimensional. At the same time, for the input feature F Ei , it is encoded in the manner of a multi-dimensional multi-head mutual attention module to obtain the corresponding query matrix Q, key matrix K and value matrix V. The obtained Q, K, V matrices are projected using h different linear transformations through the mutual attention module. Finally, the mutual attention results of h different linear transformations are spliced ​​and fused, so it is called a multi-head mutual attention module, specifically:

[0127]

[0128] in,

[0129] Among them, q, k, v are coding functions, i = {2, 3} represents the coding layer index, W C is the learnable fusion matrix, CAT is the feature concatenation function, h is the number of linearly changing heads, x={x1,…,x N}.

[0130] Table 2

[0131]

[0132] Furthermore, the receptive field perception feature pyramid network Ra-FPN downsamples the input to 1 / 2, 1 / 8, 1 / 16, and 1 / 32 to encode the feature F p 、F E2 、F E3 and F E4 Mapping compression and fusion are performed, and further compression and downsampling are performed to obtain 1 / 64 times features. Figure 5 , for the feature maps of different input downsampling rates, the features are expanded and compressed according to different 4 groups of feature mapping layers. In view of the fact that the receptive field has a positive promoting effect on the detection problem, and taking into account the final detection efficiency problem, the present invention adopts the method of void convolution to expand the perception ability of the model without increasing the amount of calculation; at the same time, in order to balance the stability of the feature magnitude, that is, the model is not overly biased towards containing the features of a certain layer, the present invention adopts different channel mapping coefficients ε = [ε1, ε2, ε3, ε4] to expand and compress feature maps of different sizes. The structural parameters of the receptive field perception feature pyramid network Ra-FPN are shown in Table 3.

[0133] Table 3

[0134]

[0135] It can be seen that for large-scale feature maps, the present invention adopts channel compression, i.e., anti-bottleneck layer structure; for small-scale features, the present invention adopts channel expansion, i.e., bottleneck layer structure. Finally, the features output by the five feature mapping layers are spliced ​​by channel to obtain the final output multi-scale compressed features of the Ra-FPN module. H and W are the dimensions of the input image.

[0136] Furthermore, the region generation network RPN mainly completes the preliminary extraction of defect areas from the input feature maps of 5 different sizes, and obtains the preliminary defect area Region Generation Network Figure 6The compressed fusion features obtained from the receptive field perception feature pyramid network Ra-FPN are preliminarily decoded, and two branches are used for preliminarily decoding the defect area. One branch is used for the coordinate regression task of the defect area, and the other branch is used for the category classification task of the extracted area. At this time, only whether there are defects is distinguished, that is, the area with defects is marked as the foreground area, and the area without defects is marked as the background area. Among them, in order to make the model converge better to the defect area, the present invention also proposes a dual multi-head clustering module, which mainly completes the clustering of defect sizes. The module consists of two parts. The first part is to obtain 10 aspect ratios of the defect area under different clustering methods by using 10 clustering methods AffinityPropagation, AgglomerativeClustering, Birch, DBSCAN, KMeans, MiniBatchKMeans, MeanShift, OPTICS, SpectralClustering, and GaussianMixture. Then, the 10 aspect ratios are clustered again into the optimal 5 aspect ratios using the KMeans clustering method, and used as the optimal ratio of the anchor box used by RPN. Finally, all the obtained regions are filtered using the non-maximum suppression NMS method. In this embodiment, only the first 512 regions are retained, so the preliminary defect region matrix is ​​obtained. In this embodiment, the specific structural parameters of RPN are shown in Table 4.

[0137] Table 4

[0138]

[0139] Furthermore, the defect area position correction module is used to further refine the obtained defect area, see Figure 7 The input of this module consists of two parts. The first part is the multi-scale compressed fusion feature output from Ra-FPN. The second part is the defect area initially extracted by the RPN module First, both inputs are input into the candidate region extraction module, which mainly completes the extraction of preliminary defect regions from the multi-scale compression fusion feature Ψ. Corresponding features, the process is divided into two steps:

[0140] Step 1: Correspondence between preliminary defect areas and multi-scale compressed fusion features.

[0141] This embodiment selects a candidate area corresponding to the 224×224 level at the 16-fold downsampling position. Therefore, for the 32-fold downsampling feature map, a candidate area of ​​448×448 is used, for the 8-fold downsampling feature map, a candidate area of ​​112×112 is used, and for the 2-fold downsampling feature map, a candidate area of ​​28×28 is used. This makes the shallow features focus more on the detection of small-sized defects, while the deep features focus more on the detection of large-sized defects. Specifically, in the implementation process, for example, for the candidate area of ​​(12, 15, 425, 412), since its area is (425-12)×(412-15)=412×397, it is obvious that the corresponding regional features should be extracted in the 32-fold downsampling feature map Ψ4. Therefore, first reduce the region coordinates by 32 times, that is, (0.375, 0.469, 13.281, 12.485), and then extract the features of the corresponding position in Ψ4 according to the reduced coordinates to obtain the feature map The feature extraction methods corresponding to the remaining candidate regions are similar.

[0142] Step 2: Resizing the feature map.

[0143] According to the first step, the size of each candidate region is different. Such a size is not conducive to the subsequent data parallel calculation to improve the efficiency of the model operation. In order to solve this problem, the ROIAlign technology is used. Specifically, for any size of the candidate feature map f k , in this embodiment, the size is adjusted to 7×7. First, for any h k ⅹw k The feature map is divided into 49 small grids according to the small area of ​​h / 7×w / 7. For each small grid area, this embodiment uses a sampling rate of 2, that is, each small grid area is divided into 4 smaller areas again. Finally, the eigenvalues ​​corresponding to the center points of the current 4 small grid areas are obtained by bilinear interpolation, and are used as the final eigenvalues ​​of the current h / 7×w / 7 grid area. The eigenvalues ​​corresponding to the remaining 48 areas are obtained in the same way, which overcomes the disadvantage of the previous position alignment using ROIPooling technology.

[0144] After the above two steps, we get the features corresponding to 512 candidate regions of the same size. Resize it again to get Input it into the subsequent regression branch to obtain the final refined defect position coordinate matrix Input into the subsequent category judgment branch to obtain the final refined defect category matrix Specific implementation method two:

[0146] This embodiment is specifically carried out according to the following steps:

[0147] 1. Data preparation stage

[0148] Since the current research on the use of machine vision in engine cylinder bore defect detection is relatively limited, there is no available relevant data set for this problem. To address this issue, the data used in the present invention are all collected by a self-developed defect detection system. All sample images to be tested are taken by a defect detection system designed by the present invention team. Different sample images are obtained by adjusting the depth and brightness of the lens entering the cylinder bore during the experiment. All images are expanded after the preprocessing in the above-mentioned embodiment 1. The present invention team carefully annotated them using the annotation software LabelImg and named them HIT-EngD. Its detailed information is as follows:

[0149] 1) The dataset contains a total of 8,800 sample images, of which 8,175 contain defects and 625 do not;

[0150] 2) All sample images have a high resolution of 1000×1000;

[0151] 3) All data is collected from nearly 200 different cylinder bore types, covering common cylinder bore types;

[0152] 4) A total of 61,736 defects were annotated in the dataset, with 30,845 defects classified into three categories: cracks (30,845), pinholes (23,146), and bumps (545).

[0153] 5) It includes common and challenging scenarios, such as the detection of extremely small defects such as pinholes at high resolution, dense defects, and blurred defects caused by lighting changes.

[0154] Therefore, it is challenging to conduct relevant experiments on the HIT-EngD dataset. The inventor team plans to make it public to encourage relevant researchers to conduct subsequent research.

[0155] All training and testing data used in this example comes from HIT-EngD. 7040 sample images were selected for training and 1760 sample images for testing, with a training set:test set ratio of 4:1. During training and testing, each frame was expanded to 640×640.

[0156] 2. Training phase

[0157] Repeat steps b1 and b2 for the training process of the training network.

[0158] b1: Initialize network training parameters and hyperparameters. In this example, the maximum number of iterations is set to 200; the batch training size is set to 32; in order to have a faster convergence speed during training, the learning rate is set to 2×10 -4 , and decay the learning rate to 2×10 at 175 cycles -5 ; The AdamW algorithm is introduced as the optimizer, the weight decay coefficient is 0.0001, and a momentum factor of 0.9 is introduced; all network parameters are initialized according to the Xavier method; the loss function selected for the classification problem is the cross entropy loss, and the loss function selected for the coordinate regression problem is the Smooth-L1 loss.

[0159] b2: Use the image obtained in step S1 as the input to the training network and start training the network. Training is an iterative process. Each iteration first calculates the loss value of the forward propagation, and then updates all the parameters of the network in the reverse direction with the goal of minimizing the loss value.

[0160] 3. Testing phase

[0161] Repeat steps I to II for the testing phase.

[0162] Step I: Repeat step S1 for the test image;

[0163] Step II: Input the test image frame obtained in step I into the training network trained in steps b1 and b2 to output the defect detection result of the image to be detected.

[0164] The experimental results are shown in Table 5. The evaluation indicators used are well known to those skilled in the art and will not be described in detail in the present embodiment. Table 5 shows that the algorithm proposed in this invention has a significant improvement over FasterRCNN in all 7 evaluation indicators, among which the comprehensive performance indicator mAP is 50 Compared with the FasterRCNN algorithm, it is improved by 31.9, and the other indicators are improved by 27.9, 35.8, 40.6, 21.4, 32.7, and 37.2 in the order listed in the table.

[0165] Table 5

[0166]

[0167] At the same time, the algorithm proposed in this paper also achieves real-time performance (~10FPS, NVIDIA-A100). Therefore, the engine defect detection algorithm proposed in this paper has greater advantages in both defect detection performance and defect detection efficiency, providing a new algorithm choice for defect detection tasks.

Claims

1. A high-precision vision-based method for detecting defects on the inner wall of a cylinder bore of an automobile engine, characterized in that: The detection method comprises: S1: Preprocess the current input image to be detected; S2: Input the preprocessed image into the preliminary feature extractor for preliminary feature extraction to obtain the extracted features F p ; S3: Preliminary extracted features F based on S2 p Input into the first stage encoder E1 in the hybrid feature extraction network Hy-FEN to encode the image features and obtain the feature F encoded by E1 E1 ,The hybrid feature extraction network Hy-FEN consists of four stages of coding layers, where the first two stages of coding layers are composed of residual blocks in ResNet in a cascade manner, and the last two stages of coding layers are composed of pre-coding layers, pyramid transformation layers and normalization layers; S4: Encoding feature F obtained based on S3 E1 Input into the second stage encoder E2 in the hybrid feature extraction network Hy-FEN to obtain the feature F encoded by E2 E2 ; S5: Encoding feature F obtained based on S4 E2 Input into the third stage encoder E3 in the hybrid feature extraction network Hy-FEN to obtain the feature F encoded by E3 E3 ; S6: Encoding feature F obtained based on S5 E3 Input into the fourth stage encoder E4 in the hybrid feature extraction network Hy-FEN to obtain the feature F encoded by E4 E4 ; S7: The encoding feature F obtained based on S2, S4, S5 and S6 p 、F E2 、F E3 、F E4 The input is fed into the receptive field perception feature pyramid network Ra-FPN for further feature encoding and fusion, and the encoded fusion feature Ψ is output. The receptive field perception feature pyramid network Ra-FPN is structurally composed of 4 feature mapping layers and 1 multi-layer perceptron module MLP2. S8: Based on the encoding fusion feature Ψ obtained in S7, it is input into the region generation network RPN to generate defect candidate regions and obtain the preliminary defect region S9: Encoded fusion feature Ψ obtained based on S7 and preliminary defect area obtained based on S8 The image is input into the position correction module for further refinement of the position, and finally the defect detection results of the image to be inspected are output.

2. The high-precision vision-based method for detecting defects on the inner wall of a cylinder bore of an automobile engine according to claim 1, characterized in that: The preprocessing of the input image in S1 specifically includes: normalization, image rotation, image translation, image shearing, horizontal flipping and color conversion; The specific process of the normalization process is: Among them, I is the input image matrix, I min , I max is the minimum and maximum value of the pixels in the image matrix. At this time, the normalized image pixel value range is [0, 1]; The specific process of image rotation is as follows: for any original pixel point (x1, y1), when it rotates clockwise from the starting angle α1 to β through Δα, the trigonometric formula is: We can get: The specific process of image translation is as follows: for any original pixel point (x1, y1), it is translated by (Δx, Δy) from the starting point (x1, y1) along its respective component axes to the point (x2, y2), that is: The specific process of image shearing is: only the image to be detected is sheared along the y-axis. For any original pixel point (x1, y1), the corresponding point (x2, y2) is obtained by shearing along the y-axis according to the shearing angle Δη. Specifically, The specific process of horizontal flipping is as follows: for any image The horizontal flipping process is as follows: The specific process of the color conversion is as follows: first, the image to be detected I in the RGB image space is gamma corrected to obtain the corrected image I1, and then the corrected image I1 is converted into the HSV image space as follows: Among them, I R , I G , I B are the R, G, and B channels of the input image; The image I2 in the HSV image space is obtained, where the H dimension conversion method is: The conversion method between S dimension and V dimension is: H dimension I2 2-H Do the following processing: S-dimensional I2 2-S Do the following processing: I′ 2-S =I 2-S ×μ (12) Finally, the processed HSV image space image I3 is converted back to the RGB image space to obtain I4, and I4 is gamma-corrected again to obtain the final color-transformed output image I5. The gamma correction mentioned above is specifically as follows: I5=I4 ν (13) The parameters μ and ν used above are randomly obtained in the interval [0,1.2] according to uniform distribution, that is, μ, ν ~ U (0,1.2); Except for the normalization operation, the other preprocessing operations are also performed randomly according to the uniform distribution, that is: ζ~U(0,1), 3. The high-precision vision-based method for detecting defects on the inner wall of a cylinder bore of an automobile engine according to claim 1, characterized in that: The features input in S2-S6 are first encoded and downsampled in a cascaded manner in the straight main path. At the same time, the input features are also encoded and downsampled in the side path, thereby overcoming the gradient vanishing problem caused by the network being too deep. Finally, the features obtained in the main path and the side path are fused in a channel-by-channel and point-by-point summation manner as the output of the current residual block.

4. The high-precision vision-based method for detecting defects on the inner wall of a cylinder bore of an automobile engine according to claim 3, characterized in that: The pyramid transformation layer consists of two feature extraction blocks shaped like residual blocks, layer normalization -> multi-dimensional multi-head mutual attention module -> dropout layer -> layer normalization -> multi-layer perceptron module MLP -> dropout layer; Among them, the multi-dimensional multi-head mutual attention module considers both position and channel information, uses average pooling to obtain smoother features, and adopts maximum pooling to select more recognizable features, thereby making a higher response to specific areas and more suitable for defect detection tasks. This is called multi-dimensional; For the input feature F Ei , it is encoded in the manner of a multi-dimensional multi-head mutual attention module to obtain the corresponding query matrix Q, key matrix K and value matrix V. The obtained Q, K, V matrices are projected using h different linear transformations through the mutual attention module. Finally, the mutual attention results of h different linear transformations are spliced ​​and fused, so it is called a multi-head mutual attention module, specifically: in, Among them, q, k, v are coding functions, i = {2, 3} represents the coding layer index, W C is the learnable fusion matrix, CAT is the feature concatenation function, h is the number of linearly changing heads, x={x1,…,x N }.

5. The high-precision vision-based method for detecting defects on the inner wall of a cylinder bore of an automobile engine according to claim 3, characterized in that: The preliminary feature extractor in S2 is constructed by stacking a set of convolution, activation, and batch normalization layers to obtain a preliminary 2x downsampled feature map F. p ; The first stage encoder E1 in the hybrid feature extraction network Hy-FEN in S3 is structurally composed of stacking n1 residual blocks, and then performing a preliminary 2x downsampling of the input feature F p Perform 2x downsampling encoding to obtain 4x downsampling encoding feature F E1 ; The second stage encoder E2 in the hybrid feature extraction network Hy-FEN in S4 is structurally composed of stacking n2 residual blocks, which is used to downsample the input image feature F by 4 times. E1 Further 2 times downsampling encoding is performed to obtain 8 times downsampling encoding feature F E2 .

6. The high-precision vision-based method for detecting defects on the inner wall of a cylinder bore of an automobile engine according to claim 5, characterized in that: The third stage encoder E3 in the hybrid feature extraction network Hy-FEN in S5 is composed of a pre-coding layer PM composed of convolution, n3 pyramid transformation layers and a normalization layer, which is used to extract the 8-fold down-sampled image features F E2 Perform 2x downsampling encoding again to obtain 16x downsampling encoding feature F E3 ; The fourth stage encoder E4 in the hybrid feature extraction network Hy-FEN in S6 is composed of a pre-coding layer PM composed of convolution, n4 pyramid transformation layers and a normalization layer, which is used to extract the 16-fold down-sampled image features F E3 Perform the last 2x downsampling encoding to obtain the 32x downsampling encoding feature F E4 .

7. The high-precision vision-based method for detecting defects on the inner wall of a cylinder bore of an automobile engine according to claim 5, characterized in that: In S7, the input feature F p 、F E2 、F E3 、F E4 Input into 4 feature mapping layers respectively for compression and mapping of feature channels; At the same time, F E4 The output feature F after the corresponding feature mapping layer sa4 The result is further input into the remaining multi-layer perceptron MLP2 module to further downsample the features and obtain a 64-fold downsampled feature map F. sa5 , used to capture large targets; The receptive field perception feature pyramid network Ra-FPN in S7 also fuses feature maps of adjacent sizes, i.e., 32x and 16x fusion, 16x and 8x fusion, and 8x and 2x feature map fusion to obtain fused and compressed features, and splices 5 groups of compressed, mapped and fused features Ψ and outputs them.

8. The high-precision vision-based method for detecting defects on the inner wall of a cylinder bore of an automobile engine according to claim 7, characterized in that: The region generation network RPN in S8 shown is composed of a multi-layer perceptron. The multi-layer perceptron is structured by stacking convolutional layers and then regressing five groups of feature maps of different sizes in the input feature Ψ. All defect sizes in the defect library are clustered by the dual multi-head clustering module and input into the RPN module to obtain a preliminary capture of the defect area location. The optimal m areas are selected from all captured areas as the final output of the RPN defect preliminary area location.

9. The high-precision vision-based method for detecting defects on the inner wall of a cylinder bore of an automobile engine according to claim 8, characterized in that: The position correction module in S9 is composed of a region size adjustment module and a regression branch; wherein the region size adjustment module is to capture the m defect preliminary position regions Further adjustment to the same size is beneficial for subsequent regression and classification operations; all defect areas adjusted by the area size adjustment module are The matrix is ​​input into the regression branch and the classification branch to further refine all the initially captured defect locations and classify the defect categories, and output the final defect location coordinates. And the corresponding category information

Images