A method for detecting defects in the inner wall of a camshaft mounting hole of a vehicle with strong generalization capability

Abstract

The application provides a camshaft mounting hole inner wall defect detection method with strong generalization capability. The method is used for pre-processing all images to be detected, and the pre-processed images are firstly input into an improved open set detection algorithm for detection of all defects to obtain defect position information detected by the open set detection algorithm; the pre-processed images are further input into a selected closed set detection algorithm for detection of defects of known categories to obtain corresponding defect detection position information; and the defect position information obtained by the open set detection algorithm and the closed set detection algorithm is fused to obtain a final defect detection result corresponding to the input image to be detected. The application overcomes the inadaptability of a conventional target detection algorithm to the defect detection task of the camshaft mounting hole inner wall of an automobile engine, improves the generalization of the model, and has excellent detection capability for any category of defects of the camshaft mounting hole inner wall of the automobile engine.

Description

Technical Field

[0001] This invention belongs to the field of digital image processing technology, and in particular relates to a method for detecting defects in the inner wall of automotive camshaft mounting holes with strong generalization ability. Background Technology

[0002] As the "heart" of a car, the engine is a core component and demands high quality. Improving engine quality has become one of the most important focuses for automakers. The camshaft is a crucial part of the engine's valve train, primarily used to control the opening and closing of the valves, ensuring the intake of fresh air and the exhaust of combustion gases. Camshafts typically rotate at high speeds and must withstand significant torque; therefore, they are generally made of high-quality alloy steel. Their quality primarily affects engine valve timing, leading to other serious consequences. Common defects on the inner wall of the camshaft mounting holes include scratches, pinholes, and foreign objects. Currently, quality inspection of the inner wall mainly relies on manual visual inspection using magnifying glasses or other endoscopes, resulting in low efficiency, poor stability, difficulty in achieving lean production, and poor traceability.

[0003] With the rapid development of industrial robot technology, due to its advantages of high efficiency, high precision, low cost, and traceability, the use of computer vision technology for product defect detection is gradually replacing traditional manual visual inspection. Existing defect detection algorithms can be broadly categorized into traditional methods based on manual design and methods based on neural networks. Neural networks offer advantages in task processing: 1) they can fully fit arbitrarily complex nonlinear relationships; 2) they possess strong robustness and fault tolerance; and 3) they are model-independent, state-independent, and object-independent. As a result, in recent years, more and more researchers have focused their defect detection research on neural network-based frameworks. However, existing traditional defect detection methods, as mentioned earlier, are models carefully designed for specific problems; and the mainstream approach to neural network-based defect detection is currently based on supervised learning. The result is that both methods have excellent processing capabilities for known problems but lack the ability to handle unknown problems, meaning their model generalization performance is unsatisfactory. In practical applications, this manifests as excellent detection capabilities for known defect categories, but unstable detection capabilities for newly emerging defect categories. Summary of the Invention

[0004] The purpose of this invention is to address the problems in existing technologies by proposing a method for detecting defects in the inner wall of automotive camshaft mounting holes with strong generalization capabilities. This method enhances the defect detection algorithm, enabling it to not only detect known defect categories but also those of unknown categories, thereby reducing missed detections and improving the accuracy of camshaft inner wall defect detection. It provides a new algorithmic option for detecting defects in the inner wall of camshaft mounting holes and for detecting defects on metal surfaces.

[0005] This invention is achieved through the following technical solution: This invention proposes a method for detecting defects in the inner wall of automotive camshaft mounting holes with strong generalization ability. The method specifically comprises:

[0006] S1: Preprocess the current input image;

[0007] S2: Input the preprocessed image into the open set detection algorithm for defect detection, and obtain the defect information detected by the open set detection algorithm (B). OP C OP ), where B is the defect location and C is the corresponding defect category;

[0008] S3: Input the preprocessed image into the closed-set detection algorithm for defect detection, and obtain the defect information detected by the closed-set detection algorithm (B). CL C CL );

[0009] S4: The defect information (B) obtained through the open set detection algorithm... OP C OP ) and the defect detection information (B) obtained by the closed set detection algorithm CL C CL The input image is fused using a multi-algorithm source fusion module to obtain the final defect detection information (B, C).

[0010] Furthermore, the preprocessing of the input image in S1 specifically includes: image size resizing operation and image normalization processing;

[0011] The specific process of the image resizing operation is as follows: For any input image First, the size transformation factor α is obtained according to formula (1). Then, the transformed size (H) is obtained according to formula (2) based on the original size and the obtained size transformation factor. S W S Finally, the input image is resized according to the obtained transformed image size using bilinear interpolation.

[0012]

[0013]

[0014] The image normalization process is performed according to the normal distribution standardization process, as described in formula (3):

[0015]

[0016] Where x∈I represents any element in the input image, and μ and σ represent the mean and variance of the image, respectively. Therefore, the standardized image will highlight the unique attributes of the image itself and reduce the common attributes between images.

[0017] Furthermore, the open set defect detection algorithm in S2 is completed in 5 steps, specifically:

[0018] S21: Use a looping text prompt method to provide prompts to the open set detection model (T1, T2, ..., T...). n );

[0019] S22: Input the prompt text sequentially into the open set detection module to obtain the defect detection information (B) obtained from the current prompt text. OP1 i C OP1 i );

[0020] S23: Merge the defect detection information obtained from multiple text prompts to obtain the merged defect information;

[0021] S24: The fused defect information is filtered through a filtering module to obtain (B) OP1 C OP1 );

[0022] S25: The filtered defect information is refined through the category refinement module to obtain the final defect information (B) obtained by the open set detection algorithm. OP C OP ).

[0023] Furthermore, the cyclic text prompting method in S21 refers to the following: for any input text prompt, the keywords are separated by a specific marker. When inputting, the model splits the text according to the marker, and then inputs the split keywords into the open set detection model in sequence to perform the defect detection process of the corresponding keyword prompt.

[0024] Furthermore, the open set detection module is structurally composed of five parts, specifically including: a text feature extractor, an image feature extractor, a text-image feature fusion unit, a language-guided query unit, and a cross-modal decoding unit. The text feature extractor uses a BERT model based on the Transformer architecture to encode the input language into corresponding feature vectors through bidirectional language representation. The image feature extractor uses a Swin model based on the Transformer architecture to extract features from the input image. The text-image feature fusion unit fuses text and image features by stacking a series of self-attention and cross-attention modules and a feedforward network. The language-guided query unit further guides the defect detection process using the input text. The cross-modal decoding unit decodes the obtained optimal matching pair to obtain the defect location corresponding to the final text.

[0025] Furthermore, the defect detection information fusion in S23 specifically refers to retaining all the defect location information obtained under each keyword in the cyclic prompt, thereby obtaining a comprehensive defect location information.

[0026] Furthermore, in S24, a nonmaximum suppression algorithm is used to filter overlapping position information; specifically, all the currently fused defect detection boxes are arranged in descending order of confidence to obtain a list. Next, select the box with the highest confidence level. As the current defect baseline frame, from Starting from the second position, retrieve the corresponding defect location information and the current defect baseline frame sequentially. Calculate the positional overlap; where, positional overlap It is obtained by solving the Intersection over Union (IOU) between two defect boxes, and the obtained positional overlap is then calculated. Compare with a given threshold ζ, when When the value is high, it indicates that there is a high similarity between the current defect box and the current defect reference box, so it is placed in the reference box. Remove from the middle, and iterate through to the next middle in the same way. The last defect location in the middle; then from Starting from the second defect position and using it as the current defect reference frame, perform the same operation as above until the defect reference frame is traversed to... Up to the last position, the filtered defect location information (B) will be obtained. OP1 C OP1 ).

[0027] Furthermore, the specific operation steps of the category refinement module in S25 are as follows:

[0028] S251: The (B) obtained from S24 OP1 C OP1 All defect locations are extracted from the original image area using ROI extraction units, resulting in all corresponding extracted regions I. OP1 ;

[0029] S252: Extract all regions I obtained through S251 OP1 The dimensions were readjusted to 160×160, resulting in the adjusted size.

[0030] S253: Input the extracted region adjusted in S252 into the category determination network to perform the defect category determination task for the corresponding region, and obtain the final complete defect information (B) after defect determination. OP C OP ).

[0031] This invention proposes an electronic device, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the method for detecting defects in the inner wall of an automotive camshaft mounting hole with strong generalization ability.

[0032] This invention proposes a computer-readable storage medium for storing computer instructions, which, when executed by a processor, implement the steps of a method for detecting defects in the inner wall of an automotive camshaft mounting hole with strong generalization capabilities.

[0033] The beneficial effects of this invention are as follows:

[0034] This invention improves the open set detection algorithm Grounding DINO by introducing a loop prompting, result fusion and filtering module, as well as a category refinement module. At the same time, it integrates the defect detection information of the closed set detection algorithm and the open set detection algorithm by introducing a multi-algorithm source fusion module, forming a defect detection method for the inner wall of automotive camshaft mounting holes with strong generalization ability, thus improving the detection accuracy of defects in the inner wall of automotive camshaft mounting holes. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0036] Figure 1This is a structural diagram of a method for detecting defects in the inner wall of an automotive camshaft mounting hole, which has strong generalization capabilities according to the present invention.

[0037] Figure 2 This is a CS structure diagram.

[0038] Figure 3 This is a detailed structural diagram of the position decoding unit.

[0039] Figure 4 This diagram illustrates the ability of this invention to detect defects that appear in actual camshaft mounting holes. Detailed Implementation

[0040] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0041] This invention combines open-set and closed-set detection algorithms to detect defects on the inner wall of the camshaft mounting hole in an automotive engine. For the current image to be detected, I, it is first input into an improved open-set detection algorithm for defect detection. This improved open-set detection algorithm also requires text prompts; in this embodiment, the text prompts used are: "defect," "black spot," and "unsmooth area." The image to be detected, along with the text prompts, is input into the improved open-set detection algorithm for defect detection, and the defect location and category information (B) obtained by the improved open-set defect detection algorithm are obtained. OP C OP Next, the image to be detected is input into a closed-set detection algorithm for defect detection. In this embodiment, the closed-set detection algorithms selected are YOLO5 and the Faster R-CNN model with a joint FPN structure, and the defect information (B) detected by the closed-set detection algorithm is obtained. CL C CL Next, (B) OP C OP ) and (B CL C CL The algorithm fuses multiple algorithms to obtain the final defect detection information (B, C) corresponding to the input image, thus achieving the defect detection task for the input image. See the overall model structure diagram. Figure 1 .

[0042] Specifically, this invention proposes a method for detecting defects in the inner wall of automotive camshaft mounting holes with strong generalization ability. The method is as follows:

[0043] S1: Preprocess the current input image;

[0044] S2: Input the preprocessed image into the open set detection algorithm for defect detection, and obtain the defect information detected by the open set detection algorithm (B). OP C OP ), where B is the defect location and C is the corresponding defect category;

[0045] S3: Input the preprocessed image into the closed-set detection algorithm for defect detection, and obtain the defect information detected by the closed-set detection algorithm (B). CL C CL );

[0046] S4: The defect information (B) obtained through the open set detection algorithm... OP C OP ) and the defect detection information (B) obtained by the closed set detection algorithm CL C CL The input image is fused using a multi-algorithm source fusion module to obtain the final defect detection information (B, C).

[0047] The preprocessing of the input image in S1 specifically includes: image size resetting operation and image normalization processing;

[0048] The specific process of the image resizing operation is as follows: For any input image First, the size transformation factor α is obtained according to formula (1). Then, the transformed size (H) is obtained according to formula (2) based on the original size and the obtained size transformation factor. S W S Finally, the input image is resized according to the obtained transformed image size using bilinear interpolation.

[0049]

[0050]

[0051] The image normalization process is performed according to the normal distribution standardization process, as described in formula (3):

[0052]

[0053] Where x∈I represents any element in the input image, and μ and σ represent the mean and variance of the image, respectively. Therefore, the standardized image will highlight the unique attributes of the image itself and reduce the common attributes between images.

[0054] The open set defect detection algorithm in S2 consists of 5 steps, specifically:

[0055] S21: Use a looping text prompt method to provide prompts to the open set detection model (T1, T2, ..., T...). n );

[0056] S22: Input the prompt text sequentially into the open set detection module to obtain the defect detection information (B) obtained from the current prompt text. OP1 i C OP1 i );

[0057] S23: Merge the defect detection information obtained from multiple text prompts to obtain the merged defect information;

[0058] S24: The fused defect information is filtered through a filtering module to obtain (B) OP1 C OP1 );

[0059] S25: The filtered defect information is refined through the category refinement module to obtain the final defect information (B) obtained by the open set detection algorithm. OP C OP ).

[0060] The cyclic text prompting method in S21 refers to the following: for any input text prompt, the keywords are separated by a specific marker. When inputting, the model splits the text according to the marker, and then inputs the split keywords into the open set detection model in sequence to perform the defect detection process of corresponding keyword prompts.

[0061] The difference between the open set detection module and the closed set detection module in S22 is that the current detection algorithm introduces a language module. The model searches for the target in the input image based on linguistic text prompts. This open set detection module consists of five parts: a text feature extractor, an image feature extractor, a text-image feature fusion unit, a language-guided query unit, and a cross-modal decoding unit. The text feature extractor uses a BERT model based on the Transformer architecture, encoding the input language into corresponding feature vectors through bidirectional language representation. The image feature extractor uses a Swin model based on the Transformer architecture to extract features from the input image. The text-image feature fusion unit fuses text and image features by stacking a series of self-attention and cross-attention modules and a feedforward network. The language-guided query unit further guides the defect detection process using the input text. The cross-modal decoding unit decodes the obtained optimal matching pair to obtain the defect location corresponding to the final text.

[0062] The text feature extractor, for any input text, to achieve bidirectional language representation, is structurally composed of a series of stacked encoding modules. Each encoding module is linked to every input text element, thus achieving bidirectional representation. Each encoding module is constructed by stacking multi-head attention layers, layer normalization (LN), and feed forward network (FFN) layers to encode the input text. The encoding process can be summarized as follows:

[0063]

[0064] x2=LN[FFN(x1)+x1] (5)

[0065] Where x is the input text vector, p is the corresponding position encoding vector, and W MHA It is a learnable matrix. This is a vector concatenation operation. `x2` is the element-wise summation symbol for the vector. `x2` is the encoded feature output after encoding by the encoding module. `LN` and `FFN` refer to the multi-layer normalization operation and the forward propagation network, respectively. `att` is the attention mechanism function, as follows:

[0066]

[0067]

[0068] Here, `proj` is the mapping function, typically a linear mapping function chosen during deployment, which maps the input text vector into three different feature vectors `q`, `k`, and `v`, used for queries, keys, and values ​​in the subsequent attention mechanism, respectively. `softmmax` is the chosen normalization function, calculated as follows: δ is a scaling factor, which accelerates convergence and prevents gradient vanishing. Therefore, by repeatedly stacking the encoding modules composed of equations (4) to (7), the input text can be encoded.

[0069] The image feature extractor employs the Swin model based on the Transformer architecture to extract features from the input image. Specifically, the model consists of six parts: image feature pre-extraction, four-stage image feature encoding, and feature post-processing. The image feature pre-extraction stage involves performing a (convolution + LN) combination operation on the input image information to form preliminary features. These preliminary features are then input into four different feature encoding layers for low-level to high-level feature mapping. Each feature encoding layer consists of a certain number of encoding blocks and downsampling units. These encoding blocks can be further subdivided into six parts:

[0070] 1) Feature map scrolling operation: The feature map is shifted to the upper left, and the part that moves out of the feature map is filled into the corresponding lower right corner of the feature map.

[0071] 2) Feature map slicing operation: The input feature map is split into 12×12 pieces and stacked.

[0072] 3) Attention Module: The feature map obtained in step 2) is subjected to attention operations in the same manner as described in equations (4) to (7). The difference from equations (4) to (7) is that equation (6) forms feature matrices Q, K, and V. The size of the feature map does not change after passing through the current attention module.

[0073] 4) Feature map back-cut operation: The stacked 12×12 small feature maps are stitched back to their original size before 2).

[0074] 5) Feature map rollback operation: The large feature map after 4) rollback is shifted to the lower right. Similarly, the part that is moved out of the image is used to fill the upper left corner area.

[0075] 6) Feature post-mapping: The feature map after the rollback operation is post-mapped using a linear mapping layer, a linear activation unit, and a layer normalization combination module, and the features are fitted again.

[0076] Specifically, for the corresponding operations 1) and 5) mentioned above, due to the varying number of encoding blocks in each stage, the corresponding rolling and rollback operations are only performed on odd-numbered encoding blocks. The purpose of the rolling operation is to overcome the drawback of significantly increased computational complexity caused by long-distance modeling based on the traditional Transformer architecture. Therefore, after rolling, feature map slicing is combined to transform large feature maps into smaller feature maps, and attention operations are performed on the smaller feature maps to improve the module's computational efficiency. The size of the input feature map remains unchanged throughout the entire encoding block.

[0077] The downsampling unit performs a 2x downsampling of the input feature map. Specifically, it extracts features from the input feature map by even rows and even columns, even rows and odd columns, odd rows and even columns, and odd rows and odd columns, resulting in four downsampled feature maps: DS1, DS2, DS3, and DS4. These four downsampled feature maps are then concatenated to obtain the concatenated feature map. The concatenated DS is then subjected to a second feature mapping operation using a combination of (linear mapping layer + layer normalization) to obtain the final features obtained through the downsampling unit, thus yielding the encoded features of the input image. The specific structure of the image feature extractor in this embodiment is shown in Table 1.

[0078] Table 1

[0079]

[0080] The text-image feature fusion unit primarily achieves the fusion of text and image features by stacking a series of self-attention and cross-attention modules and a feedforward network. Specifically, the text-encoded features obtained from the BERT model are input into its corresponding attention module (SA-a) to further capture long-range interdependent features t in the text; the image-encoded features obtained from the Swin model are input into its corresponding deformable attention module (DSA-b) to further perform global modeling of the relationships between objects in the image, obtaining the corresponding extracted features F after modeling. D Next, t and F D The input is then fed back into the attention module (SA-c) for image-to-text space mapping to obtain the mapped features. Similarly, The input is fed into the attention module (SA-d) to perform a mapping from text space to image space, resulting in the mapped features. Finally, the mapped features are input into two different forward propagation networks (FFNs) to form their respective fused text and image features. e F e The attention module described herein is consistent with the above equations (4) to (7). The deformable attention module is proposed to accelerate model convergence. Specifically, it generates a set of points at equal intervals on the feature map to represent the reference point position of the sampling point. At the same time, it generates a set of learnable matrices P for the feature map to represent the offset formed at the current sampling point. The final position of the sampling point is the sum of the reference point position and the offset distance. It extracts the features corresponding to the sampling point position in the feature map and performs the attention operation process consistent with equations (4) to (7).

[0081] The language-guided query unit further uses the input text to guide the defect detection process. First, it processes the encoded text features t... e and image features F e Perform relevance matching, then for each matched text-image pair, retain the top m best matches based on the matching value, and retain the corresponding text features t. e：m and image features F e：m This enables a language-guided query mechanism.

[0082] The cross-modal decoding unit decodes the obtained optimal matching pair to obtain the defect location corresponding to the final text. Specifically, the cross-modal decoding unit is also structured by stacking a series of attention mechanisms, including one self-attention module, one image cross-attention module, one text cross-attention module, and a forward propagation network (FFN). Its specific implementation is consistent with equations (4) to (7), except that Q, K, and V in the self-attention module are initialized to the feature F corresponding to the above optimal match. e：m Therefore, it is called a self-attention module. In the image cross-attention module, Q is initialized to the output of the current self-attention module, and K and V are initialized to F. e In the text cross-attention module, Q is initialized to the output of the image cross-attention module, and K and V are initialized to t. e Finally, the output of the text cross-attention module is passed through a forward propagation network to decode the decoded features into corresponding positional coordinates and corresponding text prompts (B). OP1 i C OP1 i ).

[0083] The defect detection information fusion in S23 specifically refers to retaining all the defect location information obtained under each keyword in the loop prompt, thereby obtaining a comprehensive defect location information.

[0084] The filtering module in S24 addresses the issue that text prompts sometimes contain similar meanings and ambiguities, leading to overlapping information in the fused defect information. To solve this problem, the present invention uses a non-maximum suppression algorithm in S24 to filter overlapping position information. Specifically, all currently fused defect detection boxes are arranged in descending order of confidence level to obtain a list. Next, select the box with the highest confidence level. As the current defect baseline frame, from Starting from the second position, retrieve the corresponding defect location information and the current defect baseline frame sequentially. Calculate the positional overlap; where, positional overlap It is obtained by solving the Intersection over Union (IOU) between two defect boxes, and the obtained positional overlap is then calculated. Compare with a given threshold ζ, when When the value is high, it indicates that there is a high similarity between the current defect box and the current defect reference box, so it is placed in the reference box. Remove from the middle, and iterate through to the next middle in the same way. The last defect location in the middle; then from Starting from the second defect position and using it as the current defect reference frame, perform the same operation as above until the defect reference frame is traversed to... Up to the last position, the filtered defect location information (B) will be obtained. OP1 C OP1 ).

[0085] The category refinement module in S25 specifically corrects the category information corresponding to the defect box. Since the current defect locations are all obtained based on text prompts, each defect location is also categorized according to the text prompts. Due to the specificity of engine cylinder bore inner wall defects, directly using open-set detection algorithms based on defect names yields poor detection results. Therefore, this invention uses common text prompts, such as "defect," "unsmooth area," and "black spot." To further clarify the specific category of defects contained within the area, this invention performs defect category refinement. Specifically, a deep learning-based network model is used to determine the defect category of the contained area. The specific operation steps of the category refinement module in S25 are as follows:

[0086] S251: The (B) obtained from S24 OP1 C OP1 All defect locations are extracted from the original image area using ROI extraction units, resulting in all corresponding extracted regions I. OP1 ;

[0087] S252: Extract all regions I obtained through S251 OP1 The dimensions were readjusted to 160×160, resulting in the adjusted size.

[0088] S253: Input the extracted region adjusted in S252 into the category determination network to perform the defect category determination task for the corresponding region, and obtain the final complete defect information (B) after defect determination. OP C OP ).

[0089] In S252, the size readjustment is completed based on bilinear interpolation. In S253, the defect category determination is accomplished using a neural network model. For classification accuracy, the specific category determination model maintains the same structure as the aforementioned image feature extractor. The difference lies in that the image feature extractor introduces a feature mapping layer (pooling operation + linear mapping layer) at the end of the model to convert image feature information into the required defect category information, thereby achieving the defect category determination task. The defect category determination model in this embodiment is shown in Table 2. It can be seen that the defect category determination model and the image feature extractor differ only in the last feature mapping layer. The image feature extractor outputs an image feature matrix, the size of which is 64 times the downsampled value of the input image. The final output of the defect category determination model is a feature vector, the length of which is the number of defect categories.

[0090] Table 2

[0091]

[0092] The closed-set detection algorithm in S3 refers to a pure defect detection algorithm based on supervised learning. This type of algorithm does not include a language model, and therefore can only detect defect types trained under supervised conditions. The purpose of using such a closed-set defect detection algorithm in this invention is to ensure the detection capability for existing defect categories (using the open-set detection algorithm in S2 improves the model's ability to detect unknown defect categories, i.e., improves the model's generalization ability). This embodiment uses two closed-set detection algorithms: a two-stage high-precision closed-set detection algorithm, Faster R-CNN, and a single-stage high-speed closed-set detection algorithm, YOLO5 (small) model. The Faster R-CNN closed-set detection model in this embodiment is an integrated Feature Pyramid Network (FPN) model, thus overcoming the detection problem of small-sized defects. Specific parameters of the Faster R-CNN+FPN closed-set detection model in this embodiment are shown in Table 3.

[0093] Table 3

[0094]

[0095] The YOLO5 (small) model structure parameters are shown in Table 4. CBS refers to the combination of convolution, batch normalization, and SiLU activation function; MaxP refers to max pooling; and CAT refers to feature map concatenation. For the detailed structure of CS, please refer to [link to relevant documentation]. Figure 2 For details on the location decoding unit, please refer to [link / reference]. Figure 3 .

[0096] Table 4

[0097]

[0098]

[0099] The fusion operation in S4 further fuses the defect information obtained from the improved open-set detection algorithm and the closed-set detection algorithm, eliminating redundant defect coordinate boxes. The fusion idea here is to pair defect information from different sources (improved open-set detection algorithm and closed-set detection algorithm). For successfully paired defect information pairs, only the one with the highest confidence is retained. Specifically, the fusion algorithm used is the Hungarian matching algorithm. This invention uses the Generalized Intersection over Union (GIOU) value as the quantitative standard for the similarity between defect information. This value not only focuses on the overlapping areas but also on other non-overlapping areas, better reflecting the degree of overlap between input defect information, while being insensitive to size changes. The GIOU distance is calculated as follows:

[0100]

[0101] Here, A and B represent defect information (location + category) from the open set detection algorithm and the closed set detection algorithm, respectively; C represents the smallest bounding rectangle containing A and B; and |●| represents calculating the area. After fusing multiple algorithm sources, the final defect detection information (B, C) corresponding to the input image is obtained.

[0102] Example

[0103] This embodiment is specifically carried out according to the following steps:

[0104] 1. Data preparation stage

[0105] The dataset used in this embodiment was acquired by a data acquisition system. A large number of sample images were collected by adjusting the depth of field and exposure time, and were meticulously labeled. The data covers more than two hundred different engine camshaft mounting hole models, comprehensively encompassing existing engine camshaft mounting hole models. The data samples were expanded using appropriate data augmentation techniques. Since this invention includes a defect detection (boundary box) training part and a defect category refinement part, the dataset is also divided into a defect detection dataset and a defect category determination dataset.

[0106] (1) For the defect detection dataset:

[0107] The training dataset uses HIT-EngD, and details about this dataset are as follows:

[0108] 1) The total data volume is 8800 images, with 8175 defective samples and 625 non-defective samples.

[0109] 2) Data resolution 1000 x 1000 (1 million pixels);

[0110] 3) A total of 61,736 defects were marked, including 30,845 cracks, 23,146 pinholes, and 545 dents, which are classified into 3 categories;

[0111] 4) Includes common and challenging scenarios, such as the detection of extremely small defects like pinholes at high resolution, the problem of dense defects, and the problem of defects becoming blurred due to changes in illumination.

[0112] Therefore, conducting related experiments on the HIT-EngD dataset is challenging. This embodiment also selects 7040 sample images for training and 1760 sample images for testing. During training and testing, each image is resized to 640×640.

[0113] (2) For the defect category determination dataset:

[0114] This dataset, the latest collected and labeled dataset of this invention, is used for defect category localization and is named HIT-EngDC. Specific details are as follows:

[0115] 1) The total size of the dataset is 3406, containing 5 types of defect images, including 179 images of bumps, 1281 images of scratches, 427 images of black oil stains, 94 images of white bright spots, and 1425 images of pinholes.

[0116] 2) The sample resolution is low, the image size is small, and the range of variation is [12, 600];

[0117] 3) The sample covers a wide range, with large size variations, and the defect categories in the engine camshaft mounting holes are diverse.

[0118] Therefore, implementing HIT-EngDC is also challenging. Furthermore, the upgraded version of the current defect detection dataset (bounding boxes) adds engine cylinder bore models and generates new defect categories. The details of the differences between the newly added sample data and the previously released HIT-EngD are summarized below:

[0119] 1) The newly upgraded dataset has added new defect categories, including "black area" defects, "white area" defects, and "oil stain" defects;

[0120] 2) Newly collected images containing new defect categories are directly used as test samples without labeling.

[0121] Therefore, due to the differences between the training and test set categories, testing on the upgraded HIT-EngD dataset will be more challenging, and the generalization performance of the model must be taken into account.

[0122] 2. Training Phase

[0123] Since the algorithm of this invention also includes the closed set detection algorithms Faster RCNN+FPN and YOLO5 (small) to solve the problem of detecting defects of known categories, the algorithm of this invention also needs to train the selected closed set detection algorithm. The training process of the training network is repeated in steps b1 and b2.

[0124] b1: Initialize network training parameters and hyperparameters.

[0125] 1) For the closed set detection algorithm Faster R-CNN+FPN, this embodiment sets the maximum number of iterations to 100; the batch training size to 32; and the learning rate to 2×10 to achieve faster convergence during training. -4 And at 75 cycles, the learning rate is decayed to 2×10. -5 The Adam algorithm is introduced as the optimizer, with a weight decay coefficient of 0.0001 and a momentum factor of 0.9. All network parameters are initialized according to the Xavier method. The loss function is cross-entropy loss for classification problems and Smooth-L1 loss for coordinate regression problems.

[0126] 2) For the closed set detection algorithm YOLOv5 (small), this embodiment sets the maximum number of iterations to 100; the batch training size is set to 32; and the learning rate is set to 2×10 to achieve faster convergence during training. -4 And at 75 cycles, the learning rate is decayed to 2×10. -5 The Adam algorithm is introduced as the optimizer, with a weight decay coefficient of 0.0001 and a momentum factor of 0.9. All network parameters are initialized according to the Xavier method. The loss function is cross-entropy loss for classification problems and CIOU-Loss loss for coordinate regression problems.

[0127] 3) For the category refinement module, this embodiment sets the maximum number of iterations to 100; the batch training size to 128; and the learning rate to 1×10 to achieve faster convergence during training. -2 The SGD algorithm is introduced as the optimizer, with a weight decay coefficient of 1×10⁻⁶. -4A momentum factor of 0.9 was introduced; all network parameters were initialized according to the Xavier method; and the cross-entropy loss function was chosen. Furthermore, to investigate the class determination capability of existing algorithms on the HIT-EngDC dataset, this embodiment also used three other models known to those skilled in the art: MobileNetV3, ResNet18, and ViT-B. Model details are not elaborated here. The training of this model followed the same method as the class refinement module.

[0128] b2: Using the image obtained in step S1 as input to the training network, start training the closed set detection algorithm Fasser R-CNN+FPN, YOLO5 (small), and the category refinement module respectively. The training is an iterative process. In each iteration, the loss value of the forward propagation is calculated first, and then all parameters of the network are updated in reverse with the goal of minimizing the loss value.

[0129] 3. Testing Phase

[0130] Repeat steps I through II for the testing phase.

[0131] Step I: Repeat step S1 for the test image;

[0132] Step II: Input the test image frame obtained in Step I into the training network trained in Steps b1 and b2, and output the defect detection result of the image to be detected.

[0133] (1) The results of the category refinement experiment are shown in Table 5. The index used is classification accuracy, which is well known to those skilled in the art, and will not be elaborated upon in this embodiment of the invention. As can be seen from Table 5, the model based on the Transformer framework used in this invention has a better ability to solve the current defect category determination problem, achieving a determination accuracy of 94.44%.

[0134] Table 5

[0135]

[0136] (2) The defect detection method with strong generalization ability proposed in this invention has the following overall ability to detect defects appearing in engine camshaft mounting holes (see [reference]). Figure 4 The first column shows the detection output of the Faster R-CNN+FPN model, the second column shows the output of the YOLO5 (small) model, the third column shows the output of the improved closed-set detection algorithm DGDINO model, and the last column shows the output of the final defect detection method proposed in this invention. Meanwhile, the problem region is... Figure 4 The image is highlighted within a bright blue dashed border. (See attached image.) Figure 4 visible:

[0137] 1) The detection method proposed in this invention has a superior overall detection capability compared to any other high-performance defect detection method;

[0138] 2) See Figure 4 The new defect types that appear in (1), (2), (6), (7), and (8), namely the defect types that do not exist in the training data of the closed set model ("black area" and "white area"), show relatively poor detection capabilities in the closed set detection algorithms Faster RCNN+FPN and YOLO5 (small) model. However, the improved open set detection algorithm (untrained) shows relatively ideal detection capabilities, resulting in the algorithm proposed in this invention having a strong defect generalization ability.

[0139] 3) See Figure 4 (4), (5), (6) The improved power-on detection algorithm of the present invention (third column) can further detect and compensate for the omission problem of the closed set detection algorithm for known categories.

[0140] Meanwhile, the algorithm proposed in this invention has greater advantages in defect detection, providing a new algorithm option for defect detection tasks.

[0141] This invention proposes a method for detecting defects on the inner wall of automotive camshaft mounting holes with strong generalization ability. All images to be detected are preprocessed. The preprocessed images are first input into an improved open-set detection algorithm to detect all defects in the engine camshaft mounting holes, obtaining the defect location information detected by the open-set detection algorithm. The preprocessed images are then further input into a selected closed-set detection algorithm to detect defects of known categories in the engine camshaft mounting holes, obtaining the corresponding defect location information. The defect location information obtained from the open-set and closed-set detection algorithms is fused to obtain the final defect detection result corresponding to the input image to be detected. By jointly using the improved open-set and closed-set detection algorithms, this invention overcomes the inadequacy of conventional target detection algorithms for detecting defects on the inner wall of automotive engine camshaft mounting holes, while improving the generalization ability of the model, demonstrating excellent detection capability for defects of any category on the inner wall of automotive engine camshaft mounting holes.

[0142] This invention proposes an electronic device, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the method for detecting defects in the inner wall of an automotive camshaft mounting hole with strong generalization ability.

[0143] This invention proposes a computer-readable storage medium for storing computer instructions, which, when executed by a processor, implement the steps of a method for detecting defects in the inner wall of an automotive camshaft mounting hole with strong generalization capabilities.

[0144] The memory in this application embodiment can be volatile memory or non-volatile memory, or it can include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DRRAM). It should be noted that the memory used in the methods described in this invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0145] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., high-density digital video discs (DVDs)), or semiconductor media (e.g., solid-state disks (SSDs)).

[0146] In implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software. The steps of the method disclosed in the embodiments of this application can be directly implemented by a hardware processor, or by a combination of hardware and software modules in the processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, detailed descriptions are omitted here.

[0147] It should be noted that the processor in the embodiments of this application can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method embodiments can be completed by the integrated logic circuitry in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above methods.

[0148] The above provides a detailed description of a method for detecting defects in the inner wall of automotive camshaft mounting holes with strong generalization ability. Specific examples have been used to illustrate the principle and implementation of the invention. The description of the above embodiments is only for the purpose of helping to understand the method and core idea of ​​the invention. At the same time, for those skilled in the art, there will be changes in the specific implementation and application scope based on the idea of ​​the invention. Therefore, the content of this specification should not be construed as a limitation of the invention.

Claims

1. A method for detecting defects in the inner wall of an automotive camshaft mounting hole with strong generalization ability, characterized in that, The method is specifically as follows: S1: Preprocess the current input image; S2: Input the preprocessed image into the open set detection algorithm for defect detection, and obtain the defect information detected by the open set detection algorithm (B). OP C OP (), where B is the defect location and C is the corresponding defect category; S3: Input the preprocessed image into the closed-set detection algorithm for defect detection, and obtain the defect information detected by the closed-set detection algorithm (B). CL C CL ); S4: The defect information (B) obtained through the open set detection algorithm... OP C OP ) and the defect detection information (B) obtained by the closed set detection algorithm CL C CL The input image is fused using a multi-algorithm source fusion module to obtain the final defect detection information (B, C). The open set defect detection algorithm in S2 consists of 5 steps, specifically: S21: Use a looping text prompt method to provide prompts to the open set detection model (T1, T2, ..., T...). n ); S22: Input the prompt text sequentially into the open set detection module to obtain the defect detection information (B) obtained from the current prompt text. OP1 i C OP1 i ); S23: Merge the defect detection information obtained from multiple text prompts to obtain the merged defect information; S24: The fused defect information is filtered through a filtering module to obtain (B) OP1 C OP1 ); S25: The filtered defect information is refined through the category refinement module to obtain the final defect information obtained by the open set detection algorithm (B). OP C OP ).

2. The method according to claim 1, characterized in that, The preprocessing of the input image in S1 specifically includes: image size resetting operation and image normalization processing; The specific process of the image resizing operation is as follows: For any input image I ∈ H×W×3 First, the size transformation factor is obtained according to formula (1). α Then, based on the original drawing dimensions and the obtained size transformation factor, the transformed drawing dimensions are obtained according to formula (2). H S , W S Finally, the input image is resized according to the obtained transformed image size using bilinear interpolation. （1） （2） The image normalization process is performed according to the normal distribution standardization process, as described in formula (3): （3） in, x ∈ I , where is an element at any position in the input image. μ and σ These are the mean and variance of the image, respectively; therefore, the standardized image will highlight the unique attributes of the image itself and reduce the common attributes between images.

3. The method according to claim 1, characterized in that, The cyclic text prompting method in S21 refers to the following: for any input text prompt, the keywords are separated by a specific marker. When inputting, the model splits the text according to the marker, and then inputs the split keywords into the open set detection model in sequence to perform the defect detection process of corresponding keyword prompts.

4. The method according to claim 1, characterized in that, The open set detection module consists of five parts: a text feature extractor, an image feature extractor, a text-image feature fusion unit, a language-guided query unit, and a cross-modal decoding unit. The text feature extractor uses a BERT model based on the Transformer architecture to encode the input language into corresponding feature vectors through bidirectional language representation. The image feature extractor uses a Swin model based on the Transformer architecture to extract features from the input image. The text-image feature fusion unit fuses text and image features by stacking a series of self-attention and cross-attention modules and a feedforward network. The language-guided query unit further guides the defect detection process using the input text. The cross-modal decoding unit decodes the obtained optimal matching pair to obtain the defect location corresponding to the final text.

5. The method according to claim 1, characterized in that, The defect detection information fusion in S23 specifically refers to retaining all the defect location information obtained under each keyword in the loop prompt, thereby obtaining a comprehensive defect location information. (B) OP1 i C OP1 i ).

6. The method according to claim 1, characterized in that, In S24, a non-maximum suppression algorithm is used to filter overlapping position information; specifically, all the currently fused defect detection boxes are arranged in descending order of confidence to obtain a list. =[(B OP1 max C OP1 max ), ... , (B OP1 min C OP1 min Then, the box with the highest confidence level (B) is selected. OP1 max C OP1 max ) as the current defect baseline frame, from Starting from the second position, retrieve the corresponding defect location information and the current defect baseline (B) sequentially. OP1 max C OP1 max ) Calculate the positional overlap; where, positional overlap It is obtained by solving the Intersection over Union (IOU) between two defect boxes, and the obtained positional overlap is then calculated. Compare with a given threshold ζ, when When the value is greater than ζ, it indicates a high similarity between the current defect box and the current defect reference box, therefore it is placed in... Remove from the middle, and iterate through to the next middle in the same way. The last defect location in the middle; then from Starting from the second defect position and using it as the current defect reference frame, perform the same operation as above until the defect reference frame is traversed to... Up to the last position, the filtered defect location information (B) will be obtained. OP1 C OP1 ).

7. The method according to claim 1, characterized in that, The specific operation steps of the category refinement module in S25 are as follows: S251: The (B) obtained from S24 OP1 C OP1 All defect locations are extracted from the original image area using ROI extraction units, resulting in all corresponding extracted regions I. OP1 ; S252: Extract all regions I obtained through S251 OP1 The dimensions were readjusted to 160×160, resulting in the adjusted I. OP1 r ; S253: Input the extracted region adjusted in S252 into the category determination network to perform the defect category determination task for the corresponding region, and obtain the final complete defect information (B) after defect determination. OP C OP ).

8. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1-7.

9. A computer-readable storage medium for storing computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the steps of the method according to any one of claims 1-7.

Images