Defect detection method and device, equipment, storage medium and program product

Abstract

The invention relates to the technical field of defect detection, and particularly provides a defect detection method and device, equipment, a storage medium and a program product. The defect detection method comprises the following steps: acquiring a to-be-detected gear surface image and a pre-constructed gear defect detection model; the gear defect detection model comprises a feature extraction network and a feature segmentation network, and the feature extraction network comprises an image block division module, a backbone network formed by a feature extraction layer and a feature extraction upsampling layer, and an auxiliary branch network; and determining a defect detection result of the to-be-detected gear surface image by using the gear defect detection model. The method can introduce the auxiliary branch network to obtain the auxiliary branch multi-channel feature map with the shallow feature under the condition that the trunk network is used to obtain the trunk multi-channel feature map with the deep feature, optimizes the learning ability for the tiny target defect, solves the problem that the back propagation cannot be transmitted to the shallow, and improves the detection accuracy. And the detection efficiency and the detection precision of tiny target defects are improved.

Description

Technical Field

[0001] The present disclosure relates to the field of defect detection technology, and in particular to a defect detection method, apparatus, device, storage medium, and program product. Background Art

[0002] In recent years, the development of new energy sources, particularly wind power, has yielded significant results, with installed capacity steadily increasing. This rapid growth has also brought with it numerous challenges, one of which is the detection of defects in wind turbine gearboxes. Failures directly impact wind turbine power generation and operational maintenance costs.

[0003] In related technologies, the defect type is usually determined by manually collecting images of the inside of the wind turbine gearbox and then manually screening them. This method requires a lot of manpower, has low work efficiency, and has low detection accuracy. Summary of the Invention

[0004] In view of the above problems, the present disclosure is proposed. The present disclosure provides a defect detection method, apparatus, device, storage medium and program product.

[0005] According to one aspect of the present disclosure, a defect detection method is provided, comprising:

[0006] Obtaining a surface image of the gear to be tested and a pre-built gear defect detection model; wherein the gear defect detection model includes a feature extraction network and a feature segmentation network, wherein the feature extraction network includes an image block partitioning module, a backbone network consisting of a feature extraction layer and a feature extraction upsampling layer, and an auxiliary branch network;

[0007] Using the image block division module, the surface image of the gear to be tested is divided into a plurality of local image blocks; using the feature extraction layer and the feature extraction upsampling layer, a trunk multi-channel feature map is extracted from the plurality of local image blocks; using the auxiliary branch network, an auxiliary branch multi-channel feature map is extracted from the plurality of local image blocks; wherein the dimension of the auxiliary branch multi-channel feature map is lower than the dimension of the trunk multi-channel feature map;

[0008] The feature segmentation network is used to process the main multi-channel feature map and the auxiliary branch multi-channel feature map to obtain a defect detection result of the surface image of the gear to be tested.

[0009] According to another aspect of the present disclosure, there is provided a defect detection device, comprising:

[0010] An acquisition module, configured to acquire an image of the gear surface to be tested and a pre-built gear defect detection model; wherein the gear defect detection model comprises a feature extraction network and a feature segmentation network, wherein the feature extraction network comprises a backbone network consisting of an image block partitioning module, a feature extraction layer, and a feature extraction upsampling layer, and an auxiliary branch network;

[0011] a processing module configured to divide the surface image of the gear to be tested into a plurality of local image blocks using the image block division module; extract a main multi-channel feature map from the plurality of local image blocks using the feature extraction layer and the feature extraction upsampling layer; and extract an auxiliary branch multi-channel feature map from the plurality of local image blocks using the auxiliary branch network; wherein the dimension of the auxiliary branch multi-channel feature map is lower than the dimension of the main multi-channel feature map;

[0012] The processing module is also used to use the feature segmentation network to process the main multi-channel feature map and the auxiliary branch multi-channel feature map to obtain a defect detection result of the surface image of the gear to be tested.

[0013] According to another aspect of the exemplary embodiments of the present disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the method described in the exemplary embodiments of the present disclosure.

[0014] According to another aspect of the exemplary embodiments of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method described in the exemplary embodiments of the present disclosure is implemented.

[0015] According to another aspect of the exemplary embodiments of the present disclosure, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the method described in the exemplary embodiments of the present disclosure is implemented.

[0016] As will be described in detail below, according to the defect detection method, apparatus, device, storage medium, and program product of the embodiments of the present disclosure, an image of the gear surface to be tested and a pre-built gear defect detection model are obtained; wherein the gear defect detection model includes a feature extraction network and a feature segmentation network, and the feature extraction network includes an image block division module, a feature extraction layer, and a feature extraction upsampling layer, which constitute a backbone network, and an auxiliary branch network; using the image block division module, the surface image of the gear to be tested is divided into multiple local image blocks; using the feature extraction layer and the feature extraction upsampling layer, a backbone multi-channel feature map is extracted from the multiple local image blocks; using the auxiliary branch network, an auxiliary branch multi-channel feature map is extracted from the multiple local image blocks; Among them, the dimension of the auxiliary branch multi-channel feature map is lower than that of the main multi-channel feature map; the main multi-channel feature map and the auxiliary branch multi-channel feature map are processed by the feature segmentation network to obtain the defect detection result of the surface image of the gear to be tested. On the premise of using the main network to obtain the main multi-channel feature map with deep features, the auxiliary branch network is introduced to obtain the auxiliary branch multi-channel feature map with shallow features, thereby optimizing the learning ability of small target defects and solving the problem that back propagation cannot be transmitted to the shallow layer; and the feature segmentation network is used to determine the defect detection result of the surface image of the gear to be tested based on the main multi-channel feature map and the auxiliary branch multi-channel feature map, thereby improving the detection efficiency and detection accuracy of small target defects. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The above and other purposes, features, and advantages of the present disclosure will become more apparent through a more detailed description of the embodiments of the present disclosure in conjunction with the accompanying drawings. The accompanying drawings are intended to provide a further understanding of the embodiments of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and are not intended to limit the present disclosure. In the drawings, the same reference numerals generally represent the same components or steps.

[0018] Figure 1 The model architecture diagram of the pyramid scene parsing network in the related art is shown;

[0019] Figure 2 A model architecture diagram of a gear defect detection model provided by an embodiment of the present disclosure is shown;

[0020] Figure 3 A complete architecture block diagram of a gear defect detection model provided by an embodiment of the present disclosure is shown;

[0021] Figure 4 A model architecture diagram of a feature extraction network introducing auxiliary branches provided by an embodiment of the present disclosure is shown;

[0022] Figure 5 A schematic diagram of a defect detection method provided by an exemplary embodiment of the present disclosure is shown;

[0023] Figure 6 A schematic structural diagram of a defect detection device provided by an exemplary embodiment of the present disclosure is shown;

[0024] Figure 7 A schematic structural diagram of an electronic device provided by an exemplary embodiment of the present disclosure is shown;

[0025] Figure 8 A schematic diagram of the structure of a computer system provided by an exemplary embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solutions and advantages of the present disclosure more apparent, the following will describe in detail exemplary embodiments of the present disclosure with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described herein.

[0027] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0028] The term "including" and its variations used in this document are open inclusions, that is, "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the description below. It should be noted that the concepts of "first", "second", etc. mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0029] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0030] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0031] While gearbox technology is relatively mature, wind turbines are often installed in harsh environments such as remote mountains, deserts, and isolated islands. Factors such as large temperature swings and frequent unit starts and stops mean their service life falls far short of their theoretical design lifespan of 20 years. The average downtime for maintenance due to failures per wind turbine is 145 hours per year, resulting in significant economic losses. Gear failure is the primary cause of equipment downtime. Therefore, regular inspections of wind turbine gear health and performance are necessary. Parts should be replaced for severe failures, while moderate failures should be closely monitored to ensure accurate guidance before a failure occurs, thereby improving economic efficiency.

[0032] With the development of image processing technology, the use of deep learning methods to detect gear defects has become an important topic. Compared with traditional manual inspection methods, deep learning methods can significantly reduce manual labor intensity, improve work efficiency, and accurately detect gear defects, thereby improving the stability of wind turbine operation.

[0033] Research and statistics show that 96% of wind turbine gearbox failures originate in gears and their bearings, with gear failure accounting for as much as 60% of all gearbox failures. Gear failures primarily occur in the teeth, including tooth corrosion, pitting, tooth bonding and abrasion, and tooth breakage. Wind turbine gear failures are classified as follows: tooth wear accounts for 10%, tooth pitting accounts for 31%, tooth bonding and abrasion accounts for 10%, tooth breakage accounts for 41%, and other failures account for 8%. Therefore, to reduce the gear failure rate, a series of preventive, early warning, and repair measures, tailored to the severity of the failure, are needed to eliminate defects and reduce wind farm operating costs and energy losses.

[0034] Based on the needs of industrial sites, defect detection in wind turbine gearboxes has evolved through three stages: identification and classification, location, and segmentation. These three stages are mutually inclusive and can be transformed into each other. In recent years, several detection solutions have been developed for gear defect detection.

[0035] Some approaches can detect missing gear parts by performing denoising and segmentation preprocessing on the gear image, then using morphological methods to analyze the dimensions of each tooth. While this method is fast, it can only detect a limited number of defect types and has poor generalization capabilities.

[0036] In some solutions, images can be captured with a linear array camera, and image segmentation technology and support vector machine methods can be used to detect micro-gear defects. However, in the face of insufficient lighting or complex backgrounds, there are problems with low detection accuracy and poor robustness.

[0037] In some approaches, the gear origin can be located using the least-squares circle fitting method. A ring filter is then constructed to extract an image of the gear region of interest, and a threshold is applied to the grayscale values of the connected domain to determine gear defects. Alternatively, a standard gear and the gear to be inspected can be registered using key points. The registered images are then differentially processed to extract the defective regions. Finally, the Otsu method is used for threshold segmentation, with connected domains with area values below the threshold identified as defective gears. Alternatively, a modified Mask R-CNN network can be used to detect minor gear defects. This approach uses ResNet-101 as the feature extraction network and removes the 3×3 convolutions from the feature pyramid network to improve the detection rate of missing teeth. Finally, by setting an appropriate anchor box size and aspect ratio, detection accuracy is improved, enabling detection of minor gear defects anywhere within the acquisition range. However, these detection methods generally suffer from low detection accuracy.

[0038] Therefore, in order to solve the above problems, the embodiment of the present disclosure provides a defect detection method. After obtaining the surface image of the gear to be tested, the original method of manually screening pictures and determining defects is abandoned, and a pre-built gear defect detection model is used to detect defects in the wind turbine gearbox gear, thereby improving the work efficiency and detection accuracy of defect detection.

[0039] The defect detection method provided by the embodiments of the present disclosure may be executed by a terminal or by a chip applied to the terminal.

[0040] Exemplarily, the above-mentioned terminals may include one or more of mobile phones, tablet computers, wearable devices, vehicle-mounted devices, laptop computers, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), and wearable devices based on augmented reality (AR) and / or virtual reality (VR) technologies, and may also include but not be limited to smart terminals such as remote control devices, wearable devices, street lights, and home appliances. The exemplary embodiments of the present disclosure do not impose specific restrictions on this.

[0041] The first step is to collect image data: an industrial endoscope is used to capture multiple raw gear surface images of the wind turbine gearbox to obtain a raw dataset. Here, the raw gear surface images are in JPG format.

[0042] The second step is to construct an augmented dataset. This involves performing random processing operations on each original gear surface image in the original dataset to generate an augmented dataset. The augmented dataset may include multiple gear surface images. The random processing operations may include at least one of flipping, rotating, cropping, and translating.

[0043] For example, Labelme can be used to perform category labeling on the expanded dataset to obtain a category labeling dataset. The labels of the category labeling dataset are in the form of a json file, with the original image folder named img and the labeling folder named json.

[0044] Here, the defect categories can be divided into four types: rust, bonding, eccentric loading and cracks, which are numbered 1, 2, 3 and 4 respectively, and the image background is numbered 0.

[0045] After the category labeling is completed, the expanded dataset is converted into a format that can be used by the segmentation network. That is, the label file is converted into an 8-bit color image to obtain the defect dataset. The converted label file (i.e., the defect dataset) is uniformly stored in the segpic folder.

[0046] During the model training process, the defect dataset is divided into test set, validation set, and test set at a ratio of test set: validation set: test set = 7:2:1.

[0047] The model training process can be set to: batch_size (batch size) = 8, epochs (iterations) = 300, lr (learning rate) = 0.0001, class (number of categories, including four defect categories and image background) = 5, bias (bias) = False (this setting can reduce the number of parameters, avoid redundancy, and enhance the generalization ability of the model). After training is completed, save the model and name it swin-psp.pth.

[0048] The third step is segmentation model improvement: Figure 1 The model architecture diagram of the pyramid scene parsing network in the related art is shown. Figure 2 FIG. 1 shows a model architecture diagram of a gear defect detection model provided by an embodiment of the present disclosure. Figure 1 As shown in , the feature extraction network in the Pyramid Scene Parsing Network (PSPNet) model is a Convolutional Neural Network (CNN); Figure 2 As shown, the present disclosure will be implemented as Figure 1 The CNN shown is replaced by a modified version of Swin-Transformer that introduces an auxiliary branch as the feature extraction network, resulting in an improved PSPNet model.

[0049] Step 4: Model construction: Use the expanded data set in the second step to train the improved PSPNet model to complete the construction of the gear defect detection model.

[0050] Step 5: Defect detection: Collect the surface image of the gear to be tested, and input the surface image of the gear to be tested into the gear defect detection model for defect detection.

[0051] Figure 3 The complete architecture block diagram of the gear defect detection model provided by the embodiment of the present disclosure is shown in FIG. Figure 3 As shown in Figure 3, the gear defect detection model includes a feature extraction network and a feature segmentation network.

[0052] During the training stage, the loss of the gear defect detection model includes feature extraction loss and feature segmentation loss. The weight of the feature extraction loss is greater than the weight of the feature segmentation loss, and the ratio between the weight of the feature extraction loss and the weight of the feature segmentation loss is 7:3 to 8:2.

[0053] For example, Figure 3 As shown, the feature extraction network can include a backbone network and an auxiliary branch network.

[0054] Figure 4 The model architecture diagram of the feature extraction network with auxiliary branches provided by the embodiment of the present disclosure is shown as follows: Figure 4 As shown, the backbone network can use Swin Transformer, the size of the backbone network input is 473×473×3, and the output is the number of the defect category.

[0055] The backbone network of the feature extraction network can include an image block partitioning module, multiple sequentially connected feature extraction layers, and feature extraction upsampling layers. Each feature extraction layer includes a shifted window self-attention layer (Swin TransformerBlock). Here, multiple sequentially connected feature extraction layers can be used to construct four stages, each of which is a similar repeating unit.

[0056] 1) First, the input gear surface image to be tested is divided into a set of non-overlapping patches (local image blocks) using an image block partitioning module (such as patch partition). The size of each patch is 4×4, so the dimensions of each patch are 4×4×3=48, and the number of patch blocks is 119×119.

[0057] 2) Stage 1 (the first sub-feature extraction layer) first uses a linear embedding layer to convert the feature dimension of the input local image patch into C (C = 96), and then inputs the Swin Transformer Block (i.e., the shifted window self-attention layer). Here, it can be seen that the first sub-feature extraction layer can include a linear embedding layer and a shifted window self-attention layer.

[0058] 3) The process from stage 2 to stage 4 (the second to fourth sub-feature extraction layers) is consistent. With each stage, the number of local image patches is halved, and the feature dimension is doubled. The final output feature map serves as the input to the segmentation network. The second, third, and fourth sub-feature extraction layers can all include a patch embedding layer (Patch Merging) and a shifted window self-attention layer.

[0059] Exemplarily, for the second sub-feature extraction layer, the third sub-feature extraction layer, and the fourth sub-feature extraction layer, the image block embedding layer performs a dimensionality-increasing convolution operation on the input multiple local image blocks to obtain a reduced number of high-dimensional local image blocks.

[0060] The shifted window self-attention layer extracts features from multiple high-dimensional local image blocks to obtain high-dimensional feature maps.

[0061] The size of the gear surface image to be tested is 473×473×3, the size of the multiple local image blocks obtained after division by the image block division module is 119×119×48, the size of the high-dimensional feature map obtained after processing by the first sub-feature extraction layer is 119×119×96, the size of the high-dimensional feature map obtained after processing by the second sub-feature extraction layer is 60×60×192, the size of the high-dimensional feature map obtained after processing by the third sub-feature extraction layer is 30×30×384, and the size of the high-dimensional feature map obtained after processing by the fourth sub-feature extraction layer is 15×15×768.

[0062] The feature extraction upsampling layer performs an upsampling operation on the high-dimensional feature map obtained after processing by the fourth sub-feature extraction layer to obtain a backbone multi-channel feature map.

[0063] It can be seen that after the first to fourth sub-feature extraction layers, multiple hierarchical feature maps can be obtained. The size of the multiple hierarchical feature maps is gradually reduced, which can reduce the computational complexity of the shifted window self-attention layer and achieve a linear relationship between computational complexity and image size. At the same time, by introducing a self-attention mechanism with a sliding window (also called a shifted window), independent self-attention is used within each window to extract high-dimensional feature information from local image blocks. In addition, window division is used in multiple sub-feature extraction layers to capture smaller local information. This ensures efficient computation while still capturing rich visual information and improving the detection accuracy of small target defects.

[0064] 4) If Figure 4As shown in the figure, bilinear interpolation is used to integrate channels as auxiliary branch networks at the output of stage 3 (the third sub-feature extraction layer) and stage 4 (the fourth sub-feature extraction layer). In addition to the main branch used to train the final classifier in stage 4 (for calculating the main loss), an auxiliary branch network is added after stage 3 to calculate the auxiliary loss. This allows for joint parameter optimization and facilitates rapid convergence.

[0065] The auxiliary branch network includes an auxiliary upsampling layer and an auxiliary splicing layer. These layers sequentially perform bilinear interpolation upsampling and splicing operations on the high-dimensional feature maps output by the third sub-feature extraction layer, generating auxiliary branch multi-channel feature maps. The auxiliary branch multi-channel feature maps have a size of 30×30×384. Compared to the main multi-channel feature map, the auxiliary branch multi-channel feature map has a larger local size, allowing it to represent shallow features within a larger local area. In other words, the main multi-channel feature map can represent deep features within a smaller local area.

[0066] The backbone network of the original PSPNet is ResNet or Mobilenet. The disclosed embodiment innovatively adopts an improved version of the Swin transformer as the feature extraction network, rather than simply using the Swin transformer. The improved version of the Swin transformer uses the Swin transformer as the backbone network and introduces an auxiliary branch network. This improvement is to solve the problem that backpropagation cannot be passed to the shallow layer. Auxiliary loss is added. Both losses can be passed to the previous layer. The auxiliary loss optimizes the learning process, the main branch loss bears the main responsibility, and a weight is added to the auxiliary loss. Here, the weight of the backbone loss is greater than the weight of the auxiliary loss, and the ratio between the weight of the backbone loss and the weight of the auxiliary loss is 7:3 to 8:2.

[0067] For example, Figure 3 As shown in Figure 3, the feature segmentation network includes a pyramid pooling layer, a feature segmentation upsampling layer, and a fully connected layer.

[0068] The main multi-channel feature map and the auxiliary branch multi-channel feature map are segmented at multiple levels through the pyramid pooling layer to generate multiple segmentation feature maps at different levels; the segmentation feature maps at multiple levels are spliced through the feature segmentation upsampling layer to obtain the segmentation feature splicing map; the segmentation feature splicing map is feature fused through the fully connected layer to determine the defect type of the gear surface image to be tested.

[0069] For example, the pyramid pooling layer can use global average pooling layers of four different sizes: 1×1, 2×2, 3×3, and 6×6. This layer fuses local region information from multiple segmentation feature maps at different levels. The smaller the pooling result, the larger the receptive field and the more global information from the entire image is integrated. 1×1 convolution is used to reduce the number of channels for the four sizes of segmentation feature maps.

[0070] The feature segmentation upsampling layer can be bilinear interpolation, which performs channel splicing on the deep features (segmentation feature maps with larger sizes) and the shallow features (segmentation feature maps with smaller sizes) to obtain the final segmentation feature splicing map.

[0071] The fully connected layer performs channel adjustment on the final segmented feature mosaic using a 1×1 convolution, outputting the defect detection result corresponding to the gear surface image to be tested. This defect detection result can include the defect category number for each part of the gear surface image to be tested or the image background number.

[0072] As can be seen, the gear defect detection method provided by the embodiments of the present disclosure utilizes computer vision and deep learning methods, based on a PSP segmentation network. A modified Swin-Transformer is used as the feature extraction network, and a PSP segmentation network model is subsequently built to complete the construction of the gear defect detection model. The modified Swin-Transformer is used as the feature extraction network to extract features of tooth surface defects. By introducing self-attention within a small window, accurate identification of wind turbine gearbox gear defects is achieved, improving detection efficiency.

[0073] Based on this, Figure 5 FIG. 1 shows a flow chart of a defect detection method provided by an exemplary embodiment of the present disclosure. Figure 5 As shown, the defect detection method includes:

[0074] S501, obtaining a surface image of a gear to be tested and a pre-built gear defect detection model; wherein the gear defect detection model includes a feature extraction network and a feature segmentation network, wherein the feature extraction network includes a backbone network consisting of an image block partitioning module, a feature extraction layer, and a feature extraction upsampling layer, and an auxiliary branch network;

[0075] S502, using an image block division module to divide the surface image of the gear to be tested into multiple local image blocks; using a feature extraction layer and a feature extraction upsampling layer to extract a main multi-channel feature map from the multiple local image blocks; using an auxiliary branch network to extract an auxiliary branch multi-channel feature map from the multiple local image blocks; wherein the dimension of the auxiliary branch multi-channel feature map is lower than the dimension of the main multi-channel feature map;

[0076] S503, using a feature segmentation network to process the main multi-channel feature map and the auxiliary branch multi-channel feature map to obtain a defect detection result of the gear surface image to be tested.

[0077] Specifically, the above-mentioned gear defect detection model can be pre-constructed. The defect data set for constructing the gear defect detection model can be found in the above text and will not be repeated here.

[0078] like Figure 3 As shown, the gear defect detection model may include a feature extraction network and a feature segmentation network, wherein the feature extraction network may include a backbone network and an auxiliary branch network. Figure 4 As shown, the backbone network may include an image block division module, a feature extraction layer, and a feature extraction upsampling layer.

[0079] After obtaining the surface image of the gear to be tested, the surface image of the gear to be tested and the pre-built gear defect detection model can be input into the gear defect detection model for processing to obtain the defect detection results of the surface image of the gear to be tested. During this processing, the image block division module can be used to divide the surface image of the gear to be tested into multiple local image blocks; the feature extraction layer and the feature extraction upsampling layer can be used to extract the main multi-channel feature map from the multiple local image blocks; the auxiliary branch network can be used to extract the auxiliary branch multi-channel feature map from the multiple local image blocks; finally, the feature segmentation network is used to process the main multi-channel feature map and the auxiliary branch multi-channel feature map to obtain the defect detection results of the surface image of the gear to be tested.

[0080] Among them, the backbone network can obtain a backbone multi-channel feature map with deep features from multiple local image blocks, and the auxiliary branch network obtains an auxiliary branch multi-channel feature map with shallow features from multiple local image blocks, so that the dimension of the auxiliary branch multi-channel feature map is lower than that of the backbone multi-channel feature map. In this way, the auxiliary branch network is used to optimize the learning ability of tiny target defects, solve the problem that back propagation cannot be transmitted to the shallow layer, and improve the detection accuracy of tiny target defects.

[0081] On this basis, the embodiment of the present disclosure abandons the original method of manually screening pictures and determining defects, and uses a pre-built gear defect detection model to detect defects in wind turbine gearbox gears, thereby improving the efficiency of defect detection.

[0082] According to the technical solution of the exemplary embodiment of the present disclosure, by obtaining the surface image of the gear to be tested and the pre-built gear defect detection model; wherein the gear defect detection model includes a feature extraction network and a feature segmentation network, the feature extraction network includes an image block division module, a feature extraction layer and a feature extraction upsampling layer to form a backbone network, and an auxiliary branch network; using the image block division module, the surface image of the gear to be tested is divided into multiple local image blocks; using the feature extraction layer and the feature extraction upsampling layer, a backbone multi-channel feature map is extracted from the multiple local image blocks; using the auxiliary branch network, an auxiliary branch multi-channel feature map is extracted from the multiple local image blocks; wherein the auxiliary branch multi-channel feature map The dimension of the main multi-channel feature map is lower than that of the main multi-channel feature map; the main multi-channel feature map and the auxiliary branch multi-channel feature map are processed by the feature segmentation network to obtain the defect detection result of the surface image of the gear to be tested. On the premise of using the main network to obtain the main multi-channel feature map with deep features, the auxiliary branch network is introduced to obtain the auxiliary branch multi-channel feature map with shallow features, thereby optimizing the learning ability of small target defects and solving the problem that back propagation cannot be transmitted to the shallow layer; and the feature segmentation network is used to determine the defect detection result of the surface image of the gear to be tested based on the main multi-channel feature map and the auxiliary branch multi-channel feature map, thereby improving the detection efficiency and detection accuracy of small target defects.

[0083] In some embodiments, the feature extraction layer includes a first sub-feature extraction layer, a second sub-feature extraction layer, a third sub-feature extraction layer, and a fourth sub-feature extraction layer;

[0084] The first sub-feature extraction layer is used to extract a first local feature map from the multiple local image blocks, the second sub-feature extraction layer is used to extract a second local feature map from the first local feature map, the third sub-feature extraction layer is used to extract a third local feature map from the second local feature map, and the fourth sub-feature extraction layer is used to extract a fourth local feature map from the third local feature map;

[0085] Among them, the local ranges of the first local feature map, the second local feature map, the third local feature map and the fourth local feature map gradually decrease, the sizes gradually decrease and the dimensions gradually increase.

[0086] Specifically, the above-mentioned feature extraction layer may include a first sub-feature extraction layer, a second sub-feature extraction layer, a third sub-feature extraction layer and a fourth sub-feature extraction layer; wherein the first sub-feature extraction layer is used to extract a first local feature map from multiple local image blocks, the second sub-feature extraction layer is used to extract a second local feature map from the first local feature map, the third sub-feature extraction layer is used to extract a third local feature map from the second local feature map, and the fourth sub-feature extraction layer is used to extract a fourth local feature map from the third local feature map.

[0087] Based on the previous information, we can see that the size of the gear surface image to be tested is 473×473×3, the size of the multiple local image blocks obtained after the image block division module is 119×119×48, the size of the first local feature map obtained after processing by the first sub-feature extraction layer is 119×119×96, the size of the second local feature map obtained after processing by the second sub-feature extraction layer is 60×60×192, the size of the third local feature map obtained after processing by the third sub-feature extraction layer is 30×30×384, and the size of the fourth local feature map obtained after processing by the fourth sub-feature extraction layer is 15×15×768. It can be seen that the local range and size of the first, second, third, and fourth local feature maps gradually decrease, and the dimensionality gradually increases.

[0088] After the first sub-feature extraction layer to the fourth sub-feature extraction layer, multiple hierarchical feature maps can be obtained. The sizes of the multiple hierarchical feature maps are gradually reduced, which can reduce the computational complexity of the shift window self-attention layer and realize the linear relationship between computational complexity and image size.

[0089] In some embodiments, the first sub-feature extraction layer includes a linear embedding layer and a shifted window self-attention layer; the second sub-feature extraction layer, the third sub-feature extraction layer, and the fourth sub-feature extraction layer each include an image block embedding layer and a shifted window self-attention layer;

[0090] Among them, the linear embedding layer and the image block embedding layer are used to perform convolution operations with reduced size and increased dimension, and the shifted window self-attention layer is used to perform convolution operations for deep feature extraction.

[0091] Specifically, the first sub-feature extraction layer may include a linear embedding layer and a shift window self-attention layer; the second sub-feature extraction layer, the third sub-feature extraction layer, and the fourth sub-feature extraction layer may each include an image block embedding layer and a shift window self-attention layer;

[0092] Among them, the linear embedding layer and the image block embedding layer are used to perform convolution operations with reduced size and increased dimension. Multiple hierarchical feature maps with gradually reduced sizes are constructed from the first sub-feature extraction layer to the fourth sub-feature extraction layer, gradually reducing the computational complexity of the shift window self-attention layer.

[0093] The shifted window self-attention layer is used to perform convolution operations for deep feature extraction. The size of the shifted windows of each sub-feature extraction layer is the same. By introducing a self-attention mechanism with a sliding window, independent self-attention is used in each window to extract high-dimensional feature information in local image blocks, and window division is used in multiple sub-feature extraction layers to capture smaller local information. In this way, while ensuring efficient calculation, rich visual information can still be captured, thereby improving the detection accuracy of small target defects.

[0094] In some embodiments, the auxiliary branch network is connected between the third sub-feature extraction layer and the fourth sub-feature extraction layer, and the auxiliary branch network includes an auxiliary upsampling layer and an auxiliary splicing layer;

[0095] Using the auxiliary branch network, auxiliary branch multi-channel feature maps are extracted from multiple local image blocks, including:

[0096] Extracting local deep features of the third local feature map from the third local feature map using the auxiliary upsampling layer;

[0097] The auxiliary splicing layer is used to perform feature splicing on the local deep features of the third local feature map to obtain an auxiliary branch multi-channel feature map; wherein the auxiliary branch multi-channel feature map and the local deep features of the third local feature map have the same size and dimension as the third local feature map.

[0098] Specifically, if Figure 4 As shown, the auxiliary branch network can be connected between the third sub-feature extraction layer and the fourth sub-feature extraction layer. The auxiliary branch network can include an auxiliary upsampling layer and an auxiliary splicing layer. The auxiliary upsampling layer can be used to extract the local deep features of the third local feature map from the third local feature map output by the third sub-feature extraction layer; the auxiliary splicing layer can be used to perform feature splicing on the local deep features of the third local feature map to obtain an auxiliary branch multi-channel feature map. For details, please refer to the previous article and will not be repeated here.

[0099] Here, the auxiliary branch multi-channel feature map and the local deep features of the third local feature map have the same size and dimension as the third local feature map.

[0100] As can be seen from the foregoing, the fourth local feature map is smaller in size and higher in dimension than the third local feature map, and the main multi-channel feature map obtained by the feature extraction upsampling layer upsampling the fourth local feature map is the same size and dimension as the fourth local feature map. In other words, the main multi-channel feature map is smaller in size and higher in dimension than the auxiliary branch multi-channel feature map, and the main multi-channel feature map has deeper feature information, while the auxiliary branch multi-channel feature map has shallower feature information. It is further explained that the embodiment of the present disclosure can solve the problem that back propagation cannot be transmitted to the shallow layer by introducing an auxiliary branch network, optimize the learning ability of small target defects, and improve the detection accuracy of small target defects.

[0101] In some embodiments, the feature segmentation network includes a pyramid pooling layer, a feature segmentation upsampling layer, and a fully connected layer;

[0102] Using the feature segmentation network, the main multi-channel feature map and the auxiliary branch multi-channel feature map are processed to obtain the defect detection results of the gear surface image to be tested, including:

[0103] Using the pyramid pooling layer, the main multi-channel feature map and the auxiliary branch multi-channel feature map are segmented at multiple levels to generate multiple segmentation feature maps at different levels;

[0104] Using the feature segmentation upsampling layer, multiple segmentation feature maps of different levels are spliced to obtain a segmentation feature splicing map;

[0105] The fully connected layer is used to perform feature fusion on the segmented feature splicing image to obtain the defect detection results of the gear surface image to be tested.

[0106] Specifically, if Figure 3 As shown, the feature segmentation network can include a pyramid pooling layer, a feature segmentation upsampling layer, and a fully connected layer.

[0107] The pyramid pooling layer can be used to perform feature segmentation at different levels on the main multi-channel feature map and the auxiliary branch multi-channel feature map, generating multiple segmentation feature maps at different levels. The feature segmentation upsampling layer can be used to stitch the multiple segmentation feature maps at different levels to obtain a segmentation feature mosaic map. The fully connected layer can be used to perform feature fusion on the segmentation feature mosaic map to obtain the defect detection results of the gear surface image under test. For details, please refer to the previous article and will not be repeated here.

[0108] In some embodiments, during the training phase, the loss of the gear defect detection model includes a feature extraction loss of the feature extraction network and a feature segmentation loss of the feature segmentation network, and the weight of the feature extraction loss is greater than the weight of the feature segmentation loss;

[0109] The feature extraction loss includes the trunk loss of the backbone network and the auxiliary loss of the auxiliary branch network, and the weight of the trunk loss is greater than that of the auxiliary loss.

[0110] Specifically, during the training phase of the gear defect detection model, the losses of the gear defect detection model can include the feature extraction loss of the feature extraction network and the feature segmentation loss of the feature segmentation network, with the weight of the feature extraction loss being greater than the weight of the feature segmentation loss, allowing the model to focus on learning the feature extraction process for small target defects. For example, the ratio between the weight of the feature extraction loss and the weight of the feature segmentation loss is 7:3 to 8:2. This ratio can also be selected based on actual needs and is not specifically limited in the present embodiments.

[0111] The feature extraction loss includes the trunk loss of the trunk network and the auxiliary loss of the auxiliary branch network, and the weight of the trunk loss is greater than the weight of the auxiliary loss. In order to solve the problem that back propagation cannot be passed to the shallow layer, the auxiliary loss is added to the feature extraction loss. Both the trunk loss and the auxiliary loss can be passed to the previous layer. The auxiliary loss optimizes the learning process. The trunk loss bears the main responsibility and adds a weight to the auxiliary loss. Here, the weight of the trunk loss is greater than the weight of the auxiliary loss. For example, the ratio between the weight of the trunk loss and the weight of the auxiliary loss is 7:3 to 8:2. The ratio can also be selected according to actual needs. The embodiment of the present disclosure does not make specific limitations on this.

[0112] The above mainly introduces the solutions provided by the embodiments of the present disclosure. It is understandable that in order to implement the above functions, the electronic device includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of the various examples described in the embodiments disclosed herein, the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present disclosure.

[0113] The embodiments of the present disclosure can divide the functional units of the electronic device according to the above method examples. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiments of the present disclosure is schematic and is only a logical function division. In actual implementation, there may be other division methods.

[0114] In the case of dividing the functional modules according to the functions, an exemplary embodiment of the present disclosure provides a defect detection device, which may be a terminal or a chip applied to a terminal. Figure 6 A schematic structural diagram of a defect detection device provided by an exemplary embodiment of the present disclosure is shown.

[0115] like Figure 6 As shown, the apparatus 600 includes:

[0116] An acquisition module 601 is configured to acquire a surface image of a gear to be tested and a pre-built gear defect detection model; wherein the gear defect detection model includes a feature extraction network and a feature segmentation network, wherein the feature extraction network includes a backbone network consisting of an image block partitioning module, a feature extraction layer, and a feature extraction upsampling layer, and an auxiliary branch network;

[0117] Processing module 602 is configured to use the image block division module to divide the surface image of the gear to be tested into multiple local image blocks; use the feature extraction layer and the feature extraction upsampling layer to extract a main multi-channel feature map from the multiple local image blocks; use the auxiliary branch network to extract an auxiliary branch multi-channel feature map from the multiple local image blocks; wherein the dimension of the auxiliary branch multi-channel feature map is lower than the dimension of the main multi-channel feature map;

[0118] The processing module 602 is further configured to process the trunk multi-channel feature map and the auxiliary branch multi-channel feature map using the feature segmentation network to obtain a defect detection result of the surface image of the gear to be tested.

[0119] In some embodiments, the feature extraction layer includes a first sub-feature extraction layer, a second sub-feature extraction layer, a third sub-feature extraction layer, and a fourth sub-feature extraction layer;

[0120] The first sub-feature extraction layer is used to extract a first local feature map from the multiple local image blocks, the second sub-feature extraction layer is used to extract a second local feature map from the first local feature map, the third sub-feature extraction layer is used to extract a third local feature map from the second local feature map, and the fourth sub-feature extraction layer is used to extract a fourth local feature map from the third local feature map;

[0121] Among them, the local ranges of the first local feature map, the second local feature map, the third local feature map and the fourth local feature map gradually decrease, the sizes gradually decrease and the dimensions gradually increase.

[0122] In some embodiments, the first sub-feature extraction layer includes a linear embedding layer and a shifted window self-attention layer; the second sub-feature extraction layer, the third sub-feature extraction layer, and the fourth sub-feature extraction layer each include an image block embedding layer and the shifted window self-attention layer;

[0123] The linear embedding layer and the image block embedding layer are used to perform convolution operations with reduced size and increased dimension, and the shifted window self-attention layer is used to perform convolution operations for deep feature extraction.

[0124] In some embodiments, the auxiliary branch network is connected between the third sub-feature extraction layer and the fourth sub-feature extraction layer, and the auxiliary branch network includes an auxiliary upsampling layer and an auxiliary splicing layer;

[0125] The processing module 602 is further configured to extract the local deep features of the third local feature map from the third local feature map using the auxiliary upsampling layer; and perform feature splicing on the local deep features of the third local feature map using the auxiliary splicing layer to obtain an auxiliary branch multi-channel feature map; wherein the auxiliary branch multi-channel feature map and the local deep features of the third local feature map have the same size and dimension as the third local feature map.

[0126] In some embodiments, the feature segmentation network includes a pyramid pooling layer, a feature segmentation upsampling layer, and a fully connected layer;

[0127] The processing module 602 is also used to use the pyramid pooling layer to perform multiple levels of feature segmentation on the trunk multi-channel feature map and the auxiliary branch multi-channel feature map to generate multiple levels of segmentation feature maps; use the feature segmentation upsampling layer to splice the multiple levels of segmentation feature maps to obtain a segmentation feature splicing map; use the fully connected layer to perform feature fusion on the segmentation feature splicing map to obtain a defect detection result of the surface image of the gear to be tested.

[0128] In some embodiments, during the training phase, the loss of the gear defect detection model includes the feature extraction loss of the feature extraction network and the feature segmentation loss of the feature segmentation network, and the weight of the feature extraction loss is greater than the weight of the feature segmentation loss;

[0129] The feature extraction loss includes a trunk loss of the trunk network and an auxiliary loss of the auxiliary branch network, and a weight of the trunk loss is greater than a weight of the auxiliary loss.

[0130] An embodiment of the present disclosure further provides an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the above method disclosed in the embodiment of the present disclosure.

[0131] Figure 7 FIG. 1 shows a schematic diagram of the structure of an electronic device provided by an exemplary embodiment of the present disclosure. Figure 7 As shown, the electronic device 700 includes at least one processor 701 and a memory 702 coupled to the processor 701. The processor 701 can execute corresponding steps in the above method disclosed in the embodiment of the present disclosure.

[0132] The processor 701 can also be referred to as a central processing unit (CPU), which can be an integrated circuit chip with signal processing capabilities. Each step in the above method disclosed in the embodiment of the present disclosure can be completed by the hardware integrated logic circuit in the processor 701 or by instructions in the form of software. The processor 701 can be a general-purpose processor, a digital signal processor (DSP), an ASIC, a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiment of the present disclosure can be directly embodied as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in the memory 702, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, or other mature storage media in the art. The processor 701 reads the information in the memory 702 and completes the steps of the above method in combination with its hardware.

[0133] In addition, when various operations / processes according to the present disclosure are implemented by software and / or firmware, they can be transferred from a storage medium or a network to a computer system having a dedicated hardware structure, for example, Figure 8 The computer system 800 shown is installed with the programs constituting the software. When the various programs are installed, the computer system can perform various functions, including the functions described above. Figure 8 A schematic diagram of the structure of a computer system provided by an exemplary embodiment of the present disclosure is shown.

[0134] Computer system 800 is intended to represent various forms of digital electronic computer equipment, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic equipment can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0135] like Figure 8As shown, the computer system 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. Various programs and data required for the operation of the computer system 800 can also be stored in the RAM 803. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0136] Multiple components within computer system 800 are connected to I / O interface 805, including an input unit 806, an output unit 807, a storage unit 808, and a communication unit 809. Input unit 806 can be any type of device capable of inputting information into computer system 800. Input unit 806 can receive input numeric or character information and generate key input signals related to user settings and / or function control of an electronic device. Output unit 807 can be any type of device capable of presenting information and may include, but is not limited to, a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. Storage unit 808 may include, but is not limited to, a magnetic disk or an optical disk. Communication unit 809 allows computer system 800 to exchange information / data with other devices over a network, such as the Internet, and may include, but is not limited to, a modem, a network card, an infrared communication device, a wireless communication transceiver and / or chipset, such as a Bluetooth™ device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0137] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units for running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above. For example, in some embodiments, the above-mentioned method disclosed in the embodiments of the present disclosure may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on an electronic device via ROM 802 and / or communication unit 809. In some embodiments, the computing unit 801 may be configured to perform the above-mentioned method disclosed in the embodiments of the present disclosure by any other appropriate means (e.g., by means of firmware).

[0138] An embodiment of the present disclosure further provides a computer-readable storage medium, wherein when instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the above method disclosed in the embodiment of the present disclosure.

[0139] The computer-readable storage medium in the embodiments of the present disclosure can be a tangible medium that can contain or store a program for use by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. The above-mentioned computer-readable storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the above. More specifically, the above-mentioned computer-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0140] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0141] The embodiments of the present disclosure further provide a computer program product, including a computer program, wherein when the computer program is executed by a processor, the method disclosed in the embodiments of the present disclosure is implemented.

[0142] In embodiments of the present disclosure, computer program code for performing the operations of the present disclosure may be written in one or more programming languages or combinations thereof, including but not limited to object-oriented programming languages such as Python, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In situations involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer.

[0143] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0144] The modules, components, or units described in the embodiments of the present disclosure may be implemented in software or hardware. The names of the modules, components, or units do not necessarily limit the modules, components, or units themselves.

[0145] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, and without limitation, exemplary hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0146] The above descriptions are merely some embodiments of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but also encompasses other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in the present disclosure.

[0147] Although some specific embodiments of the present disclosure have been described in detail by way of examples, those skilled in the art will appreciate that the above examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Those skilled in the art will appreciate that modifications may be made to the above embodiments without departing from the scope and spirit of the present disclosure. The scope of the present disclosure is defined by the appended claims.

Claims

1. A defect detection method, characterized in that: include: Obtaining a surface image of the gear to be tested and a pre-built gear defect detection model; wherein the gear defect detection model includes a feature extraction network and a feature segmentation network, wherein the feature extraction network includes an image block partitioning module, a backbone network consisting of a feature extraction layer and a feature extraction upsampling layer, and an auxiliary branch network; Using the image block division module, the surface image of the gear to be tested is divided into a plurality of local image blocks; using the feature extraction layer and the feature extraction upsampling layer, a trunk multi-channel feature map is extracted from the plurality of local image blocks; using the auxiliary branch network, an auxiliary branch multi-channel feature map is extracted from the plurality of local image blocks; wherein the dimension of the auxiliary branch multi-channel feature map is lower than the dimension of the trunk multi-channel feature map; The main multi-channel feature map and the auxiliary branch multi-channel feature map are processed using the feature segmentation network to obtain a defect detection result of the surface image of the gear to be tested.

2. The method according to claim 1, wherein The feature extraction layer includes a first sub-feature extraction layer, a second sub-feature extraction layer, a third sub-feature extraction layer and a fourth sub-feature extraction layer; The first sub-feature extraction layer is used to extract a first local feature map from the multiple local image blocks, the second sub-feature extraction layer is used to extract a second local feature map from the first local feature map, the third sub-feature extraction layer is used to extract a third local feature map from the second local feature map, and the fourth sub-feature extraction layer is used to extract a fourth local feature map from the third local feature map; Among them, the local ranges of the first local feature map, the second local feature map, the third local feature map and the fourth local feature map gradually decrease, the sizes gradually decrease and the dimensions gradually increase.

3. The method according to claim 2, wherein The first sub-feature extraction layer includes a linear embedding layer and a shifted window self-attention layer; the second sub-feature extraction layer, the third sub-feature extraction layer, and the fourth sub-feature extraction layer each include an image block embedding layer and the shifted window self-attention layer; The linear embedding layer and the image block embedding layer are used to perform convolution operations with reduced size and increased dimension, and the shifted window self-attention layer is used to perform convolution operations for deep feature extraction.

4. The method according to claim 2, wherein The auxiliary branch network is connected between the third sub-feature extraction layer and the fourth sub-feature extraction layer, and the auxiliary branch network includes an auxiliary upsampling layer and an auxiliary splicing layer; The extracting of auxiliary branch multi-channel feature maps from the multiple local image blocks by using the auxiliary branch network includes: Extracting local deep features of the third local feature map from the third local feature map using the auxiliary upsampling layer; The auxiliary splicing layer is used to perform feature splicing on the local deep features of the third local feature map to obtain an auxiliary branch multi-channel feature map; wherein the auxiliary branch multi-channel feature map and the local deep features of the third local feature map have the same size and dimension as the third local feature map.

5. The method according to claim 1, wherein The feature segmentation network includes a pyramid pooling layer, a feature segmentation upsampling layer and a fully connected layer; The method of using the feature segmentation network to process the main multi-channel feature map and the auxiliary branch multi-channel feature map to obtain a defect detection result of the surface image of the gear to be tested includes: Using the pyramid pooling layer, performing feature segmentation at multiple levels on the trunk multi-channel feature map and the auxiliary branch multi-channel feature map to generate multiple segmentation feature maps at different levels; Using the feature segmentation upsampling layer, the plurality of segmentation feature maps at different levels are spliced to obtain a segmentation feature splicing map; The fully connected layer is used to perform feature fusion on the segmented feature mosaic image to obtain a defect detection result of the surface image of the gear to be tested.

6. The method according to any one of claims 1 to 5, wherein During the training phase, the loss of the gear defect detection model includes the feature extraction loss of the feature extraction network and the feature segmentation loss of the feature segmentation network, and the weight of the feature extraction loss is greater than the weight of the feature segmentation loss; The feature extraction loss includes a trunk loss of the trunk network and an auxiliary loss of the auxiliary branch network, and a weight of the trunk loss is greater than a weight of the auxiliary loss.

7. A defect detection device, characterized in that: include: An acquisition module, configured to acquire an image of the gear surface to be tested and a pre-built gear defect detection model; wherein the gear defect detection model comprises a feature extraction network and a feature segmentation network, wherein the feature extraction network comprises a backbone network consisting of an image block partitioning module, a feature extraction layer, and a feature extraction upsampling layer, and an auxiliary branch network; a processing module configured to divide the surface image of the gear to be tested into a plurality of local image blocks using the image block division module; extract a main multi-channel feature map from the plurality of local image blocks using the feature extraction layer and the feature extraction upsampling layer; and extract an auxiliary branch multi-channel feature map from the plurality of local image blocks using the auxiliary branch network; wherein the dimension of the auxiliary branch multi-channel feature map is lower than the dimension of the main multi-channel feature map; The processing module is also used to use the feature segmentation network to process the main multi-channel feature map and the auxiliary branch multi-channel feature map to obtain a defect detection result of the surface image of the gear to be tested.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to claim 1.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to claim 1 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to claim 1 are implemented.

Images