Small-difference target extraction method and device and terminal equipment

Abstract

The invention relates to the technical field of semiconductor detection, relates to a method for extracting a semiconductor surface object by using an image technology, and particularly relates to a small-difference target extraction method and device and terminal equipment. An encoder, an aggregation network and a decoder are arranged, multi-stage feature extraction is performed on an image through a feature extraction network in the encoder, and the aggregation network fuses output features of the encoder to obtain a deep feature containing global context information. And the decoder gradually fuses the multi-level features and the deep features to obtain an output image, and performs segmentation prediction through a segmentation head based on the output image to obtain an extraction result. The method provided by the invention can be compatible with imaging fluctuation based on an improved model structure, and noise and needle marks can be effectively distinguished without manually setting parameters, so that the boundary and number extraction of the needle marks are more accurate.

Description

Technical Field

[0001] The present application relates to the field of semiconductor detection technology, and is a method for extracting semiconductor surface objects using image technology, specifically a method, device and terminal equipment for extracting small-difference targets. Background Art

[0002] In the prior art, the semiconductor surface detection method generally adopts the method of binarization of the collected regional images, and the recognition of the target is realized through the change of grayscale. For this method, the change of grayscale needs to be more obvious, which requires relatively stable imaging in the imaging stage. The same set of lighting parameters cannot adapt to the slight changes in the process and material of the same material; and the setting of grayscale requires manual setting of the complete and accurate extraction parameters. Different extraction parameters need to be set for different materials and different imaging pictures, which requires the manually set extraction parameters to have high accuracy. However, in actual operation, for small difference targets, the grayscale of noise and the grayscale of target are often very close, so it is impossible to distinguish between noise and real target areas; and because there are multiple targets in one area and the multiple targets are close to each other, the traditional method of extracting the target will be connected as a whole, which cannot reflect the real number of targets. Summary of the invention

[0003] In order to solve the technical problems described in the background technology, an embodiment of the present application provides a small difference target extraction method, which realizes the accurate segmentation of needle marks by constructing a segmentation model in a convergent state. This method is compatible with the imaging fluctuations of the PAD area and does not rely on the accuracy of imaging. It can effectively distinguish between noise and needle marks without manually setting parameters, and is more accurate in extracting the boundaries and quantity of needle marks.

[0004] In order to achieve the above purpose, the technical solution adopted in the embodiment of the present application is as follows:

[0005] In a first aspect, a method for extracting a small-difference target is provided, the method comprising: receiving a complete image to be extracted, and performing multi-level feature extraction on the complete image based on a feature extraction network in an encoder; the feature extraction network comprises a plurality of feature extraction modules, the number of channels of the plurality of feature extraction modules increases, the spatial size decreases, and features from low-level to high-level are extracted respectively; the output features of the encoder are used to generate a deep feature containing global context information based on an aggregation network; the multi-level features and the fused features are gradually fused through a decoder to obtain an output image, and segmentation extraction is performed based on the output image to obtain an extraction result; the decoder comprises a plurality of unified attention fusion modules and a segmentation head, each of the unified attention fusion modules is used to fuse the multi-level features with the deep features to obtain a fused image, and the label of each pixel in the fused image is predicted based on the segmentation head to obtain an extraction result.

[0006] In some specific implementations, the feature extraction network includes five extraction stages constructed by a backbone network, and the backbone network includes five depth-separable convolution modules arranged corresponding to the extraction stages.

[0007] In some specific implementations, in the third feature extraction stage and the fourth feature extraction stage, depthwise separable convolutions with convolution kernels of 5, 3, and 1 are combined in the same layer, and the depthwise separable convolutions in the same layer are fused through a parameterized strategy.

[0008] In some specific implementations, the depthwise separable convolution in the first feature extraction stage and the second feature extraction stage includes one layer of DW convolution and two layers of PW convolution; the first layer of the PW convolution compresses the feature map in the channel dimension, and the second layer of convolution restores the feature map channel by enlarging it.

[0009] In some specific implementations, a loss function layer is further provided in the fourth feature extraction stage, and the loss function layer includes DW / h-swish and PW / h-swish.

[0010] In some specific implementations, the aggregation network fuses the input features through three global average poolings, the fused output features are upsampled by a convolution kernel, the upsampled features are added, and the features are refined and extracted through one convolution to obtain deep features.

[0011] In some specific implementations, the unified attention fusion module generates weights through an attention module, and fuses the multi-level features and the deep features with the weights through multiplication and addition operations.

[0012] In some specific implementations, the unified attention fusion module upsamples the deep features to the same size as the multi-level features through bilinear interpolation, takes the upsampled features and the multi-level features as input and generates weights, applies a multiplication operation to the upsampled features and the multi-level features and performs an element-by-element addition operation on the weighted features to obtain the fused features.

[0013] In a second aspect, a small difference target extraction device is provided, which includes: an encoding unit, used to extract features from a received complete image to obtain multi-level features about the complete image; a feature aggregation unit, used to generate a deep feature containing global context information based on an aggregation network; a decoding unit, used to gradually fuse the multi-level features and the fused features through a decoder to obtain an output image, and perform segmentation extraction based on the output image to obtain an extraction result.

[0014] According to a third aspect, a terminal device is provided, comprising: a processor and a memory connected to the processor, wherein the memory stores instructions executed by the processor, and the instructions enable the processor to perform operations to perform a small difference target extraction method as described in any one of the above.

[0015] In the technical solution provided by the embodiment of the present application, an encoder, an aggregation network and a decoder are provided, and a feature extraction network in the encoder is used to extract multi-level features of the image. The aggregation network fuses the output features of the encoder to obtain a deep feature containing global context information. The decoder gradually fuses the multi-level features and the deep features to obtain an output image, and performs segmentation prediction based on the output image through the segmentation head to obtain the extraction result. The method provided by the present method is based on an improved model structure that is compatible with imaging fluctuations, and can effectively distinguish between noise and needle marks without manually setting parameters, so that the extraction of needle mark boundaries and quantity is more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0017] The methods, systems and / or programs in the accompanying drawings will be further described according to exemplary embodiments. These exemplary embodiments will be described in detail with reference to the drawings. These exemplary embodiments are non-limiting exemplary embodiments, wherein example numbers represent similar mechanisms in the various views of the accompanying drawings.

[0018] Figure 1 It is a schematic diagram of needle marks in the prior art.

[0019] Figure 2 It is a schematic diagram of the result of extracting needle marks using existing technology.

[0020] Figure 3 It is another schematic diagram of the result of extracting the needle mark using the existing technology.

[0021] Figure 4 It is a flowchart of the small difference target extraction method provided in the embodiment of the present application.

[0022] Figure 5 It is a schematic diagram of the model structure provided in the embodiment of the present application.

[0023] Figure 6 It is a schematic diagram of the backbone network structure in an embodiment of the present application.

[0024] Figure 7 It is a schematic diagram of the depthwise separable convolution structure in an embodiment of the present application.

[0025] Figure 8 It is a schematic diagram of the structure of a small difference target extraction device provided in an embodiment of the present application.

[0026] Fig. 9 It is a schematic diagram of the terminal device structure provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to better understand the above technical scheme, the technical scheme of the present application is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present application and the specific features in the embodiments are detailed descriptions of the technical scheme of the present application, rather than limitations on the technical scheme of the present application. In the absence of conflict, the embodiments of the present application and the technical features in the embodiments can be combined with each other.

[0028] In the following detailed description, numerous specific details are set forth by way of example in order to provide a comprehensive understanding of the relevant guidance. However, it will be apparent to those skilled in the art that the present application may be practiced without these details. In other cases, well-known methods, procedures, systems, compositions and / or circuits have been described at a relatively high level, without detail, in order to avoid unnecessarily obscuring aspects of the present application.

[0029] Flowcharts are used in the present application to illustrate the execution process performed by the system according to the embodiment of the present application. It should be clearly understood that the execution process of the flowchart may not be performed in order. On the contrary, these execution processes may be performed in reverse order or simultaneously. In addition, at least one other execution process may be added to the flowchart. One or more execution processes may be deleted from the flowchart.

[0030] Before further describing the embodiments of the present invention in detail, the nouns and terms involved in the embodiments of the present invention are described. The nouns and terms involved in the embodiments of the present invention are subject to the following interpretations.

[0031] (1) In response, it is used to indicate the conditions or states on which the executed operations depend. When the dependent conditions or states are met, one or more operations executed may be in real time or with a set delay. Unless otherwise specified, there is no restriction on the order in which the multiple operations executed are executed.

[0032] (2) Based on is used to indicate the conditions or states on which the executed operations depend. When the dependent conditions or states are met, one or more operations executed may be in real time or have a set delay. Unless otherwise specified, there is no restriction on the order in which the multiple operations executed are executed.

[0033] The embodiment of the present application provides a method for extracting small difference targets, which is suitable for surface defect detection scenarios of semiconductor products, especially for designing a detection of needle marks in a semiconductor manufacturing process.

[0034] Among them, the detection of needle marks produces wafer probe test scenarios. For wafer probe test (Chip Probing, referred to as CP), it is a key link in the semiconductor manufacturing process, located between wafer manufacturing and chip packaging steps. CP testing is to verify the electrical characteristics and functionality of each independent Die (that is, uncut, unpackaged chip unit) at the wafer level. At this stage, since the chip has not been cut and packaged, its pins (or Pads) are directly exposed. During the test, a precision probe card is used to contact each Pad to test whether the die meets the preset performance standards. However, after each test, a probe mark (PM) will be left on the surface of the PAD. If the probe mark is too large, too small, or too close to the edge of the PAD, it will affect the performance of the subsequent Die, so it is necessary to detect the PM on the PAD.

[0035] See also Figure 1 , is a schematic diagram of a needle mark in the embodiment of the present application. In the prior art, the PM detection method generally adopts the method of binarizing the collected PAD area image to realize the identification of the needle mark through the change of grayscale. Figure 1 It can be seen that the grayscale of noise and the grayscale of PM are often very close, making it impossible to distinguish between noise and the actual PM area; and because there may be multiple PMs in a PAD area and the multiple PMs are close to each other, the traditional method of extracting PM will connect them into a whole and cannot reflect the actual number of PMs.

[0036] See also Figure 2 and Figure 3 , are the adverse results caused by the extraction of probe marks using the existing technology, among which Figure 2 It can be seen that the PM extracted by the binary processing method is in a split state, which is an incomplete extraction result. Figure 3 It can be seen that the extracted PM is partial PM, which is also an incomplete extraction result.

[0037] Therefore, in order to solve the technical problem that the above-mentioned prior art has incomplete results for small difference targets, especially for PM extraction in this scene, a small difference target extraction method is provided in the embodiment of the present application. Figure 4 The following steps are involved:

[0038] Step S41. Receive a complete image to be extracted, and perform multi-level feature extraction on the complete image based on a feature extraction network in the encoder.

[0039] Step S42: Generate a deep feature containing global context information based on the output feature of the encoder based on the aggregation network.

[0040] Step S43: The multi-level features and the fusion features are gradually fused through a decoder to obtain an output image, and segmentation and extraction are performed based on the output image to obtain an extraction result.

[0041] In the embodiment of the present application, the extraction of PM is performed by a neural network model, and this model includes three main structures, namely an encoder, an aggregation network and a decoder. For more information about this structure, please refer to Figure 5 As shown, the encoder is configured with a feature extraction network for extracting features from the collected complete image to obtain multi-level features, the aggregation network is used to aggregate the output features of the encoder to generate a deep feature containing global context information, the decoder is used to gradually fuse the multi-level features with the deep features to obtain an output image, and the output image is segmented and extracted to obtain the extraction result.

[0042] See Figure 5 It can be seen that the feature extraction process of the encoder in this embodiment includes five stages, and the extraction of features at different levels is achieved through five stages. Among them, the multiple feature extraction modules in the substructure have a smaller spatial size based on the increase in the number of channels in the extraction stage, and extract features from low levels to high levels respectively. For the decoder to realize the fusion of features, in order to further enhance the expression of the fusion results, in this embodiment, a unified attention fusion module is set up step by step in the decoder. In this module, an attention module is used to generate the weight α, and the features generated in the encoder are fused with α through multiplication (Mul) and addition (Add) operations.

[0043] In this embodiment, the features generated in the encoder include hierarchical features in the feature extraction process performed by the encoder, and also include deep features output by the aggregation network.

[0044] The fusion process of the unified attention module first uses a bilinear difference operation to transform the deep feature F high Upsample to the multi-level feature F low The same size, the upsampled features are recorded as F up Then agree that the attention module is F up and F low As input and generate weight α. Among them, the attention module can be a spatial attention module or a channel attention module. After that, in order to obtain the features with attention weights, the element-level Mul operation is applied to and. Finally, the unified attention module performs an element-by-element addition operation on the features with attention weights and outputs the fused features. The above process is expressed based on the following formula:

[0045] F up =Upsample(F high ), α=Attention(F up ,F low ), F out =F up α+F low ·(1-α).

[0046] Among them, the backbone structure of the feature extraction network in the encoder can be found in Figure 6 As shown, it includes five depth-separable convolution modules corresponding to the extraction stage. The backbone network of the encoder in this embodiment is optimized with reference to the PPLCNetV2 network, and the optimization direction is to remove the SE module in the original PPLCNetV2 network.

[0047] Specifically, the feature extraction module is composed of two kinds of depth-wise separable convolutions with kernel sizes of 3×3 and 5×5, where the convolution settings of the depth-wise separable convolutions in different extraction stages are different. In the first and second extraction stages, in order to make the depth-wise separable convolution have stronger fitting ability, one layer of DW convolution and two layers of PW convolution are used, while controlling the model efficiency not to be affected. The two layers of convolution are respectively set as follows: the first one compresses the feature map in the channel dimension, and the second one restores the feature map channel by enlarging it. This strategy can significantly improve the model performance, while balancing the problems brought about by the model efficiency.

[0048] In the third and fourth extraction stages, DW convolutions with kernels of 5, 3, and 1 are used in combination on the same layer. At the same time, in order to avoid affecting the efficiency of the model, the DW convolutions of the same layer are fused using the reparameterization (Rep) strategy in the fourth extraction stage of the backbone network. For more information about this structure, please refer to Figure 6 and Figure 7 As shown. In addition, an H-Swish activation function is also provided in the fourth extraction stage to improve network performance while keeping the inference time almost unchanged. The H-Swish activation function includes a DW / h-swish activation function and a PW / h-swish activation function. In this embodiment, since a unified attention module is provided in the decoder, the attention module in the original PPLCNetV2 network is eliminated to avoid redundant attention modules causing overfitting of the model, and the model size can be further compressed to improve the real-time performance of the model for inference. In the fifth feature extraction stage, one layer of DW convolution and two layers of PW convolution are still used, and the residual structure is also used in the feature extraction stage again, and a Shortcut is added to the Block.

[0049] In this embodiment, context information is fused through an aggregation network, wherein the aggregation network is implemented using a simplified pyramid pooling module. The aggregation network first uses a pyramid pooling layer to fuse input features, including three global average pooling operations, and the corresponding pooling window sizes are 1×1, 2×2, and 4×4, respectively. Afterwards, the output features undergo convolution and upsampling operations. For the convolution operation, the convolution kernel size is 1×1, and the number of output channels is less than the number of input channels. Finally, these upsampled features are added, and a convolution operation is applied to generate refined features. Compared with the original pyramid pooling module, the simplified pyramid pooling module in this embodiment reduces the number of intermediate and output channels, removes short-circuit connections, and replaces the splicing operation with an addition operation, which is more in line with the lightweight and efficient characteristics of the real-time model.

[0050] In this embodiment, a segmentation head is set in the decoder to implement labeling of each pixel in the above fused image, thereby obtaining the final extraction result.

[0051] For the small difference target extraction method provided in the embodiment of the present application, an encoder, an aggregation network and a decoder are provided, and a feature extraction network in the encoder is used to perform multi-level feature extraction on the image. The aggregation network fuses the output features of the encoder to obtain a deep feature containing global context information. The decoder gradually fuses the multi-level features and the deep features to obtain the output image, and performs segmentation prediction based on the output image through the segmentation head to obtain the extraction result. The method provided by the present method is based on an improved model structure that is compatible with imaging fluctuations, and can effectively distinguish between noise and needle marks without manually setting parameters, so that the extraction of needle mark boundaries and quantity is more accurate.

[0052] See also Figure 8 , with respect to the method provided in step S41 to step S43, a small difference extraction device 80 is also provided, the system comprising:

[0053] The encoding unit 81 is used to extract features from the received complete image to obtain multi-level features of the complete image.

[0054] The feature aggregation unit 82 generates a deep feature containing global context information based on the aggregation network.

[0055] The decoding unit 83 is used to gradually fuse the multi-level features and the fusion features through a decoder to obtain an output image, and perform segmentation extraction based on the output image to obtain an extraction result.

[0056] See also Fig. 9 , the above method can also be integrated into the provided terminal device 90, and the device may have relatively large differences due to different configurations or performances, and may include one or more processors 901 and memory 902, and the memory 902 may store one or more storage applications or data. Among them, the memory 902 can be a temporary storage or a permanent storage. The application stored in the memory 902 may include one or more modules (not shown in the figure), and each module may include a series of computer executable instructions in the terminal device. Furthermore, the processor 901 can be configured to communicate with the memory 902, and the terminal device executes a series of computer executable instructions in the memory 902. The terminal device may also include one or more power supplies 903, one or more wired / wireless network interfaces 904, one or more input / output interfaces 905, one or more keyboards 906, etc.

[0057] In a specific embodiment, the terminal device includes a memory and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer executable instructions in the terminal device, and the one or more programs are configured to be executed by one or more processors, including computer executable instructions for performing the following:

[0058] Receiving a complete image to be extracted, and performing multi-level feature extraction on the complete image based on a feature extraction network in an encoder;

[0059] The output features of the encoder are used to generate a deep feature containing global context information based on an aggregation network;

[0060] The multi-level features and the fusion features are gradually fused through a decoder to obtain an output image, and segmentation extraction is performed based on the output image to obtain an extraction result.

[0061] The following is a detailed introduction to the various components of the processor:

[0062] In this embodiment, the processor is an application specific integrated circuit (ASIC), or is configured to implement one or more integrated circuits of the embodiments of the present application, such as one or more microprocessors (digital signal processors, DSP), or one or more field programmable gate arrays (field programmable gate arrays, FPGA).

[0063] Optionally, the processor may execute various functions by running or executing a software program stored in the memory and calling data stored in the memory, such as executing the above-mentioned Figure 4 The method shown.

[0064] In a specific implementation, as an embodiment, the processor may include one or more microprocessors.

[0065] Among them, the memory is used to store the software program that executes the solution of the present application, and the execution is controlled by the processor. The specific implementation method can refer to the above method embodiment and will not be repeated here.

[0066] Optionally, the memory may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory may be integrated with the processor, or may exist independently and be coupled to the processing unit through the interface circuit of the processor, and the embodiments of the present application do not specifically limit this.

[0067] It should be noted that the structure of the processor shown in this embodiment does not constitute a limitation on the device, and the actual device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0068] In addition, the technical effects of the processor can refer to the technical effects of the method described in the above method embodiment, which will not be repeated here.

[0069] It should be understood that the processor in the embodiments of the present application may be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0070] It should also be understood that the memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0071] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0072] In this application, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0073] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0074] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0075] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0076] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0077] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0078] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0079] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks or optical disks.

[0080] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A small difference target extraction method, characterized in that: The method comprises: Receiving a complete image to be extracted, and performing multi-level feature extraction on the complete image based on a feature extraction network in an encoder; the feature extraction network includes a plurality of feature extraction modules, the number of channels of the plurality of feature extraction modules increases, the spatial size decreases, and features from low level to high level are extracted respectively; The output features of the encoder are used to generate a deep feature containing global context information based on an aggregation network; The multi-level features and the deep features are gradually fused through the decoder to obtain an output image, and segmentation and extraction are performed based on the output image to obtain an extraction result; the decoder includes multiple unified attention fusion modules and a segmentation head, each of the unified attention fusion modules is used to fuse the multi-level features and the deep features to obtain a fused image, and the label of each pixel in the fused image is predicted based on the segmentation head to obtain an extraction result.

2. The small difference target extraction method according to claim 1, characterized in that: The feature extraction network includes five extraction stages constructed by a backbone network, and the backbone network includes five depth-separable convolution modules arranged corresponding to the extraction stages.

3. The small difference target extraction method according to claim 2, characterized in that: In the third and fourth feature extraction stages, depthwise separable convolutions with convolution kernels of 5, 3, and 1 are combined in the same layer, and the depthwise separable convolutions in the same layer are fused through a parameterized strategy.

4. The small difference target extraction method according to claim 3 is characterized in that: The depthwise separable convolution in the first feature extraction stage and the second feature extraction stage includes one layer of DW convolution and two layers of PW convolution; the first layer of the PW convolution compresses the feature map in the channel dimension, and the second layer of the convolution restores the feature map channel by enlarging it.

5. The method for extracting small difference target features according to claim 4, characterized in that: A loss function layer is also provided in the fourth feature extraction stage, and the loss function layer includes DW / h-swish and PW / h-swish.

6. The small difference target extraction method according to claim 1, characterized in that: The aggregation network fuses the input features through three global average pooling operations, and the fused output features are upsampled by convolution kernels. The upsampled features are added and refined and extracted through one convolution to obtain deep features.

7. The small difference target extraction method according to claim 6, characterized in that: The unified attention fusion module generates weights through the attention module, and fuses the multi-level features and the deep features with the weights through multiplication and addition operations.

8. The small difference target extraction method according to claim 7, characterized in that: The unified attention fusion module upsamples the deep features to the same size as the multi-level features through bilinear interpolation, takes the upsampled features and the multi-level features as input and generates weights, applies multiplication operations to the upsampled features and the multi-level features and performs element-by-element addition operations on the weighted features to obtain fused features.

9. A small difference target extraction device, characterized in that: The device comprises: An encoding unit, used for extracting features from the received complete image to obtain multi-level features of the complete image; Feature aggregation unit, which generates a deep feature containing global context information based on the aggregation network; The decoding unit is used to gradually fuse the multi-level features and the fusion features through a decoder to obtain an output image, and perform segmentation extraction based on the output image to obtain an extraction result.

10. A terminal device, characterized in that: include: A processor and a memory connected to the processor, wherein the memory stores instructions executed by the processor, and the instructions enable the processor to perform operations to perform the small difference target extraction method as described in any one of claims 1-8.

Images