Complex aluminum alloy casting defect area segmentation method and system

Abstract

The invention provides a complex aluminum alloy casting defect area segmentation method and system. The method comprises the steps that deep layer features and shallow layer features of an original image are extracted based on a constructed ENM-DSDCS coding network; the deep features and the shallow features are subjected to bidirectional guiding feature fusion decoding based on the constructed DNM-ABGF decoding network, a defect area segmentation image is obtained, the deep features and the shallow features in the complex aluminum alloy casting image are subjected to double-path coding through the ENM-DSDCS coding network and then input into the DNM-ABGF decoding network for bidirectional guiding feature fusion decoding, and the defect area segmentation image is obtained. Effective segmentation of a casting defect target is realized through a structure from double-path coding to fusion decoding, and compared with an existing defect region segmentation method, the detection time is basically the same, the accuracy and segmentation precision are higher, and better comprehensive performance is achieved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of casting defect region segmentation, and particularly relates to a complex aluminum alloy casting defect region segmentation method and system. BACKGROUND

[0002] The DR nondestructive detection technology has become a core means for internal quality control of complex aluminum alloy castings due to its advantages of dynamic imaging and high resolution. The defect size is one of important bases for realizing defect grade and product quality evaluation. At present, the quantitative information of defects is mainly obtained from images by using a magnifying glass, a ruler, a template and other tools in an artificial manner, which is easy to be affected by the level of workers and has problems of low efficiency and poor consistency.

[0003] In the prior art, a double-branch encoder-decoder network is proposed in Du W, Shen H, Fu J. Automatic defect segmentation in X-ray images based on deep learning[J]. IEEE Transactions on Industrial Electronics, 2020, 68(12): 12912-12920. file. Although the double-branch encoder-decoder network can realize an "end-to-end" analysis effect, that is, the entire process from inputting an image to outputting a segmentation map is completed by one model without manually designing intermediate steps or a feature extraction process, due to the unique and complex structure of the casting, the attenuation and scattering phenomenon of the ray when passing through the casting, the internal connection of the defect region on the image being fuzzy, the boundary being unclear and the like, the method can only segment the significant region of the defect when segmenting the defect region, and cannot realize effective segmentation of the fuzzy region and the weak texture region. SUMMARY

[0004] In order to solve the problem that the existing alloy casting defect region segmentation method cannot realize effective segmentation of the fuzzy region and the weak texture region, the application provides a complex aluminum alloy casting defect region segmentation method and system.

[0005] To achieve the above purpose, the application provides the following technical scheme: The application provides a complex aluminum alloy casting defect region segmentation method, which comprises the following steps: extracting deep features and shallow features of the original image based on the constructed ENM-DSDCS encoding network; Based on the constructed DNM-ABGF decoding network, bidirectional guided feature fusion decoding is performed on the deep features and the shallow features to obtain a defect region segmentation image.

[0006] Preferably, the constructed ENM-DSDCS coding network includes a deep feature structure network (DFN) and a shallow feature structure network (SFN). The deep feature structure network (DFN) includes a first deformable convolutional composite layer D_Conv1, a second deformable convolutional composite layer D_Conv2, a third deformable convolutional composite layer D_Conv3, a fourth deformable convolutional composite layer D_Conv4, and a fifth deformable convolutional composite layer D_Conv5 connected in series. The shallow feature structure network (SFN) includes a first shallow convolutional composite layer S_Conv1, a second shallow convolutional composite layer S_Conv2, a third shallow convolutional composite layer S_Conv3, and a fourth shallow convolutional composite layer S_Conv4, which are connected in series.

[0007] Preferably, the constructed ENM-DSDCS coding network extracts both deep and shallow features of the original image, including: The original image of the defect area of ​​the complex aluminum alloy casting is obtained, and the original image is processed to obtain a standard three-channel image. The standard three-channel image is processed by a Deep Feature Structure Network (DFN) to extract features, resulting in an image of size [size missing]. deep feature matrix ; The standard three-channel image is processed by a shallow feature structure network (SFN) to extract features, resulting in shallow feature matrices at multiple scales.

[0008] Preferably, the standard three-channel image is used for feature extraction via a Deep Feature Structure Network (DFN) to obtain a size of [size missing]. deep feature matrix ,include: The standard three-channel image is processed by the first deformable convolutional composite layer D_Conv1 to increase the receptive field and normalize the feature dimension, thus obtaining the first deep feature map. The first deep feature map is passed through the second deformable convolutional composite layer D_Conv2 to cover a larger receptive field and normalize the feature dimension, and expand the number of channels to achieve feature dimensionality enhancement. Then, the number of channels is compressed, the inter-layer output is normalized and regularized. The above operations are repeated three times to obtain the second deep feature map. After downsampling the second deep feature map through the third deformable convolutional composite layer D_Conv3, the following operation is repeated three times to cover a larger receptive field and normalize the feature dimension, expand the number of channels to achieve feature dimensionality upgrade, compress the number of channels, normalize the inter-layer output and perform regularization processing to obtain the third deep feature map. After downsampling the third deep feature map through the fourth deformable convolutional composite layer D_Conv4, it covers a larger receptive field and normalizes the feature dimension, expands the number of channels to achieve feature dimensionality enhancement, then compresses the number of channels, normalizes the inter-layer output and performs regularization processing to obtain the fourth deep feature map. After downsampling the fourth deep feature map through the fifth deformable convolutional composite layer D_Conv5, the following operation is repeated three times to cover a larger receptive field and normalize the feature dimension, expand the number of channels to achieve feature dimensionality enhancement, then compress the number of channels, normalize the inter-layer output, and perform regularization to obtain a size of [size missing]. The deep feature matrix.

[0009] Preferably, the standard three-channel image is used for feature extraction via a shallow feature structure network (SFN), including: The standard three-channel image is processed by feature extraction through the first shallow convolutional composite layer S_Conv1 to obtain a size of [size missing]. First shallow feature matrix ; The size is The first shallow feature matrix Feature extraction is performed using the second shallow convolutional composite layer S_Conv2 to obtain a size of The second shallow feature matrix ; The size is The second shallow feature matrix Feature extraction is performed through the third shallow convolutional composite layer S_Conv3 to obtain a size of The third shallow feature matrix ; The size is The third shallow feature matrix Feature extraction is performed using the fourth shallow convolutional composite layer S_Conv4, resulting in a size of... The fourth shallow feature matrix ; Statistical analysis of the first shallow feature matrix The second shallow feature matrix The third shallow layer feature matrix and the fourth shallow feature matrix This yields shallow feature matrices at multiple scales.

[0010] Preferably, the DNM-ABGF decoding network comprises: a shallow enhancement branch and a down-sampling branch for processing the shallow features; an up-sampling branch and a deep enhancement branch for processing the deep features; an attention enhancement branch for recombining and fusing the features.

[0011] Preferably, the DNM-ABGF decoding network based on the constructed DNM-ABGF decoding network bidirectional guided feature fusion decoding of the deep features and the shallow features, comprising: the shallow feature matrix is processed by the shallow enhancement branch for feature increase and channel adjustment to obtain a shallow feature enhancement matrix; the deep feature matrix is processed by the up-sampling branch for feature map resolution adjustment to obtain a deep feature adjustment matrix; and the shallow feature enhancement matrix and the deep feature adjustment matrix are recombined and fused to obtain a first recombined and fused feature map layer; the shallow feature matrix is processed by the down-sampling branch for image resolution adjustment to obtain a shallow feature adjustment matrix; the deep feature matrix is processed by the deep enhancement branch for feature enhancement and channel adjustment to obtain a deep enhancement feature matrix; the shallow feature adjustment matrix and the deep feature adjustment matrix are recombined and fused to obtain a second recombined and fused feature map layer; the second recombined and fused feature layer and the first recombined and fused feature layer are fused by the attention enhancement branch to obtain the defect area segmentation image.

[0012] The present application proposes a complex aluminum alloy casting defect area segmentation system, which realizes the above-mentioned complex aluminum alloy casting defect area segmentation method, comprising: an image acquisition module configured to acquire an original image of a complex aluminum alloy casting; an ENM-DSDCS encoding module configured to extract deep features and shallow features of the original image based on a constructed ENM-DSDCS encoding network; a DNM-ABGF decoding module configured to bidirectional guided feature fusion decoding of the deep features and the shallow features based on a constructed DNM-ABGF decoding network to obtain a defect area segmentation image; a segmentation image output module configured to output the defect area segmentation image.

[0013] The application provides a computer device, including a memory, a processor and a computer program stored in the memory and executable in the processor, wherein the processor implements the steps of the above-mentioned complex aluminum alloy casting defect region segmentation method when executing the computer program.

[0014] The application provides a computer readable storage medium, which stores a computer program, wherein the computer program implements the steps of the above-mentioned complex aluminum alloy casting defect region segmentation method when executed by a processor.

[0015] Compared with the prior art, the application has the following beneficial technical effects: The application provides a complex aluminum alloy casting defect region segmentation method, which combines a logic structure of "encoding-decoding" into a network structure, organically combines the two, and jointly constructs a "double-path encoding-fusion decoding" segmentation method, that is, the input complex aluminum alloy casting DR image is encoded by an ENM-DSDCS encoding network in a double-path mode, and then is input into a DNMA-BGFA decoding network for bidirectional guided feature fusion decoding, so that the effective segmentation of the casting defect target is realized through the structure of double-path encoding to fusion decoding, and compared with the prior art, the detection time is basically the same, the accuracy and segmentation precision are higher, and the comprehensive performance is better.

[0016] Further, in the method, the ENM-DSDCS encoding network extracts deep features and shallow features of the complex aluminum alloy casting image by using the more obvious macroscopic and overall features in the deep network and the information and shallow detail features of more pixel points contained in the shallow network, thereby effectively realizing the extraction of overall features and detail features of the defect significant region and the fuzzy region, and improving the segmentation capability for the defect region.

[0017] Further, in the method, the DNMA-BGFA decoding network realizes the effective fusion of different scale features by using the deep features and the shallow features, and finally obtains a defect region segmentation image, thereby improving the segmentation precision of the defect fuzzy region. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 A flowchart of a complex aluminum alloy casting defect region segmentation method provided by the application is shown in the figure. Figure 2 An ENM-DSDCS encoding network framework constructed in the complex aluminum alloy casting defect region segmentation method provided by the application is shown in the figure. Figure 3 A deep feature extraction flowchart in the complex aluminum alloy casting defect region segmentation method provided by the application is shown in the figure. Figure 4A flowchart of a shallow feature proposal process in a complex aluminum alloy casting defect region segmentation method provided by the present application is shown in the figure; Figure 5 A flowchart of bidirectional guided feature fusion decoding in a complex aluminum alloy casting defect region segmentation method provided by the present application is shown in the figure; Figure 6 A flowchart of deep feature matrix and shallow feature matrix fusion in a complex aluminum alloy casting defect region segmentation method provided by the present application is shown in the figure; Figure 7 A processing logic architecture of a complex aluminum alloy casting defect region segmentation method provided by the present application is shown in the figure; Figure 8 A schematic diagram of a computer device provided by an embodiment of the present application is shown in the figure; Figure 9 A block diagram of a chip provided by an embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0019] In the following, only certain exemplary embodiments are simply described. As those skilled in the art can recognize, the described embodiments can be modified in various different ways without departing from the spirit or scope of the present application. Therefore, the drawings and the description are considered to be exemplary in nature rather than limiting.

[0020] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0021] The present application proposes a complex aluminum alloy casting defect region segmentation method, as shown in Figure 1 and Figure 7 , comprising the following steps: Based on the constructed ENM-DSDCS encoding network, the deep feature matrix and the shallow feature matrix of the original image are extracted respectively; Specifically, in order to solve the problem of incomplete extraction of defect fuzzy area and weak texture area detail features, the macroscopic and overall features in the deep network are more obvious, and the shallow network contains more pixel point information and shallow detail features, so an ENM-DSDCS encoding network with deep and shallow dual-channel structure is constructed, as shown in Figure 2 The ENM-DSDCS encoding network includes a deep feature structure network DFN and a shallow feature structure network SFN; The deep feature network DFN includes a first deformable convolution composite layer D_Conv1, a second deformable convolution composite layer D_Conv2, a third deformable convolution composite layer D_Conv3, a fourth deformable convolution composite layer D_Conv4, and a fifth deformable convolution composite layer D_Conv5; and the shallow feature network SFN includes a first shallow convolution composite layer S_Conv1, a second shallow convolution composite layer S_Conv2, a third shallow convolution composite layer S_Conv3, and a fourth shallow convolution composite layer S_Conv4. An original image of a defect area of a complex aluminum alloy casting is acquired, and shape processing is performed on the original image to obtain a standard three-channel image, wherein the size of the standard three-channel image is WxHx3; The standard three-channel image is sequentially subjected to feature extraction through deformable convolution composite layers to obtain a deep feature matrix with a size of ; Specifically, the standard three-channel image is subjected to receptive field lifting and feature dimension normalization processing through the first deformable convolution composite layer D_Conv1 to obtain a first deep feature map; the first deep feature map is subjected to larger receptive field covering and normalized feature dimension, and expanded channel number to realize feature dimension lifting, and then compressed channel number, normalized interlayer output, and regularized processing, and the above operations are repeated three times to obtain a second deep feature map; the second deep feature map is subjected to down-sampling through the third deformable convolution composite layer D_Conv3, and the following operations are repeated three times: larger receptive field covering and normalized feature dimension, and expanded channel number to realize feature dimension lifting, and then compressed channel number, normalized interlayer output, and regularized processing, to obtain a third deep feature map; the third deep feature map is subjected to down-sampling through the fourth deformable convolution composite layer D_Conv4, and the following operations are repeated nine times: larger receptive field covering and normalized feature dimension, and expanded channel number to realize feature dimension lifting, and then compressed channel number, normalized interlayer output, and regularized processing, to obtain a fourth deep feature map; the fourth deep feature map is subjected to down-sampling through the fifth deformable convolution composite layer D_Conv5, and the following operations are repeated three times: larger receptive field covering and normalized feature dimension, and expanded channel number to realize feature dimension lifting, and then compressed channel number, normalized interlayer output, and regularized processing, to obtain a deep feature matrix with a size of , as shown in Figure 3 ; ​​The first deformable convolution composite layer D_Conv1 includes a 4x4 convolution layer (4x4 CONV) and a normalization layer (Layer Norm); the second deformable convolution composite layer D_Conv2 includes a first triple convolutional neural network block layer (ConvNeXtBlockx3); the third deformable convolution composite layer D_Conv3 and the fifth deformable convolution composite layer D_Conv5 each include a first down-sampling layer (Downsample) and a second triple convolutional neural network block layer (ConvNeXtBlockx3); and the fourth deformable convolution composite layer D_Conv4 includes a second down-sampling layer (Downsample) and a nine-fold convolutional neural network block layer (ConvNeXtBlockx9). The convolutional neural network block layer (ConvNeXt Block) includes, in sequence, a 7x7 large-core depth convolution sublayer (depthwise convolution), a first normalization sublayer (Layer Norm), a 1x1 expansion convolution sublayer (1x1 CONV), an activation function (GELU), a 1x1 linear convolution sublayer (1x1 CONV), an activation function (GELU), a layer scaling sublayer (Layer Scale), and a random path loss sublayer (DropPath). The 7x7 large-core depth convolution sublayer covers a larger receptive field to capture more spatial features and information of minor defects in the feature map, while effectively controlling and optimizing the calculation amount and parameter quantity of the 7x7 large-core depth convolution sublayer. Then, the normalization sublayer is used for normalization processing in the feature dimension, and the 1x1 expansion convolution sublayer is used to expand the number of feature map channels to realize feature dimensionality. Subsequently, the activation function (GELU) is used instead of the commonly used ReLU as the activation function, and the 1x1 linear convolution sublayer is used to compress the number of feature map channels to reduce the calculation complexity and parameter quantity, so as to better utilize the correlation between channels and improve the expression ability of features and the accuracy of the model. Finally, the layer scaling sublayer (Layer Scale) is used to scale the input tensor between the outputs of the first normalization sublayer, and the random path loss sublayer (Drop Path) is used for regularization processing, so as to realize random deletion of sub-paths in the multi-branch structure in the model to prevent overfitting and increase the expressiveness and generalization ability of the model. The first down-sampling layer (Downsample) includes a second normalization sublayer (Layer Scale) and a 2x2 convolution sublayer (2x2 CONV).

[0022] That is, the standard three-channel image is down-sampled by 4x4 CONV, the small defect area in the standard three-channel image is captured by a large receptive field, a 4x4 convolution feature map is obtained, the 4x4 convolution feature map is independently normalized at each spatial position by a normalization layer, the dimensional difference between channels is eliminated, the feature consistency is enhanced, and a first deep feature map is obtained; The first deep feature map is covered with a larger receptive field and normalized feature dimension by a first triple convolutional neural network block (ConvNeXt Block x 3), and the channel number is expanded to realize feature dimensioning, and then the channel number is compressed, the outputs between normalization layers are normalized and regularized, the above operations are repeated three times, and a second deep feature map is obtained; that is, the first deep feature map is covered with a larger receptive field by a 7x7 large kernel deep convolution sublayer, spatial features and context information are extracted, and a 7x7 convolution feature map is obtained; the 7x7 convolution feature map is normalized in the feature dimension by a normalization sublayer, then a 1x1 expansion convolution sublayer is used to expand the channel number of the 7x7 convolution feature map to realize feature dimensioning, a feature dimensioning map is obtained, then an activation function (GELU) is used to activate the feature dimensioning map, then a 1x1 linear convolution sublayer is used to compress the channel number of the feature dimensioning map to reduce the calculation complexity and the number of parameters, so as to better utilize the correlation between channels and improve the expression ability of the feature and the accuracy of the model; finally, the outputs between the normalization layers are normalized by using a layer scaling sublayer to scale the input tensor, and a random path loss sublayer is used for regularization processing, and after repeating three times, the first deep feature map is fitted to realize random deletion of sub-paths in the multi-branch structure in the model to prevent overfitting, and a second deep feature map is obtained.

[0023] The second deep feature map is normalized by a first down-sampling layer and is down-sampled to obtain a third initial deep feature map, and then the third initial deep feature map is covered with a larger receptive field and normalized feature dimension by a second triple convolutional neural network block layer, and the channel number is expanded to realize feature dimensioning, and then the channel number is compressed, the outputs between normalization layers are normalized and regularized, the above operations are repeated three times, and a third deep feature map is obtained; that is, the second deep feature map is normalized by a second normalization sublayer to stabilize the output feature distribution and reduce the size fluctuation of the feature, a second normalized deep feature map is obtained, the second normalized deep feature map is down-sampled and reduced in dimension by a 2x2 convolution sublayer to obtain a third initial deep feature map, the third initial deep feature map is processed by a second triple convolutional neural network block layer and fitted with the second deep feature map to obtain a third deep feature map.

[0024] The third deep feature map is normalized by a second normalization sublayer, the output feature distribution is stabilized, and the size fluctuation of the feature is reduced to obtain a third normalized deep feature map. The third normalized deep feature map is down-sampled and reduced in dimension by a 2x2 convolution sublayer to obtain a fourth initial deep feature map. The fourth initial deep feature map is processed by a triple convolutional neural network block layer to fit the third deep feature map, and a fourth deep feature map is obtained. The fourth deep feature map is normalized by a second normalization sublayer, the output feature distribution is stabilized, and the size fluctuation of the feature is reduced to obtain a fourth normalized deep feature map. The fourth normalized deep feature map is down-sampled and reduced in dimension by a 2x2 convolution sublayer to obtain a fifth initial deep feature map. The fifth initial deep feature map is processed by a second triple convolutional neural network block layer (ConvNeXt Blockx3) to cover a larger receptive field and normalize the feature dimension, expand the channel number to realize feature dimension, and then compress the channel number, normalize the interlayer output and regularize the processing. The above operation is repeated three times to obtain a deep feature matrix with a size of The fourth deep feature map is normalized by a second normalization sublayer, the output feature distribution is stabilized, and the size fluctuation of the feature is reduced to obtain a fourth normalized deep feature map. The fourth normalized deep feature map is down-sampled and reduced in dimension by a 2x2 convolution sublayer to obtain a fifth initial deep feature map. The fifth initial deep feature map is processed by a second triple convolutional neural network block layer (ConvNeXt Blockx3) to cover a larger receptive field and normalize the feature dimension, expand the channel number to realize feature dimension, and then compress the channel number, normalize the interlayer output and regularize the processing. The above operation is repeated three times to obtain a deep feature matrix with a size of ; The standard three-channel image is sequentially subjected to feature extraction by a plurality of shallow convolutional layers to obtain a plurality of scale shallow feature matrices. Specifically, the standard three-channel image is subjected to feature extraction by a first shallow convolutional composite layer S_Conv1 to obtain a first shallow feature matrix with a size of The first shallow feature matrix with a size of is subjected to feature extraction by a second shallow convolutional composite layer S_Conv2 to obtain a second shallow feature matrix with a size of The second shallow feature matrix with a size of is subjected to feature extraction by a third shallow convolutional composite layer S_Conv3 to obtain a third shallow feature matrix with a size of The third shallow feature matrix with a size of ​​​​​​; the third shallow feature matrix with a size of is extracted by the fourth shallow convolution composite layer S_Conv4, and a fourth shallow feature matrix with a size of is obtained. The first shallow feature matrix Figure 4 , the second shallow feature matrix , the third shallow feature matrix and the fourth shallow feature matrix are counted to obtain a plurality of scale shallow feature matrices, as shown in ; The first shallow feature matrix with a size of is expanded in the channel number by the 1x1 convolution sublayer under the condition that the resolution of the input feature map is unchanged, the spatial detail information is extracted by the first 3x3 convolution sublayer with a step of 2 for down-sampling processing, then the feature is upgraded by the 1x1 convolution sublayer under the condition that the resolution of the input feature map is unchanged, and the first shallow output feature is obtained. ​​​Shallow feature matrix A first shallow fused feature is obtained by using a 1×1 convolutional sub-layer for convolutional fusion. Then, without changing the input feature map resolution and number of channels, the first shallow output feature is further processed by a second 3×3 convolutional sub-layer using a 1×1 convolutional sub-layer and a stride of 1 to extract detailed features, resulting in a second shallow output feature. This second shallow output feature is then directly added to the first shallow output feature and fused to obtain the final feature with the desired size. The second shallow feature matrix Using the above principles, for dimensions of The second shallow feature matrix Processing yields dimensions of The third shallow feature matrix Using the above principles, for dimensions of The third shallow feature matrix Processing yields dimensions of The fourth shallow feature matrix .

[0025] The original image, after being processed by the constructed ENM-DSDCS encoding network, can yield deep features of DR images of complex aluminum alloy castings. With shallow features of multiple sizes, it can effectively extract overall and detailed features of significant and ambiguous defect areas, thus improving the segmentation capability of defect areas.

[0026] Based on the constructed DNM-ABGF decoding network, bidirectional guided feature fusion decoding is performed on the deep feature matrix and the shallow feature matrix to obtain the defect region segmentation image; Specifically, to better integrate the acquired features and multi-scale information, thereby improving the segmentation accuracy and efficiency of fuzzy defect regions, a DNM-ABGF decoding network is constructed, such as... Figure 5 As shown, the NM-ABGF decoding network includes shallow enhancement branches, downsampling branches, upsampling branches, deep enhancement branches, and attention enhancement branches; Multiple shallow feature matrices of different sizes are processed by shallow enhancement branches to increase features and adjust channels to obtain shallow feature enhancement matrices of different sizes. Deep feature matrices are processed by upsampling branches to adjust feature map resolution to obtain deep feature adjustment matrices. The shallow feature enhancement matrices and deep feature adjustment matrices are then recombined and fused to obtain the first recombined and fused feature layer. The shallow feature matrices of multiple sizes are respectively subjected to image resolution adjustment through the downsampling branch to obtain shallow feature adjustment matrices, the deep feature matrix is subjected to feature enhancement processing and channel adjustment through the deep enhancement branch to obtain a deep enhanced feature matrix, and the shallow feature adjustment matrix and the deep feature adjustment matrix are recombined and fused to obtain a second recombined and fused feature layer; The second recombined and fused feature layer and the first recombined and fused feature layer are fused through the attention enhancement branch to obtain a fused feature layer WxHxC, i.e. a segmentation image of the defect area, as shown in Figure 6 .

[0027] The deep enhancement branch and the shallow enhancement branch each include a depth convolution layer (3x3 DWConv), a residual layer (Residual Block) and a 1x1 convolution layer (1x1Conv) connected in sequence; the upsampling branch and the downsampling branch each include a convolution layer (3x3Conv), a residual layer (Residual Block) and a 1x1 convolution layer (1x1Conv) connected in sequence, and the convolution layer (3x3Conv) has a step of 2; the attention enhancement branch includes a convolution layer (3x3Conv), an attention enhancement layer (CBAM) and a convolution layer (3x3Conv) connected in sequence; Specifically, the first shallow feature matrix , the second shallow feature matrix , the third shallow feature matrix and the fourth shallow feature matrix are respectively enhanced through the depth convolution layer and the residual layer in the shallow enhancement branch, and then subjected to channel adjustment through the 1x1 convolution layer to obtain shallow feature enhancement matrices; the deep feature is subjected to downsampling adjustment of the feature map resolution through a convolution layer in the upsampling branch, then subjected to feature enhancement using a residual layer, and finally subjected to channel adjustment using a 1x1 convolution layer, and processed through a ReLU activation function to obtain a deep feature adjustment matrix; the shallow feature enhancement matrix and the deep feature adjustment matrix are multiplied in rows to be recombined and fused to obtain a first recombined and fused feature layer; The first shallow feature matrix , the second shallow feature matrix , the third shallow feature matrix and the fourth shallow feature matrix are respectively subjected to downsampling adjustment of the feature map resolution through a convolution layer in the downsampling branch, then subjected to feature enhancement using a residual layer, and finally subjected to channel adjustment using a 1x1 convolution layer to obtain shallow feature adjustment matrices; the deep feature The deep layer feature enhancement matrix is obtained by processing through a 1*1 convolution layer after enhancement through a deep convolution layer and a residual layer in a deep layer enhancement branch, and then through a ReLU activation function; the shallow layer feature adjustment matrix and the deep layer feature adjustment matrix are multiplied to recombine and fuse, and then the resolution of the feature map is adjusted through a 2*2 up sampling layer (2*2 Upsarmple) to obtain a plurality of second recombined and fused feature map layers; After adding and fusing the first recombined and fused feature map layer and the second recombined and fused feature map layer, a convolution layer is used for channel adjustment, a BN ReLU activation function is used for processing, then a feature attention enhancement layer is used for feature attention enhancement, then a convolution layer is used for channel adjustment of the feature map processed by the attention enhancement layer, and a BN ReLU activation function is used for processing again, to obtain a fusion feature layer W*H*C, that is, a defect region segmentation image.

[0028] Deep layer feature And the shallow layer features of multiple sizes can be effectively fused through the constructed DNM-ABGF decoding network, so as to improve the defect fuzzy area segmentation precision and obtain a high-precision defect region segmentation image.

[0029] Compared with the existing defect region segmentation method, the complex aluminum alloy casting defect region segmentation method provided by the application adopts a network structure with a logical structure of "encoding-decoding", organically combines the two, and jointly constructs a "double-path encoding-fusion decoding" segmentation method, that is, the input complex aluminum alloy casting DR image is encoded by the ENM-DSDCS encoding network in a deep layer feature and a shallow layer feature double path, and then is input into the DNM-ABGF decoding network for bidirectional guided feature fusion decoding, so that the effective segmentation of the casting defect target is realized through the structure of double-path encoding to fusion decoding, compared with the existing defect region segmentation method, the detection time is basically the same, the accuracy and segmentation precision are higher, and the comprehensive performance is better.

[0030] The application provides a complex aluminum alloy casting defect region segmentation system for realizing the complex aluminum alloy casting defect region segmentation method. The image acquisition module is configured to acquire an original image of the complex aluminum alloy casting. The ENM-DSDCS encoding module is configured to extract deep layer features and shallow layer features of the original image based on the constructed ENM-DSDCS encoding network. The DNM-ABGF decoding module is configured to perform bidirectional guided feature fusion decoding on the deep features and the shallow features based on the constructed DNM-ABGF decoding network, to obtain a segmentation image of the defect area. The segmentation image output module is configured to output the defect area segmentation image.

[0031] In another embodiment of the present application, an electronic device is provided, which includes a processor and a memory. The memory is configured to store a computer program, and the computer program includes program instructions. The processor is configured to execute the program instructions stored in the computer storage medium. The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor is the computing core and control core of the terminal, and is suitable for implementing one or more instructions, and is specifically suitable for loading and executing one or more instructions to implement a corresponding method flow or a corresponding function. The processor in the embodiments of the present application can be used for the operation of the complex aluminum alloy casting defect area segmentation method, including: The deep feature matrix and the shallow feature matrix of the original image are extracted based on the constructed ENM-DSDCS encoding network, and bidirectional guided feature fusion decoding is performed on the deep feature matrix and the shallow feature matrix based on the constructed DNM-ABGF decoding network, to obtain a defect area segmentation image.

[0032] In still another embodiment of the present application, the present application also provides a storage medium, specifically a computer readable storage medium (Memory), which is a memory device in the terminal device, used for storing programs and data. It can be understood that the computer readable storage medium herein can include the built-in storage medium in the terminal device, and of course can also include the expansion storage medium supported by the terminal device. The computer readable storage medium provides a storage space, which stores the operating system of the terminal. Moreover, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and the instructions can be one or more computer programs (including program codes). It should be noted that the computer readable storage medium herein can be a high-speed RAM memory, or a non-volatile memory such as at least one disk memory.

[0033] The one or more instructions stored in the computer readable storage medium can be loaded and executed by the processor to implement the corresponding steps of the complex aluminum alloy casting defect region segmentation method in the above embodiments; the one or more instructions in the computer readable storage medium are loaded and executed by the processor as follows: Based on the constructed ENM-DSDCS encoding network, the deep feature matrix and the shallow feature matrix of the original image are extracted respectively; based on the constructed DNM-ABGF decoding network, the deep feature matrix and the shallow feature matrix are subjected to bidirectional guided feature fusion decoding to obtain a defect region segmentation image.

[0034] Please refer to Figure 8 The terminal device is a computer device, and the computer device 60 of this embodiment includes a processor 61, a memory 62, and a computer program 63 stored in the memory 62 and executable on the processor 61. When the computer program 63 is executed by the processor 61, the method for calculating the fluid composition in the reservoir stimulation wellbore in the embodiment is implemented. To avoid repetition, details are not described here. Alternatively, when the computer program 63 is executed by the processor 61, the functions of each model / unit in the system for calculating the fluid composition in the reservoir stimulation wellbore in the embodiment are implemented. To avoid repetition, details are not described here.

[0035] The computer device 60 can be a desktop computer, a notebook computer, a palm computer, and a cloud server, etc. The computer device 60 can include, but is not limited to, the processor 61 and the memory 62. Those skilled in the art can understand that the computer device 60 can further include other components, and the components are not limited to the components shown in the figure. Figure 8 The computer device 60 is only an example and does not constitute a limitation on the computer device 60, and can include more or fewer components than shown, or combine certain components, or different components, for example, the computer device can also include an input / output device, a network access device, a bus, etc.

[0036] The processor 61 can be a central processing unit (CPU), and can also be other general-purpose processors, central processing units, graphics processing units, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic components, quantum computing-based data processing logic components, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0037] The memory 62 can be an internal storage unit of the computer device 60, such as a hard disk or a memory of the computer device 60. The memory 62 can also be an external storage device of the computer device 60, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device 60.

[0038] Further, the memory 62 can include both an internal storage unit and an external storage device of the computer device 60. The memory 62 is used to store computer programs and other programs and data required by the computer device. The memory 62 can also be used to temporarily store data that has been output or will be output.

[0039] Any reference to memory, database, or other media herein includes at least one of volatile and non-volatile memory. Non-volatile memory can include read-only memory (ROM), tape, floppy disks, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, and the like. Volatile memory can include random access memory (RAM), external cache memory, and the like. By way of illustration, and not limitation, RAM can be a variety of forms, such as static random access memory (SRAM), dynamic random access memory (DRAM), or the like.

[0040] A database referred to in embodiments provided herein can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a blockchain, without being limited thereto. A processor referred to in embodiments provided herein can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, and the like, without being limited thereto.

[0041] Referring to Figure 9 , the terminal device is a chip, and the chip 600 of the embodiment includes a processor 622, the number of which can be one or more, and a memory 632 for storing a computer program executable by the processor 622. The computer program stored in the memory 632 can include one or more modules each corresponding to a set of instructions. In addition, the processor 622 can be configured to execute the computer program to perform the generalizable monocular absolute depth map estimation method described above.

[0042] In addition, the chip 600 can further include a power supply component 626 and a communication component 650, the power supply component 626 can be configured to perform power management of the chip 600, and the communication component 650 can be configured to implement communication of the chip 600, such as wired or wireless communication. In addition, the chip 600 can further include an input / output interface 658. The chip 600 can operate based on an operating system stored in the memory 632.

[0043] The foregoing merely illustrates the principles of the application and application of its leading features. This application is not limited to the exact details shown above, and various modifications can be made to the embodiments described without departing from the spirit or scope of the application. Accordingly, the embodiments are to be considered as illustrative and not restrictive, and the scope of the application is to be determined not with reference to the above description but with reference to the appended claims, and their equivalents. No admission is made that any reference constitutes prior art. It is the combination of elements that is claimed. The scope of the technology is thus those limitations as indicated by the appended claims.

[0044] Furthermore, it should be appreciated that a single independent technical solution is not contained in each embodiment, and the description of the specification is only for the sake of clarity, and the skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can be combined appropriately to form other embodiments that can be understood by the skilled in the art. The above is only to illustrate the technical idea of the present application, and cannot limit the protection scope of the present application. Any modification made on the basis of the technical idea of the present application and technical solution falls within the protection scope of the claims of the present application.

Claims

1. A method of complex aluminum alloy casting defect region segmentation, characterized by, The method comprises the following steps: extracting deep features and shallow features of the original image based on the constructed ENM-DSDCS encoding network; performing bidirectional guided feature fusion decoding on the deep features and the shallow features based on the constructed DNM-ABGF decoding network to obtain a defect region segmentation image.

2. The method of claim 1, wherein, The constructed ENM-DSDCS encoding network comprises a deep feature structure network DFN and a shallow feature structure network SFN. The deep feature structure network DFN comprises a first deformable convolution composite layer D_Conv1, a second deformable convolution composite layer D_Conv2, a third deformable convolution composite layer D_Conv3, a fourth deformable convolution composite layer D_Conv4 and a fifth deformable convolution composite layer D_Conv5 which are sequentially connected in series. The shallow feature structure network SFN comprises a first shallow convolution composite layer S_Conv1, a second shallow convolution composite layer S_Conv2, a third shallow convolution composite layer S_Conv3 and a fourth shallow convolution composite layer S_Conv4 which are sequentially connected in series.

3. The method of claim 2, wherein the complex aluminum alloy casting defect region segmentation method is characterized by, The method of extracting deep features and shallow features of the original image based on the constructed ENM-DSDCS encoding network comprises: obtaining an original image of a complex aluminum alloy casting defect region, and performing shape processing on the original image to obtain a standard three-channel image; The standard three-channel image is subjected to feature extraction by a deep feature structure network (DFN) to obtain a deep feature matrix with a size of . ​ performing feature extraction on the standard three-channel image through the shallow feature structure network SFN to obtain a plurality of scale shallow feature matrices.

4. The method of claim 3, wherein the complex aluminum alloy casting defect region segmentation method is characterized by, The standard three-channel image is subjected to feature extraction by a deep feature structure network (DFN) to obtain a deep feature matrix of size , comprising:​ performing receptive field lifting and feature dimension normalization processing on the standard three-channel image through the first deformable convolution composite layer D_Conv1 to obtain a first deep feature map; performing larger receptive field coverage and normalized feature dimension on the first deep feature map through the second deformable convolution composite layer D_Conv2, expanding the number of channels to realize feature dimension lifting, then compressing the number of channels, normalizing the interlayer output and performing regularization processing to obtain a second deep feature map; performing larger receptive field coverage and normalized feature dimension on the second deep feature map through the third deformable convolution composite layer D_Conv3 after downsampling, expanding the number of channels to realize feature dimension lifting, then compressing the number of channels, normalizing the interlayer output and performing regularization processing to obtain a third deep feature map; performing larger receptive field coverage and normalized feature dimension on the third deep feature map through the fourth deformable convolution composite layer D_Conv4 after downsampling, repeating the following operations nine times, expanding the number of channels to realize feature dimension lifting, then compressing the number of channels, normalizing the interlayer output and performing regularization processing to obtain a fourth deep feature map; After the fourth deep feature map is down-sampled by the fifth deformable convolution composite layer D_Conv5, a larger receptive field and normalized feature dimension are covered, the channel number is expanded to realize feature dimension increase, the channel number is compressed, the inter-layer output is normalized and regularized, and a deep feature matrix with a size of is obtained.

5. The method of claim 1, wherein, performing feature extraction on the standard three-channel image through the shallow feature structure network SFN, comprising: The standard three-channel image is subjected to feature extraction by the first shallow convolution composite layer S_Conv1 to obtain a first shallow feature matrix with a size of . ​ The first shallow feature matrix with a size of The second shallow feature matrix with a size of is extracted by the second shallow convolution composite layer S_Conv2, and a second shallow feature matrix with a size of ;​ The second shallow feature matrix with a size of The third shallow feature matrix with a size of is obtained by feature extraction through the third shallow convolution composite layer S_Conv3. ;​ The size is The third shallow feature matrix Feature extraction is performed using the fourth shallow convolutional composite layer S_Conv4, resulting in a size of... The fourth shallow feature matrix ; statistically processing the first shallow feature matrix , the second shallow feature matrix , the third shallow feature matrix , and the fourth shallow feature matrix to obtain the shallow feature matrix of multiple scales.

6. The method of claim 1, wherein, The DNM-ABGF decoding network comprises: a shallow enhancement branch and a down-sampling branch for processing the shallow features; an up-sampling branch and a deep enhancement branch for processing the deep features; an attention enhancement branch for recombining and fusing the features.

7. The method of claim 1, wherein The method of performing bidirectional guided feature fusion decoding on the deep features and the shallow features based on the constructed DNM-ABGF decoding network comprises: The shallow feature matrix is subjected to feature enhancement processing and channel adjustment through the shallow enhancement branch to obtain a shallow feature enhancement matrix; The deep feature matrix is subjected to feature map resolution adjustment through the up-sampling branch to obtain a deep feature adjustment matrix; The shallow feature enhancement matrix and the deep feature adjustment matrix are recombined and fused to obtain a first recombined and fused feature map layer; The shallow feature matrix is subjected to image resolution adjustment through the down-sampling branch to obtain a shallow feature adjustment matrix; The deep feature matrix is subjected to feature enhancement processing and channel adjustment through the deep enhancement branch to obtain a deep enhanced feature matrix; The shallow feature adjustment matrix and the deep feature adjustment matrix are recombined and fused to obtain a second recombined and fused feature map layer; The second recombined and fused feature layer and the first recombined and fused feature layer are fused through the attention enhancement branch to obtain a segmentation image of the defect area.

8. A complex aluminum alloy casting defect region segmentation system, implementing the complex aluminum alloy casting defect region segmentation method of any one of claims 1-7, characterized in that, The method comprises the following steps: An image acquisition module is configured to acquire an original image of a complex aluminum alloy casting; An ENM-DSDCS encoding module is configured to extract deep features and shallow features of the original image based on a constructed ENM-DSDCS encoding network; A DNM-ABGF decoding module is configured to perform bidirectional guided feature fusion decoding on the deep features and the shallow features based on a constructed DNM-ABGF decoding network to obtain a defect area segmentation image; A segmented image output module is configured to output the defect area segmentation image.

9. A computer apparatus, comprising: A computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the defect area segmentation method of the complex aluminum alloy casting according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the defect area segmentation method of the complex aluminum alloy casting according to any one of claims 1-7.

Images