Flange accessory forging defect early warning system and method based on digital twinning

Abstract

The invention discloses a flange accessory forging defect early warning system and method based on digital twinning. The method comprises the steps that pre-processed flange accessory surface multi-view image data is obtained; an improved Mask2Former segmentation model is constructed on the basis of the flange accessory forging space-time digital twinborn database; inputting the preprocessed flange accessory surface multi-view image data into an improved Mask2Former segmentation model, and outputting a first segmentation result mask; forming a physical state vector; constructing a visual physical multi-modal fusion vector; inputting the visual physical multi-modal fusion vector into a fusion discrimination network, and outputting a second segmentation result mask; and performing post-processing operation on the second segmentation result mask to obtain an accurate defect mask set, generating a flange accessory forging multi-class defect identification result, and performing flange accessory forging defect early warning. According to the method, the fineness of the segmentation boundary is improved, and the problems of large-scale defect boundary roughness and small-scale defect missing detection are remarkably reduced.

Description

Technical Field

[0001] This invention relates to the field of flange fittings technology, and in particular to a flange fittings forging defect early warning system and method based on digital twins. Background Technology

[0002] With the continuous advancement of intelligent manufacturing and digital transformation, automatic detection of forging defects in key industrial components of flange fittings has gradually become an important means to ensure product quality and production efficiency.

[0003] Currently, machine vision methods based on convolutional neural networks are widely used in industrial surface defect recognition tasks. However, they usually rely on convolutional kernels of fixed size or preset anchor frame structures to extract features and segment the target area. In the flange forging process, the types of defects are rich and the scale range is extremely wide. There are both extremely fine cracks and large-scale defects such as missing parts and folds. Existing convolutional neural networks often exhibit low segmentation accuracy and severe boundary adhesion when dealing with multi-scale and irregular defects, making it difficult to recognize both small-scale details and large-scale overall areas.

[0004] Most existing visual defect detection methods rely solely on single-frame image features for discrimination, making it difficult to distinguish between genuine defects and pseudo-defects such as water stains and oxide scale. Due to the complex environmental factors of high temperatures, debris, and process residues at the forging site, relying solely on image signals can easily lead to a high false alarm rate, resulting in a large amount of manual verification still required in subsequent quality inspection stages. This severely limits the practical application value of automated intelligent detection systems. Although some studies have introduced multimodal fusion or temporal analysis, they are mostly based on shallow feature splicing and lack effective coordination with deep spatiotemporal information such as process parameters and physical states, making it difficult to achieve logical verification and robust discrimination of the physical causes of defects. Summary of the Invention

[0005] One objective of this invention is to propose a digital twin-based early warning system and method for forging defects in flange fittings. This invention not only improves the fineness of the segmentation boundary, but also significantly reduces the problems of rough large-scale defect boundaries and missed detection of small-scale defects.

[0006] A method for early warning of forging defects in flange fittings based on digital twins according to an embodiment of the present invention includes: Collect multi-source process parameter data and multi-view image data of flange fitting forging to generate a spatiotemporal digital twin database of flange fitting forging. Perform image preprocessing on the multi-view image data of flange fitting surface to obtain preprocessed multi-view image data of flange fitting surface. An improved Mask2Former segmentation model was constructed based on a spatiotemporal digital twin database of flange fitting forging. The preprocessed multi-view image data of the flange fitting surface is input into the improved Mask2Former segmentation model. Adaptive query vectors are used to generate multi-scale defect candidate masks. The defect candidate masks are optimized by dynamic attention module, and the first segmentation result mask is output. Using the spatiotemporal digital twin database of flange fitting forging, the multi-source process parameter data of flange fitting forging within the spatial location and time period corresponding to the mask of the first segmentation result are retrieved to form a physical state vector; In the spatiotemporal digital twin database of flange fitting forging, the correspondence between the first segmentation result mask and the physical state vector is recorded to construct a visual-physical multimodal fusion vector; The visual-physical multimodal fusion vector is input into the fusion discriminant network, and a fusion discriminant vector is generated through a cross-modal attention mechanism. The first segmentation result mask is then used to verify its authenticity and determine its defect category, and the second segmentation result mask is output. Post-processing operations are performed on the second segmentation result mask to obtain an accurate defect mask set, and multi-type defect identification results of flange fitting forging are generated, and defect warning of flange fitting forging is performed.

[0007] Optionally, the construction of the spatiotemporal digital twin database for flange fitting forging includes: Multi-source process parameter data and multi-view image data of flange fitting forging are collected. The multi-source process parameter data and multi-view image data of flange fitting surface are synchronously mapped by a unified timestamp and spatial coordinates to generate a spatiotemporal digital twin database of flange fitting forging. Image preprocessing is performed on the multi-view image data of flange fitting surface, including noise reduction, illumination equalization, reflection suppression and contrast enhancement, to obtain preprocessed multi-view image data of flange fitting surface. The preprocessed multi-view image data of flange fitting surface is then stored in the spatiotemporal digital twin database of flange fitting forging, resulting in the final spatiotemporal digital twin database of flange fitting forging.

[0008] Optionally, the improved Mask2Former segmentation model includes: Read the multi-view image feature tensor of the flange component surface and the multi-source process parameter matrix of flange component forging from the spatiotemporal digital twin database of flange component forging, calculate the temperature-stress joint weight coefficient, use the temperature-stress joint weight coefficient as the modulation factor, multiply it with each channel of the learnable mapping matrix and the image feature tensor to obtain an adaptive query vector set. In each layer of the dynamic attention module, the temperature gradient and stress gradient are calculated based on the input features of this layer. The normalized scaling factor of the temperature gradient and stress gradient is used, combined with the adaptive query vector of the previous layer, the learnable transformation matrix and the image features of the current layer, and then the current layer adaptive query vector set is updated in a weighted manner after being processed by the activation function. In the dynamic attention module, for each adaptive query vector and each key vector, the normalized ratio of their dot product to the vector dimension is calculated, and combined with the forging stress difference modulation factor, the normalized attention score is obtained. Through a multi-scale feature pyramid structure, three sets of masks with different spatial resolutions are output, corresponding to small-scale, long-scale and global-scale masks respectively. For each scale mask, multi-scale gating weights are calculated. Based on the multi-scale gating weights, the masks of each scale are weighted and summed to obtain the multi-scale fused mask.

[0009] Optionally, the first segmentation result mask includes: The preprocessed multi-view image data of the flange fitting surface is converted into an image input tensor. The image input tensor is then input into the backbone encoding network of the dynamic Mask2Former segmentation model to obtain a set of multi-scale semantic feature maps. The adaptive query vector set and the multi-scale semantic feature map set are interactively fused at each scale level through the Transformer decoder to obtain mask candidate features for each adaptive query vector on each scale feature map. The attention score of each mask candidate feature at the corresponding spatial location is calculated based on the dynamic attention module, and the normalized optimized mask is calculated. For each mask spatial location, the forging temperature parameters and forging stress parameters corresponding to the same spatial location and time period are obtained by calling the spatiotemporal digital twin database of flange fitting forging, and the forging sensitivity modulation factor of the corresponding spatial location is calculated. At each spatial location, the normalized optimized mask value is multiplied by the forging sensitivity modulation factor to obtain the modulated mask features; The modulated mask features of all scale levels and all adaptive query vectors are fused using a multi-scale gating fusion strategy to obtain the final first segmentation result mask.

[0010] Optionally, the retrieval of multi-source process parameter data for flange fitting forging within the spatial location and time period corresponding to the first segmentation result mask includes: For each mask in the first segmentation result mask set, based on the spatial coordinate information and time number recorded by the mask, retrieve the data entry that completely matches the spatial coordinate information and time number in the flange fitting forging spatiotemporal digital twin database, and obtain the forging temperature parameters, forging pressure parameters and forging strain parameters corresponding to the mask spatial position and time number; For each mask spatial location, the continuous forging temperature parameters within a fixed-length time window before and after the time number are combined into a temperature parameter sequence, and the continuous forging pressure parameters are combined into a pressure parameter sequence. For a temperature parameter sequence, calculate the average temperature feature, the maximum temperature gradient feature, and the temperature variance feature to form a set of temperature parameter features; For the pressure parameter sequence, calculate the maximum pressure characteristic and the pressure change rate characteristic to form a pressure parameter characteristic set. For the strain parameter sequence, calculate the cumulative strain characteristics and the characteristics of the amplitude of drastic strain changes to form a strain parameter characteristic set; The temperature parameter feature set, pressure parameter feature set, and strain parameter feature set are concatenated in sequence to form the physical state vector of the first segmentation result mask at the spatial location and time number.

[0011] Optionally, constructing the visual-physical multimodal fusion vector includes: For each mask in the first segmentation result mask set, extract its corresponding mask visual feature vector from the output of the dynamic Mask2Former segmentation model; The visual feature vector and the physical state vector of the mask are concatenated in a preset order to obtain a visual-physical multimodal fusion vector.

[0012] Optionally, the second segmentation result mask includes: Dimensional standardization is performed on each visual-physical multimodal fusion vector to obtain dimensionless visual feature components and dimensionless physical state feature components; Based on dimensionless physical state characteristic components, the forging consistency gating coefficient is calculated through linear transformation and nonlinear activation function; Based on dimensionless physical state characteristic components, the pseudo-defect suppression gating coefficient is calculated through another set of linear transformations and nonlinear activation functions. In the fusion discriminant network, the dimensionless visual feature components are mapped to queries, and the physical state feature components are mapped to keys respectively. By combining the forging consistency gating coefficient and the false defect suppression gating coefficient, cross-modal attention coefficients that reflect real defect evidence and suppress false defect evidence are obtained. Based on the cross-modal attention coefficient, linear transformations are applied to the dimensionless visual feature components to obtain the visual principal component. At the same time, the physical state feature components are weighted and aggregated using the cross-modal attention coefficient to obtain the physical modulation component. The visual principal component and the physical modulation component are added together to form a fusion discriminant vector. Based on the fusion discriminant vector, a linear transformation and activation function are used to calculate the authenticity test score. When the authenticity test score is higher than the preset authenticity threshold, the corresponding first segmentation result mask is determined to be a real defect; otherwise, it is determined to be a false defect. Based on the fused discriminant vector, a fully connected layer and normalization operation are used to calculate the defect category probability distribution. The category corresponding to the largest component of the defect category probability distribution is the defect category discrimination result of the corresponding mask. For the first segmentation result mask that passes the authenticity test, the amplitude of the mask pixel probability response is recalibrated based on the authenticity test score and the maximum component of the defect category probability distribution to obtain the second segmentation result mask.

[0013] Optionally, the generation of multiple defect identification results for flange fitting forging includes: Edge refinement processing is performed on each second segmentation result mask to obtain an edge refinement mask; Perform connected component extraction and connected component integrity verification on the edge refinement mask, and output a complete connected component mask set; Scale consistency verification is performed on the complete mask set of connected components to obtain the accurate defect mask set; Geometric feature information is extracted from each mask in the precise defect mask set. Based on the geometric feature information, the identification results of multiple types of defects in flange fitting forging are output, and early warning of flange fitting forging defects is provided.

[0014] Optionally, the geometric feature information includes: The defect location is the geometric center coordinate of the spatial coordinates of all pixels within the mask; The defect area is the product of the total number of pixels in the mask and the actual physical area factor of a single pixel. The defect shape index is composed of aspect ratio, principal axis direction and compactness. The defect shape index reflects the irregularity of the defect shape. Defect confidence score is the product of the authenticity test score and the largest component of the defect category probability distribution. It is used to quantify the credibility of defects in the physical behavior and visual response throughout the flange fitting forging process.

[0015] A digital twin-based early warning system for forging defects in flange fittings, used to implement a digital twin-based early warning method for forging defects in flange fittings, including: The spatiotemporal twin database construction module is used to collect multi-source process parameter data and multi-view image data of flange fittings forging, establish a spatiotemporal digital twin database of flange fittings forging, and obtain pre-processed image data. An improved Mask2Former segmentation module was developed to construct an improved Mask2Former segmentation model with adaptive query vectors and dynamic attention mechanism based on a spatiotemporal digital twin database of flange fitting forging. The defect candidate mask generation and optimization module is used to input the preprocessed image data into the improved Mask2Former segmentation model and output the first segmentation result mask. The physical state vector extraction module is used to retrieve multi-source process parameter data of the spatial location and time period corresponding to the mask of the first segmentation result using the spatiotemporal digital twin database of flange fitting forging, and form a physical state vector. The multimodal fusion module is used to establish the correspondence between the first segmentation result mask and the physical state vector in the spatiotemporal digital twin database of flange fitting forging, and to construct a visual-physical multimodal fusion vector. The fusion discrimination and segmentation optimization module is used to input the visual-physical multimodal fusion vector into the fusion discrimination network, perform authenticity verification and defect category discrimination on the first segmentation result mask, and output the second segmentation result mask; The early warning module is used to perform post-processing on the mask of the second segmentation result to obtain an accurate defect mask set, output the identification results of multiple types of defects in flange fitting forging, and realize defect early warning.

[0016] The beneficial effects of this invention are: This invention proposes an improved Mask2Former segmentation network based on the synergistic effect of dynamic adaptive query vectors, dynamic attention modules, and multi-scale mask fusion modules. The process parameters and physical state features in the spatiotemporal digital twin database of flange fitting forging are used as network modulation factors, which effectively improves the adaptive segmentation capability for extremely fine cracks and large-scale defects. Through adaptive query and multi-modal modulation, not only is the fineness of the segmentation boundary improved, but the problems of rough boundary of large-scale defects and missed detection of small-scale defects are also significantly reduced.

[0017] This invention unifies and standardizes visual features with multidimensional physical state features from a spatiotemporal digital twin database of flange fitting forging. It also dynamically adjusts the cross-modal attention mechanism through forging consistency gating coefficient and false defect suppression gating coefficient to achieve refined modeling of defect authenticity and active suppression of false defects. It can weight the credibility of visual discrimination results according to physical state features and effectively distinguish false defects.

[0018] This invention effectively filters out misjudgment masks with abnormal area or inconsistent deformation by verifying the consistency of physical scale. This ensures that the final output of defect location, area, category, and confidence index not only conforms to the visual segmentation rules but also meets the physical tolerance standards of the forging process. This guarantees the practical usability of the identification results in quality reports and production feedback and effectively improves the accuracy of early warning. Attached Figure Description

[0019] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of a forging defect early warning system and method for flange fittings based on digital twin proposed in this invention; Figure 2This is a schematic diagram of the network structure framework of the improved Mask2Former segmentation model in the early warning system and method for forging defects of flange fittings based on digital twin proposed in this invention. Figure 3 This invention presents a multi-class defect early warning framework and visualization diagram for an improved Mask2Former segmentation model in a digital twin-based flange fitting forging defect early warning system and method. Detailed Implementation

[0020] Example 1: Reference Figures 1-3 A method for early warning of forging defects in flange fittings based on digital twins, comprising: Collect multi-source process parameter data and multi-view image data of flange fitting forging to generate a spatiotemporal digital twin database of flange fitting forging. Perform image preprocessing on the multi-view image data of flange fitting surface to obtain preprocessed multi-view image data of flange fitting surface. In this embodiment, the construction of the spatiotemporal digital twin database for flange fitting forging includes: Multi-source process parameter data and multi-view image data of flange fitting forging are collected. The multi-source process parameter data and multi-view image data of flange fitting surface are synchronously mapped by a unified timestamp and spatial coordinates to generate a spatiotemporal digital twin database of flange fitting forging. Image preprocessing is performed on the multi-view image data of flange fitting surface, including noise reduction, illumination equalization, reflection suppression and contrast enhancement, to obtain preprocessed multi-view image data of flange fitting surface. The preprocessed multi-view image data of flange fitting surface is then stored in the spatiotemporal digital twin database of flange fitting forging, resulting in the final spatiotemporal digital twin database of flange fitting forging.

[0021] An improved Mask2Former segmentation model was constructed based on a spatiotemporal digital twin database of flange fitting forging. In this embodiment, the Mask2Former segmentation model is improved, including: Read the multi-view image feature tensor of the flange component surface and the multi-source process parameter matrix of flange component forging from the spatiotemporal digital twin database of flange component forging, calculate the temperature-stress joint weight coefficient, use the temperature-stress joint weight coefficient as the modulation factor, multiply it with each channel of the learnable mapping matrix and the image feature tensor to obtain an adaptive query vector set. In Example 1, a multi-view image feature tensor of the flange component surface is read from the spatiotemporal digital twin database of flange component forging. This multi-view image feature tensor is a three-dimensional tensor composed of spatial resolution and channel dimension. Simultaneously, a multi-source process parameter matrix of the flange component forging is read. This matrix includes forging temperature parameters, forging stress parameters, and forging strain parameters that correspond one-to-one with the spatial position of the multi-view image feature tensor of the flange component surface. Using the forging temperature parameter as the numerator and the sum of the forging temperature parameter, forging stress parameter, and forging strain parameter as the denominator, the feature tensor for each spatial position is calculated. The temperature-stress joint weighting coefficient for each spatial location is used as a modulation factor. It is then multiplied point-by-point with the value of each channel of the multi-view image feature tensor of the flange fitting surface at the same spatial location to form a modulated image feature tensor. The modulated image feature tensor is then input into a learnable mapping matrix, which is used as a parameter to be trained. This matrix is ​​used to map the modulated image feature tensor from the original feature space to the adaptive query vector space. After mapping all spatial locations, the mapping results are aggregated according to the preset number of query vectors to obtain an adaptive query vector set.

[0022] In each layer of the dynamic attention module, the temperature gradient and stress gradient are calculated based on the input features of this layer. The normalized scaling factor of the temperature gradient and stress gradient is used, combined with the adaptive query vector of the previous layer, the learnable transformation matrix and the image features of the current layer, and then the current layer adaptive query vector set is updated in a weighted manner after being processed by the activation function. In Example 1, at each layer of the dynamic attention module, based on the multi-view image feature tensor of the flange fitting surface of the current layer, the forging temperature change corresponding to each position on the flange fitting surface is calculated as the temperature gradient, and the forging stress change is calculated as the stress gradient, using spatial coordinates and forging time information. The temperature gradient is the difference between the forging temperature of the spatial position at the current moment and the previous moment, and the stress gradient is the difference between the forging stress of the spatial position at the current moment and the previous moment. The temperature gradient and stress gradient are summed and normalized to obtain the normalization coefficient of the temperature gradient and the normalization coefficient of the stress gradient of the current layer. The normalization coefficient of the temperature gradient is weighted and added to the adaptive query vector set of the previous layer. At the same time, the normalization coefficient of the stress gradient is weighted and added to the result of the current layer image features after processing by the learnable transformation matrix and activation function. The sum of the two items is used to obtain the updated adaptive query vector set of the current layer.

[0023] In the dynamic attention module, for each adaptive query vector and each key vector, the normalized ratio of their dot product to the vector dimension is calculated, and combined with the forging stress difference modulation factor, the normalized attention score is obtained. By using the absolute value of the difference between the forging stress at the corresponding positions of the adaptive query vector and the key vector, and through exponential mapping, a forging stress difference modulation factor is obtained, which is used to adjust the stress sensitivity of different spatial regions.

[0024] Through a multi-scale feature pyramid structure, three sets of masks with different spatial resolutions are output, corresponding to small-scale, long-scale and global-scale masks respectively. For each scale mask, multi-scale gating weights are calculated. The gating weights are determined by the area of ​​the corresponding scale mask and the scale-specific learnable parameters. After exponential normalization, they reflect the weight contribution of each scale mask in multi-scale fusion. The scale mask area is the sum of the number of pixels in the binary region of the corresponding mask, and the scale-specific learnable parameters are the weight parameters trained separately for each scale mask.

[0025] Based on the multi-scale gating weights, the masks of each scale are weighted and summed to obtain the multi-scale fused mask.

[0026] The multi-scale fusion mask, as the output of the current decoding layer, is passed to the next decoding layer. The multi-scale fusion mask integrates information from small-scale, long-scale, and global-scale masks to improve the adaptive segmentation capability for various defects in flange fittings forging.

[0027] The preprocessed multi-view image data of the flange fitting surface is input into the improved Mask2Former segmentation model. Adaptive query vectors are used to generate multi-scale defect candidate masks. The defect candidate masks are optimized by dynamic attention module, and the first segmentation result mask is output. In this embodiment, the first segmentation result mask includes: The preprocessed multi-view image data of the flange fitting surface is converted into an image input tensor. The image input tensor is then input into the backbone encoding network of the dynamic Mask2Former segmentation model to obtain a set of multi-scale semantic feature maps. The image input tensor has a height of H, a width of W, and a number of channels of C. Each feature map in the multi-scale semantic feature map set has a specific spatial resolution and channel dimension.

[0028] The adaptive query vector set and the multi-scale semantic feature map set are interactively fused at each scale level through the Transformer decoder to obtain mask candidate features for each adaptive query vector on each scale feature map. In Example 1, the set of adaptive query vectors is used as the input query sequence of the Transformer decoder. The semantic feature map corresponding to the scale level is flattened into a key sequence and a value sequence. The Transformer decoder performs a fully connected computation on each adaptive query vector and all spatial location features in the feature map of that scale through an attention mechanism to obtain the attention weight between each adaptive query vector and each spatial location. Using the attention weight between each adaptive query vector and each spatial location, the feature vectors of all spatial locations in the corresponding scale feature map are weighted and summed. The weighted summation result is used as the spatial response value of the candidate mask feature of the query vector at the corresponding scale level. The weighted summation operation is performed on all spatial locations to obtain the candidate mask features indexed by spatial coordinates.

[0029] The attention score of each mask candidate feature at the corresponding spatial location is calculated based on the dynamic attention module. The value of each mask candidate feature at each spatial location is multiplied by the attention score at the spatial location as the numerator, and the attention scores of all spatial locations are summed as the normalized denominator to obtain the normalized optimized mask. The normalized optimized mask value for each spatial location is equal to the result of multiplying the mask candidate features of the spatial location by the attention score, divided by the sum of the attention scores for all spatial locations.

[0030] For each mask spatial location, the forging temperature parameters and forging stress parameters corresponding to the same spatial location and time period are obtained by calling the spatiotemporal digital twin database of flange fitting forging. The forging temperature parameters of the spatial location are used as the numerator, and the forging temperature parameters of the spatial location are added to the forging stress parameters to obtain the forging sensitivity modulation factor of the corresponding spatial location. In Example 1, by recording the spatial coordinate index of the mask's spatial position in the input tensor of the original image and the time tag corresponding to the multi-view image data of the preprocessed flange fitting surface, the data entries that completely correspond to the spatial coordinate index and time tag are retrieved in the spatiotemporal digital twin database of flange fitting forging based on the spatial coordinate index and time tag, and the forging temperature parameters and forging stress parameters are extracted from the data entries.

[0031] At each spatial location, the normalized optimized mask value is multiplied by the forging sensitivity modulation factor to obtain the modulated mask features; The modulated mask features of all scale levels and all adaptive query vectors are fused using a multi-scale gating fusion strategy to obtain the final first segmentation result mask.

[0032] The multi-scale gating fusion strategy involves multiplying the modulated mask features of the same query vector at all scale levels by the corresponding multi-scale gating weights and summing them. Each fused mask is then used as a member of the final first segmentation result mask set. The multi-scale gating weights are adaptively calculated based on the response intensity of each scale mask. The first segmentation result mask records the source multi-scale semantic feature map index, adaptive query vector index, spatial coordinate information, and time number.

[0033] Using the spatiotemporal digital twin database of flange fitting forging, the multi-source process parameter data of flange fitting forging within the spatial location and time period corresponding to the mask of the first segmentation result are retrieved to form a physical state vector; In this embodiment, the retrieval of multi-source process parameter data for flange fitting forging within the spatial location and time period corresponding to the first segmentation result mask includes: For each mask in the first segmentation result mask set, based on the spatial coordinate information and time number recorded by the mask, retrieve the data entry that completely matches the spatial coordinate information and time number in the flange fitting forging spatiotemporal digital twin database, and obtain the forging temperature parameters, forging pressure parameters and forging strain parameters corresponding to the mask spatial position and time number; The spatial coordinate information is the two-dimensional coordinate index of the first segmentation result mask in the image input tensor, and the time number is the time sequence identifier of the image frame during the forging process.

[0034] For each mask spatial location, the continuous forging temperature parameters within a fixed-length time window before and after the time number are combined into a temperature parameter sequence, and the continuous forging pressure parameters are combined into a pressure parameter sequence. The strain parameters of continuous forging form a strain parameter sequence. Each parameter sequence is arranged continuously in time, representing the physical evolution process of the flange fitting at the corresponding spatial position within the forging window period, forming temperature change trajectory, pressure change trajectory and strain change trajectory.

[0035] For a temperature parameter sequence, calculate the average temperature feature, the maximum temperature gradient feature, and the temperature variance feature to form a set of temperature parameter features; The average temperature feature is the sum of all temperature parameters in the temperature parameter sequence divided by the length of the temperature parameter sequence, used to reflect the overall temperature level at the corresponding location of the mask area; the maximum temperature gradient feature is the maximum absolute value of the difference between two adjacent temperature parameters in the temperature parameter sequence, used to reflect the severity of temperature changes; the temperature variance feature is the sum of the squares of the differences between all temperature parameters in the temperature parameter sequence and the average temperature feature divided by the length of the temperature parameter sequence, used to reflect the temperature stability at that location.

[0036] For the pressure parameter sequence, calculate the maximum pressure characteristic and the pressure change rate characteristic to form a pressure parameter characteristic set. The maximum pressure characteristic is the maximum pressure parameter value in the pressure parameter sequence, which reflects the maximum pressure intensity applied to the mask spatial location during the forging window period; the pressure change rate characteristic is the difference between the last and first terms of the pressure parameter sequence divided by the time window length, which reflects the pressure growth rate at that spatial location during the window period.

[0037] For the strain parameter sequence, calculate the cumulative strain characteristics and the characteristics of the amplitude of drastic strain changes to form a strain parameter characteristic set; The cumulative strain characteristic is the sum of the absolute values ​​of all strain parameters in the strain parameter sequence, used to reflect the total deformation within the forging window period; the drastic strain change amplitude characteristic is the maximum absolute value of the difference between two adjacent strain parameters in the strain parameter sequence, used to reflect the maximum degree of abrupt deformation in the spatial position of the mask.

[0038] The temperature parameter feature set, pressure parameter feature set, and strain parameter feature set are concatenated in sequence to form the physical state vector of the first segmentation result mask at the spatial location and time number.

[0039] The physical state vector is a seven-dimensional vector with a fixed length, representing the comprehensive physical evolution characteristics of the region corresponding to the mask's spatial location during the forging process.

[0040] In the spatiotemporal digital twin database of flange fitting forging, the correspondence between the first segmentation result mask and the physical state vector is recorded to construct a visual-physical multimodal fusion vector; In this embodiment, constructing a visual-physical multimodal fusion vector includes: For each mask in the first segmentation result mask set, extract its corresponding mask visual feature vector from the output of the dynamic Mask2Former segmentation model; In Example 1, the adaptive query vector number, semantic feature layer level number, spatial coordinate index, and image frame time number corresponding to each mask in the first segmentation result mask set are used as index conditions. The adaptive query vector corresponding to the mask is found in the Transformer decoder structure and interacted with the semantic feature map at the same scale for fusion. All spatial location features of the semantic feature map are weighted through the attention mechanism to obtain the spatial response of the mask at that scale level. All spatial responses are globally pooled through the feature compression layer in the model to obtain a fixed-length high-dimensional vector, which serves as the visual feature vector of the mask. The visual feature vector of the mask represents the image texture, boundary, structure, and segmentation response visual characteristics of the flange fitting surface area corresponding to the mask.

[0041] The visual feature vector and the physical state vector of the mask are concatenated in a preset order to obtain a visual-physical multimodal fusion vector.

[0042] The visual-physical multimodal fusion vector includes all components of the mask visual feature vector and all components of the physical state vector. The former part is the high-dimensional visual features of the mask, and the latter part is the seven physical features of the mask during the forging window. Each visual-physical multimodal fusion vector and its unique mask identifier tuple establish a unique key-value pair correspondence in the spatiotemporal digital twin database of flange fitting forging.

[0043] The unique mask identifier tuple of each first segmentation result mask in the spatiotemporal digital twin database of flange fitting forging is paired with its corresponding visual-physical multimodal fusion vector in a one-to-one manner as the input to the fusion discriminant network, and the input sequence is organized in batches.

[0044] The visual-physical multimodal fusion vector is input into the fusion discriminant network, and a fusion discriminant vector is generated through a cross-modal attention mechanism. The first segmentation result mask is then used to verify its authenticity and determine its defect category, and the second segmentation result mask is output. In this embodiment, the second segmentation result mask includes: Dimensional standardization is performed on each visual-physical multimodal fusion vector to obtain dimensionless visual feature components and dimensionless physical state feature components; The dimensional standardization process includes: subtracting the mean of the same type of feature in the entire dataset from the visual feature component and the physical state feature component respectively, and dividing by its standard deviation, so that each component becomes a dimensionless standard component, and obtaining a standardized fusion vector. The first few dimensions of the standardized fusion vector are dimensionless visual feature components, and the last seven dimensions of the standardized fusion vector are dimensionless physical state feature components.

[0045] Based on dimensionless physical state characteristic components, the forging consistency gating coefficient is calculated through linear transformation and nonlinear activation function; The forging consistency gating coefficient is a value between zero and one, used to measure the strength of the support of the physical state characteristics of the mask in the spatiotemporal process of flange fitting forging for real defects. ; in, Forging consistency gating coefficient, For learnable weight vectors, For bias, For the Sigmoid function; It is a dimensionless physical state characteristic component.

[0046] Based on dimensionless physical state characteristic components, the pseudo-defect suppression gating coefficient is calculated through another set of linear transformations and nonlinear activation functions. The false defect suppression gating coefficient is a value between zero and one, used to measure the tendency of a mask to be identified as a false defect in the flange fitting forging environment due to water stains, oxide scale, or light-induced false defects. The calculation method is the same as the forging consistency gating coefficient.

[0047] In the fusion discriminant network, the dimensionless visual feature components are mapped to queries, and the physical state feature components are mapped to keys respectively. By calculating the weighted inner product of the query and the key and combining the difference between the forging consistency gating coefficient and the false defect suppression gating coefficient as the attention gating factor, the physical state feature components are weighted to obtain the cross-modal attention coefficient that reflects the evidence of real defects and the evidence of suppressing false defects. Taking dimensionless visual feature components as input, multiplying them by a set of trainable weight matrices and adding trainable biases, the output is used as a query; taking dimensionless physical state feature components as input, multiplying them by a set of trainable weight matrices and adding trainable biases, the outputs are used as keys respectively. ; in, For cross-modal attention coefficients, The pseudo-defect suppression gating coefficient, For query, Let d be the key and d be the dimension of the attention subspace.

[0048] Based on the cross-modal attention coefficient, linear transformations are applied to the dimensionless visual feature components to obtain the visual principal component. At the same time, the physical state feature components are weighted and aggregated using the cross-modal attention coefficient to obtain the physical modulation component. The visual principal component and the physical modulation component are added together to form a fusion discriminant vector. The fusion discriminant vector represents the co-relationship between the surface texture structure response of flange fittings and the prior forging thermo-mechanical-deformation state.

[0049] Based on the fusion discriminant vector, a linear transformation and activation function are used to calculate the authenticity test score. When the authenticity test score is higher than the preset authenticity threshold, the corresponding first segmentation result mask is determined to be a real defect; otherwise, it is determined to be a false defect. In Example 1, all components of the fused discriminant vector are multiplied one by one by a set of learnable weights and then summed. A learnable bias is added to obtain a scalar representation. The scalar representation is input into an activation function for normalization mapping. The activation function is the Sigmoid function, which outputs a value between zero and one, which is the base score for authenticity verification. The base score for authenticity verification is multiplied by the forging consistency gating coefficient and then multiplied by one minus the pseudo-defect suppression gating coefficient to obtain the authenticity verification score.

[0050] Based on the fused discriminant vector, a fully connected layer and normalization operation are used to calculate the defect category probability distribution. The category corresponding to the largest component of the defect category probability distribution is the defect category discrimination result of the corresponding mask. In Example 1, the fusion discriminant vector is multiplied and accumulated one by one with the weight parameters of the fully connected layer to obtain a category discriminant score vector for all categories. Each component of the category discriminant score vector corresponds to a type of flange fitting forging defect category. Normalization is performed on all components of the category discriminant score vector, and the exponent value of each component is divided by the sum of the exponent values ​​of all components to obtain the defect category probability distribution. The defect category probability distribution is a vector form of the probability corresponding to each defect category.

[0051] For the first segmentation result mask that passes the authenticity test, the amplitude of the mask pixel probability response is recalibrated based on the authenticity test score and the maximum component of the defect category probability distribution to obtain the second segmentation result mask.

[0052] After recalibration, the probability response of the second segmentation result mask pixel at each spatial location is equal to the product of the authenticity test score and the maximum component of the defect category probability distribution, and then multiplied by the probability response of the first segmentation result mask pixel. ; in, The score is for the authenticity test. The first segmentation result mask is located in space. pixel probability response, The mask for the second segmentation result is located in space. pixel probability response, This represents the maximum component of the probability distribution of defect categories.

[0053] Post-processing operations are performed on the second segmentation result mask to obtain an accurate defect mask set, and multi-type defect identification results of flange fitting forging are generated, and defect warning of flange fitting forging is performed.

[0054] In this embodiment, multiple defect identification results for flange fitting forging are generated, including: Edge refinement processing is performed on each second segmentation result mask to obtain an edge refinement mask; The edge refinement process employs a gradient-direction-based non-maximum suppression method to calculate the gradient direction of the second segmentation result mask at each spatial location. By comparing the response intensity in the neighborhood of the gradient direction, only local maxima along the gradient direction are retained, while the responses of the remaining pixels are set to zero. The edge refinement mask retains the adaptive query vector number, semantic feature layer-level number, spatial coordinate index, and image frame time number corresponding to the second segmentation result mask.

[0055] Perform connected component extraction and connected component integrity verification on the edge refinement mask, and output a complete connected component mask set; The connected component extraction uses the four-neighbor method to decompose the edge refinement mask into several connected regions. For each connected region, its area, perimeter, and compactness are calculated. The compactness is used to measure the roundness of the connected region structure. The compactness is calculated by multiplying four times pi, then multiplying by the area of ​​the connected region, and then dividing by the square of the perimeter of the connected region. If the area of ​​the connected region is less than the area threshold or the compactness is less than the compactness threshold, the connected region is determined to be an invalid region and is removed. The complete set of connected component masks is output.

[0056] Scale consistency verification is performed on the complete mask set of connected components to obtain the accurate defect mask set; The scale consistency verification is based on the spatial coordinate index and image frame time number of each connected region. The strain cumulative feature and strain drastic change amplitude feature at that location are retrieved from the spatiotemporal digital twin database of flange fitting forging. The strain cumulative feature is compared with the allowable deformation of that region in the part structure tolerance library. When the area of ​​the connected region is greater than the maximum allowable deformation in the structure tolerance library or less than the minimum effective defect area of ​​the vision system resolution, the connected region is determined to be a scale inconsistency mask and is removed to obtain an accurate defect mask set.

[0057] Geometric feature information is extracted from each mask in the precise defect mask set. Based on the geometric feature information, the identification results of multiple types of defects in flange fitting forging are output, and early warning of flange fitting forging defects is provided.

[0058] Geometric feature information, including: The defect location is the geometric center coordinate of the spatial coordinates of all pixels within the mask; The x-coordinate of the geometric center coordinates is the sum of the x-coordinates of the mask pixels divided by the number of pixels, and the y-coordinate is the sum of the y-coordinates of the mask pixels divided by the number of pixels.

[0059] The defect area is the product of the total number of pixels in the mask and the actual physical area factor of a single pixel. The actual physical area factor of a single pixel is equal to the ratio of the actual physical surface size corresponding to the image input tensor to the total number of pixels.

[0060] The defect shape index is composed of aspect ratio, principal axis direction and compactness. The defect shape index reflects the irregularity of the defect shape. The aspect ratio is the ratio of the length of the long side to the length of the short side of the rectangle circumscribed in the mask coverage area. The principal axis direction is the angle between the major axis obtained by least-squares ellipse fitting of the mask coverage area and the horizontal axis. The compactness is calculated by multiplying four times pi, then multiplying by the mask area, and then dividing by the square of the mask perimeter. The mask perimeter is obtained by counting the pixels at the mask boundary.

[0061] Defect confidence score is the product of the authenticity test score and the largest component of the defect category probability distribution. It is used to quantify the credibility of defects in the physical behavior and visual response throughout the flange fitting forging process.

[0062] A digital twin-based early warning system for forging defects in flange fittings, used to implement a digital twin-based early warning method for forging defects in flange fittings, including: The spatiotemporal twin database construction module is used to collect multi-source process parameter data and multi-view image data of flange fittings forging, establish a spatiotemporal digital twin database of flange fittings forging, and obtain pre-processed image data. An improved Mask2Former segmentation module was developed to construct an improved Mask2Former segmentation model with adaptive query vectors and dynamic attention mechanism based on a spatiotemporal digital twin database of flange fitting forging. The defect candidate mask generation and optimization module is used to input the preprocessed image data into the improved Mask2Former segmentation model and output the first segmentation result mask. The physical state vector extraction module is used to retrieve multi-source process parameter data of the spatial location and time period corresponding to the mask of the first segmentation result using the spatiotemporal digital twin database of flange fitting forging, and form a physical state vector. The multimodal fusion module is used to establish the correspondence between the first segmentation result mask and the physical state vector in the spatiotemporal digital twin database of flange fitting forging, and to construct a visual-physical multimodal fusion vector. The fusion discrimination and segmentation optimization module is used to input the visual-physical multimodal fusion vector into the fusion discrimination network, perform authenticity verification and defect category discrimination on the first segmentation result mask, and output the second segmentation result mask; The early warning module is used to perform post-processing on the mask of the second segmentation result to obtain an accurate defect mask set, output the identification results of multiple types of defects in flange fitting forging, and realize defect early warning.

[0063] Example 2: A complete real-world case study of a flange fitting numbered "F12345" during the forging and intelligent detection of multiple defects throughout the entire process was presented on the flange fitting forging production line.

[0064] During the flange fitting forging process, the process data acquisition module continuously acquires multi-source parameters such as die temperature, die pressure, and workpiece strain, and simultaneously records data from each sensor at the time of acquisition. Taking the flange fitting as an example, the die temperature parameters acquired in a certain process segment are "1182.4°C, 1179.1°C, 1174.8°C", the die pressure parameters are "413.2MPa, 415.0MPa, 418.7MPa", and the workpiece strain parameters are "0.018, 0.021, 0.024". Simultaneously, an industrial control camera simultaneously captures eight high-resolution images of the flange fitting surface from different angles.

[0065] The system preprocesses the acquired multi-view images by denoising and illumination equalization. The processed third frame (image tensor size 2048×2048×3) is then fed into the dynamic Mask2Former segmentation model. The backbone network extracts multi-scale semantic feature maps and automatically calculates the temperature-stress joint weight coefficient for each spatial location using flange fitting process parameters and spatial coordinates. In Example 2, the image coordinates (512, 1300) correspond to a temperature of 1182.4°C, a stress of 413.2 MPa, and a strain of 0.021. The calculated temperature-stress joint weight coefficient is 0.739, which is used to modulate the feature vector of the corresponding channel for that pixel.

[0066] The set of adaptive query vectors is mapped to 64 sets of adaptive query vectors by a learnable mapping matrix and the modulated feature tensor described above. Each set of vectors records the source spatial coordinates and feature channel index. In the Transformer decoder, each set of adaptive query vectors is fused with multi-scale feature maps to generate 64 sets of mask candidate features. Taking the 17th set of query vectors as an example, the response value of the mask candidate feature generated at scale level 2 at spatial location (510, 1305) is 0.81, and the response value is 0.75 after normalization by the dynamic attention module.

[0067] The spatial response of each mask candidate feature is combined with the process parameters of the spatiotemporal digital twin database of flange fitting forging to calculate the forging sensitivity modulation factor. The forging temperature at spatial location (510, 1305) is 1179.1°C, the forging stress is 415.0 MPa, and the forging sensitivity modulation factor is 0.739. The modulated mask value at this spatial location is 0.75 × 0.739 = 0.555. Through a multi-scale gating fusion strategy, the modulated mask features at each scale level are weighted and summed. Finally, the pixel response of the first segmentation result mask corresponding to the 17th query vector at spatial location (510, 1305) is 0.68.

[0068] All first-segmentation masks were processed to retrieve the process parameters for five frames before and after time t=34 from the mask spatial coordinates (510, 1305), obtaining the temperature sequence (1182.4°C, 1179.1°C, 1174.8°C), pressure sequence (413.2MPa, 415.0MPa, 418.7MPa), and strain sequence (0.018, 0.021, 0.024). The average temperature parameter was 1178.77°C, the maximum temperature gradient was 4.3°C, and the temperature variance was 7.5°C². The maximum pressure parameter was 418.7MPa, and the pressure change rate was 1.83MPa / frame. The cumulative strain feature was 0.063, and the drastic strain change amplitude was 0.003. After these seven physical state features were stitched together with the mask visual features, they were written into the spatiotemporal digital twin database of flange fitting forging, establishing a unique key-value pair mapping.

[0069] The fusion discriminant network applies forging consistency gating to the standardized fusion vector. After linear mapping and sigmoid activation, the physical state feature components yield a forging consistency gating coefficient of 0.93 and a pseudo-defect suppression gating coefficient of 0.06. The dimensionless visual feature components are mapped to the query, and the physical state feature components are mapped to the key. The weighted inner product of the two yields a cross-modal attention coefficient of 0.84, which finally forms the fusion discriminant vector. The authenticity test score is 0.82, and the maximum component in the defect category probability distribution is 0.88. The mask is judged as a real defect, and the category is crack.

[0070] The product of the authenticity test score and the maximum component of the defect category probability distribution is 0.72. This is used as the amplitude recalibration coefficient of the mask pixel probability response. Finally, the probability response of the second segmentation result mask at the spatial position (510, 1305) is the probability response of the first segmentation result mask, which is 0.68 multiplied by 0.72, resulting in 0.49.

[0071] Edge refinement was applied to the mask resulting from the second segmentation. Only the pixel with the largest gradient response in its local 8-neighborhood (510, 1305) was retained. All retained pixels underwent four-neighbor connected component analysis to form several connected regions. The main connected region of this mask has an area of ​​23 pixels, a perimeter of 21 pixels, and a compactness of 0.66, all greater than the effective region threshold. The maximum allowable deformation of the connected region at the corresponding position in the structural tolerance library is 26 pixels, while the minimum effective detection area of ​​the vision system is 5 pixels. The mask area is within a reasonable range and is retained as a precise defect mask.

[0072] The geometric center coordinates of the mask are calculated to be (511, 1303), with an area of ​​23 pixels. The physical area of ​​a single pixel is 0.022 square millimeters, and the actual area is 0.506 square millimeters. The shape index is an aspect ratio of 1.22, the principal axis is 52 degrees, the boundary compactness is 0.66, and the mask confidence score is 0.72.

[0073] Finally, the system outputs the defect detection results in a structured format: Defect number: C00017; Defect type: Crack; Defect location: (511, 1303); Defect area: 0.506 square millimeters; Defect confidence level: 0.72; Shape index: Aspect ratio 1.22, main axis 52 degrees, compactness 0.66; Compared with the results of the traditional U-Net method in the same batch and region, the U-Net model failed to detect the crack at this location (probability response 0.18, below the discrimination threshold), while the YOLOv5 model detected it with a probability response of 0.37, which was misclassified as a water stain. Manual verification confirmed the missed detection. In similar cases, the detection rate of the present invention's method for microcracks in this batch was 89%, compared to 59% for traditional U-Net and 62% for YOLOv5. Regarding false defect suppression, the false alarm rate of the present invention's method was 2.1%, while that of traditional U-Net and YOLOv5 were 15.8% and 13.5%, respectively.

[0074] During continuous inspection, the operator observed a large defect mask with an area of ​​187 pixels in the 5th frame image of a flange. After checking the corresponding process parameters, the cumulative strain feature was 0.167, the maximum allowable deformation was 0.173, and the area met the standard. It was determined to be a missing part defect with a defect confidence level of 0.83, a defect location of (1840, 992), and an actual area of ​​4.11 square millimeters. Compared with traditional methods, the defect segmentation area error is less than 0.21 square millimeters, while the U-Net error is 1.04 square millimeters.

[0075] Example 2 demonstrates that the technology of the present invention has deep coordination of process parameters and visual signals throughout the entire process and at every step. The segmentation accuracy, location and physical interpretation capability of real defects are significantly better than traditional methods, which greatly improves the automatic detection rate of small and large-scale defects in actual production.

[0076] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for early warning of forging defects in flange fittings based on digital twins, characterized in that, include: Collect multi-source process parameter data and multi-view image data of flange fitting forging to generate a spatiotemporal digital twin database of flange fitting forging. Perform image preprocessing on the multi-view image data of flange fitting surface to obtain preprocessed multi-view image data of flange fitting surface. An improved Mask2Former segmentation model was constructed based on a spatiotemporal digital twin database of flange fitting forging. The preprocessed multi-view image data of the flange fitting surface is input into the improved Mask2Former segmentation model. Adaptive query vectors are used to generate multi-scale defect candidate masks. The defect candidate masks are optimized by dynamic attention module, and the first segmentation result mask is output. Using the spatiotemporal digital twin database of flange fitting forging, the multi-source process parameter data of flange fitting forging within the spatial location and time period corresponding to the mask of the first segmentation result are retrieved to form a physical state vector; In the spatiotemporal digital twin database of flange fitting forging, the correspondence between the first segmentation result mask and the physical state vector is recorded to construct a visual-physical multimodal fusion vector; The visual-physical multimodal fusion vector is input into the fusion discriminant network, and a fusion discriminant vector is generated through a cross-modal attention mechanism. The first segmentation result mask is then used to verify its authenticity and determine its defect category, and the second segmentation result mask is output. Post-processing operations are performed on the second segmentation result mask to obtain an accurate defect mask set, and multi-type defect identification results of flange fitting forging are generated, and flange fitting forging defect early warning is performed.

2. The method for early warning of forging defects in flange fittings based on digital twins according to claim 1, characterized in that, The construction of the spatiotemporal digital twin database for flange fitting forging includes: Multi-source process parameter data and multi-view image data of flange fitting forging are collected. The multi-source process parameter data and multi-view image data of flange fitting surface are synchronously mapped by a unified timestamp and spatial coordinates to generate a spatiotemporal digital twin database of flange fitting forging. Image preprocessing is performed on the multi-view image data of flange fitting surface, including noise reduction, illumination equalization, reflection suppression and contrast enhancement, to obtain preprocessed multi-view image data of flange fitting surface. The preprocessed multi-view image data of flange fitting surface is then stored in the spatiotemporal digital twin database of flange fitting forging, resulting in the final spatiotemporal digital twin database of flange fitting forging.

3. The method for early warning of forging defects in flange fittings based on digital twins according to claim 1, characterized in that, The improved Mask2Former segmentation model includes: Read the multi-view image feature tensor of the flange component surface and the multi-source process parameter matrix of flange component forging from the spatiotemporal digital twin database of flange component forging, calculate the temperature-stress joint weight coefficient, use the temperature-stress joint weight coefficient as the modulation factor, multiply it with each channel of the learnable mapping matrix and the image feature tensor to obtain an adaptive query vector set. In each layer of the dynamic attention module, the temperature gradient and stress gradient are calculated based on the input features of this layer. The normalized scaling factor of the temperature gradient and stress gradient is used, combined with the adaptive query vector of the previous layer, the learnable transformation matrix and the image features of the current layer, and then the current layer adaptive query vector set is updated in a weighted manner after being processed by the activation function. In the dynamic attention module, for each adaptive query vector and each key vector, the normalized ratio of their dot product to the vector dimension is calculated, and combined with the forging stress difference modulation factor, the normalized attention score is obtained. Through a multi-scale feature pyramid structure, three sets of masks with different spatial resolutions are output, corresponding to small-scale, long-scale and global-scale masks respectively. For each scale mask, multi-scale gating weights are calculated. Based on the multi-scale gating weights, the masks of each scale are weighted and summed to obtain the multi-scale fused mask.

4. The method for early warning of forging defects in flange fittings based on digital twins according to claim 1, characterized in that, The first segmentation result mask includes: The preprocessed multi-view image data of the flange fitting surface is converted into an image input tensor. The image input tensor is then input into the backbone encoding network of the dynamic Mask2Former segmentation model to obtain a set of multi-scale semantic feature maps. The adaptive query vector set and the multi-scale semantic feature map set are interactively fused at each scale level through the Transformer decoder to obtain mask candidate features for each adaptive query vector on each scale feature map. The attention score of each mask candidate feature at the corresponding spatial location is calculated based on the dynamic attention module, and the normalized optimized mask is calculated. For each mask spatial location, the forging temperature parameters and forging stress parameters corresponding to the same spatial location and time period are obtained by calling the spatiotemporal digital twin database of flange fitting forging, and the forging sensitivity modulation factor of the corresponding spatial location is calculated. At each spatial location, the normalized optimized mask value is multiplied by the forging sensitivity modulation factor to obtain the modulated mask features; The modulated mask features of all scale levels and all adaptive query vectors are fused using a multi-scale gating fusion strategy to obtain the final first segmentation result mask.

5. The method for early warning of forging defects in flange fittings based on digital twins according to claim 1, characterized in that, The retrieved multi-source process parameter data for flange fitting forging within the spatial location and time period corresponding to the first segmentation result mask includes: For each mask in the first segmentation result mask set, based on the spatial coordinate information and time number recorded by the mask, retrieve the data entry that completely matches the spatial coordinate information and time number in the flange fitting forging spatiotemporal digital twin database, and obtain the forging temperature parameters, forging pressure parameters and forging strain parameters corresponding to the mask spatial position and time number; For each mask spatial location, the continuous forging temperature parameters within a fixed-length time window before and after the time number are combined into a temperature parameter sequence, and the continuous forging pressure parameters are combined into a pressure parameter sequence. For a temperature parameter sequence, calculate the average temperature feature, the maximum temperature gradient feature, and the temperature variance feature to form a set of temperature parameter features; For the pressure parameter sequence, calculate the maximum pressure characteristic and the pressure change rate characteristic to form a pressure parameter characteristic set. For the strain parameter sequence, calculate the cumulative strain characteristics and the characteristics of the amplitude of drastic strain changes to form a strain parameter characteristic set; The temperature parameter feature set, pressure parameter feature set, and strain parameter feature set are concatenated in sequence to form the physical state vector of the first segmentation result mask at the spatial location and time number.

6. The method for early warning of forging defects in flange fittings based on digital twins according to claim 1, characterized in that, The construction of the visual-physical multimodal fusion vector includes: For each mask in the first segmentation result mask set, extract its corresponding mask visual feature vector from the output of the dynamic Mask2Former segmentation model; The visual feature vector and the physical state vector of the mask are concatenated in a preset order to obtain a visual-physical multimodal fusion vector.

7. The method for early warning of forging defects in flange fittings based on digital twins according to claim 1, characterized in that, The second segmentation result mask includes: Dimensional standardization is performed on each visual-physical multimodal fusion vector to obtain dimensionless visual feature components and dimensionless physical state feature components; Based on dimensionless physical state characteristic components, the forging consistency gating coefficient is calculated through linear transformation and nonlinear activation function; Based on dimensionless physical state characteristic components, the pseudo-defect suppression gating coefficient is calculated through another set of linear transformations and nonlinear activation functions. In the fusion discriminant network, the dimensionless visual feature components are mapped to queries, and the physical state feature components are mapped to keys respectively. By combining the forging consistency gating coefficient and the false defect suppression gating coefficient, cross-modal attention coefficients that reflect real defect evidence and suppress false defect evidence are obtained. Based on the cross-modal attention coefficient, linear transformations are applied to the dimensionless visual feature components to obtain the visual principal component. At the same time, the physical state feature components are weighted and aggregated using the cross-modal attention coefficient to obtain the physical modulation component. The visual principal component and the physical modulation component are added together to form a fusion discriminant vector. Based on the fusion discriminant vector, a linear transformation and activation function are used to calculate the authenticity test score. When the authenticity test score is higher than the preset authenticity threshold, the corresponding first segmentation result mask is determined to be a real defect; otherwise, it is determined to be a false defect. Based on the fused discriminant vector, a fully connected layer and normalization operation are used to calculate the defect category probability distribution. The category corresponding to the largest component of the defect category probability distribution is the defect category discrimination result of the corresponding mask. For the first segmentation result mask that passes the authenticity test, the amplitude of the mask pixel probability response is recalibrated based on the authenticity test score and the maximum component of the defect category probability distribution to obtain the second segmentation result mask.

8. The method for early warning of forging defects in flange fittings based on digital twins according to claim 1, characterized in that, The generated flange fitting forging multi-type defect identification results include: Edge refinement processing is performed on each second segmentation result mask to obtain an edge refinement mask; Perform connected component extraction and connected component integrity verification on the edge refinement mask, and output a complete connected component mask set; Scale consistency verification is performed on the complete mask set of connected components to obtain the accurate defect mask set; Geometric feature information is extracted from each mask in the precise defect mask set. Based on the geometric feature information, the identification results of multiple types of defects in flange fitting forging are output, and early warning of flange fitting forging defects is provided.

9. A method for early warning of forging defects in flange fittings based on digital twins according to claim 8, characterized in that, The geometric feature information includes: The defect location is the geometric center coordinate of the spatial coordinates of all pixels within the mask; The defect area is the product of the total number of pixels in the mask and the actual physical area factor of a single pixel. The defect shape index, composed of aspect ratio, principal axis direction, and compactness, reflects the irregularity of the defect shape. Defect confidence score is the product of the authenticity test score and the largest component of the defect category probability distribution. It is used to quantify the credibility of defects in the physical behavior and visual response throughout the flange fitting forging process.

10. A digital twin-based early warning system for forging defects in flange fittings, used to execute the digital twin-based early warning method for forging defects in flange fittings as described in any one of claims 1-9, characterized in that, include: The spatiotemporal twin database construction module is used to collect multi-source process parameter data and multi-view image data of flange fittings forging, establish a spatiotemporal digital twin database of flange fittings forging, and obtain pre-processed image data. An improved Mask2Former segmentation module was developed to construct an improved Mask2Former segmentation model with adaptive query vectors and dynamic attention mechanism based on a spatiotemporal digital twin database of flange fitting forging. The defect candidate mask generation and optimization module is used to input the preprocessed image data into the improved Mask2Former segmentation model and output the first segmentation result mask. The physical state vector extraction module is used to retrieve multi-source process parameter data of the spatial location and time period corresponding to the mask of the first segmentation result using the spatiotemporal digital twin database of flange fitting forging, and form a physical state vector. The multimodal fusion module is used to establish the correspondence between the first segmentation result mask and the physical state vector in the spatiotemporal digital twin database of flange fitting forging, and to construct a visual-physical multimodal fusion vector. The fusion discrimination and segmentation optimization module is used to input the visual-physical multimodal fusion vector into the fusion discrimination network, perform authenticity verification and defect category discrimination on the first segmentation result mask, and output the second segmentation result mask; The early warning module is used to perform post-processing on the mask of the second segmentation result to obtain an accurate defect mask set, output the identification results of multiple types of defects in flange fitting forging, and realize defect early warning.

Images