Adipic acid reaction chamber crystallization abnormality monitoring method and system based on attention fusion

Abstract

The present invention discloses a method for monitoring crystallization abnormality of an adipic acid reaction chamber based on attention fusion, comprising: S1: constructing an image deblurring model based on space-frequency attention fusion, S2: acquiring an adipic acid reaction chamber image, and dividing the adipic acid reaction chamber image into N non-overlapping image blocks, S3: extracting implicit semantic features in each image block according to the image deblurring model, and reconstructing a deblurred image; S4: reconstructing the deblurred image I deblur Deblurring is performed to obtain a clear image; S5: extracting clear image features and target attention features through a multi-head self-attention module, and obtaining representative abnormal image features of the adipic acid reaction chamber based on heterogeneous feature attention fusion; S6: constructing an abnormal target detection model for the adipic acid reaction chamber through bounding box loss, classification loss and confidence loss functions to perform real-time abnormal monitoring. The present invention can automatically monitor and warn, effectively improving the safety of the production process.

Description

Technical Field

[0001] The present invention belongs to the technical field of adipic acid production, and in particular relates to a method and system for monitoring crystallization abnormality in an adipic acid reaction chamber based on attention fusion. Background Art

[0002] Adipic acid reaction chamber crystallization refers to the phenomenon that adipic acid precipitates from the solution to form crystals inside the reaction chamber due to various reasons (such as temperature changes, excessive concentration, etc.) during the production or use of adipic acid (fatty acid). This crystallization will affect the reaction efficiency and the normal operation of the equipment. This process will cause abnormal liquid level, shaking of the stirring rod, crystallization of the stirring rod contact area, boiling of the liquid, crystallization of the indoor wall, crystallization of the siphon, abnormal siphon liquid level and other abnormal phenomena. If the relevant abnormal phenomena are not handled in time, it will affect the product quality, cause production interruptions or delays, and thus reduce production efficiency. Even when the abnormal phenomenon is serious, it will cause safety accidents, such as equipment damage, chemical leakage, etc., leading to safety production risks. Therefore, monitoring of abnormal crystallization in the adipic acid reaction chamber is very necessary to improve high quality and safe production.

[0003] At present, the monitoring of abnormal crystallization in the adipic acid reaction chamber mainly relies on manual observation and recognition of images captured by the camera or on simple recognition algorithms. For manual observation, the labor intensity is large, the labor cost is high and the efficiency is low. For recognition algorithms, such as: the method and device for identifying objects from videos with the publication number of CN 116030387A, this method scheme uses a cascaded transformer network to extract and fuse the spatiotemporal features of the image to identify the target. However, due to the complex and changeable industrial site environment, affected by complex factors such as temperature, humidity, dust, and light, the image quality is uneven, resulting in the lack of accuracy of the existing recognition algorithm for abnormal phenomenon monitoring. In particular, when there is colored gas in the reaction chamber, the camera shooting field of view is relatively blurred, resulting in unclear targets in the image. In addition, in large-scale industrial production, it is necessary to respond to abnormal phenomena quickly. The existing monitoring methods are difficult to detect abnormal phenomena in time, and to provide real-time reminders or warnings, and it is difficult to effectively prevent production safety. Summary of the invention

[0004] The purpose of the present invention is to provide a method and system for monitoring abnormal crystallization in an adipic acid reaction chamber based on attention fusion. The method automatically performs monitoring and early warning by constructing an integrated deep learning model that can reduce image blur and achieve target monitoring, thereby effectively improving the safety level in the adipic acid production process.

[0005] In order to achieve one of the above purposes, the present invention adopts the following technical solution:

[0006] A method for monitoring abnormal crystallization of an adipic acid reaction chamber based on attention fusion comprises the following steps:

[0007] S1: Acquire the image I of the adipic acid reaction chamber blur , and the adipic acid reaction chamber image I blur Split into N non-overlapping image blocks,

[0008] S2: construct an image deblurring model based on space-frequency attention fusion, extract the implicit semantic features in each image block according to the image deblurring model, and reconstruct the deblurred image I deblur ;

[0009] S3: Deblurring the image I deblur Perform deblurring to obtain a clear image;

[0010] S4: The clear image in step S3 is subjected to a multi-head self-attention module to extract clear image features, and the related targets are subjected to a multi-head self-attention module to extract target attention features. Based on heterogeneous feature attention fusion, the clear image features and the target attention features are subjected to a multi-head cross-attention module to perform fusion of different categories of features to obtain representative abnormal image features of the adipic acid reaction chamber;

[0011] S5: Predict the target category and bounding box of the abnormal image features of the adipic acid reaction chamber, and then build an abnormal target detection model for the adipic acid reaction chamber through bounding box loss, classification loss and confidence loss function to perform real-time monitoring of abnormal targets.

[0012] Furthermore, the method further includes S6: if an abnormality is detected, issuing a target abnormality alarm.

[0013] Furthermore, in S2, an image deblurring model based on space-frequency attention fusion is constructed, and the implicit semantic features in each image block are extracted according to the image deblurring model to reconstruct the deblurred image I deblur ,include:

[0014] S201: In the spatial domain, a spatial attention mechanism is used to obtain the representative spatial features F of each image block. s ;

[0015] S202: In the frequency domain, the frequency domain attention features of each image block are mined and fused with the spatial domain features to enhance the feature expression and obtain the frequency domain feature F f ;

[0016] S203: The frequency domain feature F f Perform inverse Fourier transform to obtain the deblurred image I deblur .

[0017] Further, the S201: in the spatial domain, using the spatial attention mechanism to obtain the representative spatial features F of each of the image blockss , including:

[0018] For each image patch feature X in the image P ,

[0019] S2011: image block feature X P Add a learnable matrix E SPos To retain its original spatial position information, the image block feature is transformed into X' P =X P +E SPos , and then calculate the attention score Attention(Q,K,V) between it and all other image patches:

[0020] Q=LN(W Q ·X' P ),K=LN(W K ·X' P ),V=LN(W V ·X' P )

[0021]

[0022] Among them, W Q , W K and W V is a learnable weight matrix, d is the feature dimension, and LN is the layer normalization layer:

[0023] S2012: Use the softmax function to normalize the attention scores, obtain the attention weights, and use the multi-head attention mechanism to mine the implicit information of image space in different dimensions MHA(Q,K,V):

[0024]

[0025] S2013: Residual connection is used to retain the original features, and MLP layer is applied to realize feature conversion and aggregation to obtain spatial domain features F s , as shown in the following formula:

[0026] F′ s =MHA(Q,K,V)+X′ P

[0027] F s =MLP(LN(F′ s ))+F′ s .

[0028] Further, S202: in the frequency domain, the frequency domain attention features of each image block are mined and fused with the spatial domain features to enhance the feature expression and obtain the frequency domain feature Ff , including:

[0029] S2021: Use the fully connected layer to transform the spatial features F of each image block mentioned above s Convert and perform Fourier transform to obtain the feature representation F in the corresponding frequency domain Q 、F K 、F V ;

[0030] Q s =LN(W Q ·F s ),K s =LN(W K ·F s ),V s =LN(W V ·F s )

[0031]

[0032] in is Fourier transform;

[0033] S2022: Apply the multi-head attention mechanism in the frequency domain to calculate the frequency domain attention feature A between all image blocks f ;

[0034] A f =MHA(F Q ,F K ,F V )

[0035] S2023: Frequency domain feature F f It is expressed as:

[0036]

[0037] Where ⊙ represents the dot product operation and σ is a transformation function.

[0038] Furthermore, the deblurred image I deblur Deblurring is performed, including comprehensive processing of gradient loss, total variation loss and residual loss function.

[0039] Furthermore, the comprehensive processing using gradient loss, total variation loss and residual loss function specifically includes:

[0040] S301: Gradient loss focuses on the high-frequency information of the image and is used to measure the clarity of the image, expressed as L grad :

[0041]

[0042] in is the gradient at (i, j) in the deblurred image;

[0043] S302: Total variation loss promotes the sparsity of the image and enhances the edge information of the image, expressed as L VT :

[0044]

[0045] S303: The residual loss focuses on the difference between the image before and after deblurring, denoted as L res :

[0046]

[0047] S304: Considering the above loss functions comprehensively, construct a deblurring loss function L for the adipic acid reaction chamber image deblur It is expressed as:

[0048] L deblur =αL grad +βL VT +γL res .

[0049] Furthermore, in step S4, the clear image in step S3 is subjected to a multi-head self-attention module to extract clear image features, and the related targets are subjected to a multi-head self-attention module to extract target attention features. Based on heterogeneous feature attention fusion, the clear image features and the target attention features are subjected to a multi-head cross-attention module to perform fusion of different categories of features to obtain characteristic abnormal image features of the adipic acid reaction chamber, specifically including:

[0050] S401: For the clear image block feature X, first embed a position feature matrix In this way, the original position information of each clear image block is retained, and the clear image block feature X is transformed into

[0051] Then, different linear layers are used to convert it into Q, K and V of the multi-head self-attention layer to perform attention calculation between image blocks, and then the MLP layer is used to transform the features to obtain the clear image feature F img ;

[0052] X″=MHA(LN(X′),LN(X′),LN(X′))

[0053] F img =MLP(LN(X″+X′))+X″+X′;

[0054] S402: For the target feature Y, first embed a position feature matrix To retain the original information of the target in the image, the target feature Y is transformed into

[0055] Then, different linear layers are used to convert it into Q, K and V of the multi-head self-attention layer to perform attention calculation between image blocks, and then the MLP layer is used to transform the features to obtain the target attention feature F obj , used to represent the implicit information of the target;

[0056] Y″=MHA(LN(Y′),LN(Y′),LN(Y′))

[0057] F obj =MLP(LN(Y″+Y′))+Y″+Y′

[0058] S402: In order to fuse the multi-dimensional heterogeneous features related to the target, a cross-attention module is constructed to fuse the above target-related clear image features F img and the target attention feature F obj , where the clear image feature F img After the linear layer conversion, it is used as Q of the cross-attention module and the target attention feature F obj Transformed into K and V across attention modules through different linear layers,

[0059]

[0060] F = MLP(LN(F′+F img +F obj ))+F′+F img +F obj

[0061] in and are different linear layers for feature transformation, and the abnormal image feature F of the adipic acid reaction chamber which integrates different categories of information is obtained.

[0062] Furthermore, in step S5, a bounding box loss, classification loss and confidence loss function are used to construct an abnormal target detection model for the adipic acid reaction chamber to perform real-time monitoring of abnormal targets, which specifically includes:

[0063] The bounding box loss uses GIoU loss to measure the difference between the predicted bounding box and the true bounding box. GIoU loss not only considers the overlapping area of ​​the predicted box and the true box, but also their minimum bounding rectangle.

[0064]

[0065] Among them, b pred and b gtare the predicted and true bounding boxes, respectively, I is the intersection area of ​​the predicted box and the true box, U is the union area of ​​the predicted box and the true box, and A c is the area of ​​the minimum enclosed region containing the predicted box and the true box;

[0066] The classification loss uses the cross entropy loss function to measure the accuracy of category prediction:

[0067]

[0068] Among them, C is the number of categories, y c is the one-hot encoding of the true label, p c The probability of the model detecting category c;

[0069] Confidence loss: high confidence for correct bounding boxes, low confidence for incorrect bounding boxes:

[0070]

[0071] in, is the predicted probability of the target existing, p obj is the true probability of the target existing, is the predicted probability of category c;

[0072] Through bounding box loss, classification loss and confidence loss function, a comprehensive loss function L is constructed zh The abnormal target detection model for the adipic acid reaction chamber is constructed as follows:

[0073] L zh =λ grad L grad +λ VT L VT +λ res L res

[0074] where λ grad , VT and λ res It is the weight parameter corresponding to the bounding box loss, classification loss and confidence loss.

[0075] In order to achieve the second of the above objectives, the present invention adopts the following technical solution:

[0076] An attention fusion-based adipic acid reaction chamber crystallization abnormality monitoring system comprises: a data acquisition module, an image deblurring module and an abnormal target detection module.

[0077] The data acquisition module is used to collect the image I of the adipic acid reaction chamber. blur ;

[0078] The image deblurring module is used to construct an image deblurring model based on space-frequency attention fusion, and divide the adipic acid reaction chamber image into N non-overlapping image blocks P, extract the implicit semantic features in each image block according to the image deblurring model, and reconstruct the deblurred image I deblur ; for the deblurred image I deblur Perform deblurring to obtain a clear image;

[0079] The abnormal target detection module is used to extract clear image features from the clear image through a multi-head self-attention module, extract target attention features from related targets through a multi-head self-attention module, and based on heterogeneous feature attention fusion, fuse the clear image features and the target attention features with different categories through a multi-head cross-attention module to obtain representative abnormal image features of the adipic acid reaction chamber; predict the category and bounding box of the target with the abnormal image features of the adipic acid reaction chamber, and construct an abnormal target detection model for the adipic acid reaction chamber through bounding box loss, classification loss and confidence loss functions to monitor abnormal targets in real time.

[0080] Beneficial effects of the present invention:

[0081] The method and system of the present invention, aiming at the characteristics of the image related to the crystallization of the adipic acid reaction chamber, firstly construct an image deblurring method based on space-frequency attention fusion, which effectively improves the quality of the adipic acid reaction chamber image. Then, through the adipic acid reaction chamber abnormality monitoring method based on heterogeneous feature attention fusion, the characteristic adipic acid reaction chamber abnormal image features are obtained, and the real-time monitoring of the abnormal state is realized. The present invention combines the adipic acid reaction chamber image deblurring based on space-frequency attention fusion with the abnormal target monitoring of heterogeneous feature attention fusion, realizes the real-time monitoring of abnormal phenomena such as liquid level abnormality, stirring rod shaking, stirring rod contact area crystallization, liquid boiling, indoor wall crystallization, siphon crystallization, siphon liquid level abnormality in the adipic acid reaction chamber, and improves the safety level of the adipic acid reaction chamber production scene. It has the advantages of high accuracy, real-time and intelligence, can significantly improve the safety management level of the adipic acid reaction chamber production link, reduce production risks and labor costs, and ensure the safe and efficient operation of the adipic acid reaction chamber production. Therefore, the present invention has the following effects:

[0082] (a) In the deblurring process of the adipic acid reaction chamber image, the present invention constructs an attention fusion model combining spatial domain and frequency domain to extract the implicit semantic features in each image block. Without the need to use a reference clear image, the deblurred image is effectively reconstructed, thereby reducing the adverse effects of colored gases in the adipic acid reaction chamber environment on the image quality.

[0083] (b) In the abnormal monitoring process of the adipic acid reaction chamber, the present invention first extracts heterogeneous deblurred image and target image features based on the multi-head self-attention module, and then constructs a multi-head cross-attention module to fuse features of different categories, thereby effectively enhancing the representativeness of the features and keeping the network simple and efficient.

[0084] (c) In terms of loss function, the present invention simultaneously considers factors such as target box, abnormal target classification and its probability involved in the abnormal monitoring process of adipic acid reaction chamber, and accurately and efficiently realizes the prediction of abnormal targets by constructing a comprehensive loss of bounding box, classification and confidence. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] Figure 1 This is a flow chart of a specific embodiment 1 of the present invention;

[0086] Figure 2 It is a flowchart of the method for monitoring the deblurred image and fused abnormal targets in the specific embodiment 1 of the present invention;

[0087] Figure 3 It is a principle block diagram of specific embodiment 2 of the present invention;

[0088] Figure 4 This is a block diagram of the working principles of the image deblurring module and the abnormal target detection module in specific embodiment 2 of the present invention. DETAILED DESCRIPTION

[0089] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Specific embodiment 1:

[0091] See also Figure 1 and Figure 2 As shown, a method for monitoring abnormal crystallization in adipic acid reaction chamber based on attention fusion is presented.

[0092] The steps include:

[0093] Step S1: Acquire the adipic acid reaction chamber image I blur , and divide the adipic acid reaction chamber image into N non-overlapping image blocks P = {p1, p2, ..., p N},

[0094] Step S2: construct an image deblurring model based on space-frequency attention fusion, extract the implicit semantic features in each image block P according to the image deblurring model, and reconstruct the deblurred image I deblur ;

[0095] S201: In the spatial domain, a spatial attention mechanism is used to obtain a representative spatial feature F of each image block P. s ,

[0096] Specifically include:

[0097] In the spatial domain, the spatial attention mechanism is used to focus on the spatial attention features of each image block P, explore their mutual influence and dependency in space, and obtain representative spatial image features: For each image block feature X in the image P ,

[0098] S2011: image block feature X P Add a learnable matrix E SPos To retain its original spatial position information, the image block feature is transformed into X' P =X P +E SPos , and then calculate the attention score Attention(Q,K,V) between it and all other image patches:

[0099] Q=LN(W Q ·X' P ),K=LN(W K ·X' P ),V=LN(W V ·X' P )

[0100]

[0101] Among them, W Q , W K and W V is a learnable weight matrix, d is the feature dimension, and LN is the layer normalization layer:

[0102] S2012: Use the softmax function to normalize the attention scores, obtain the attention weights, and use the multi-head attention mechanism to mine the implicit information of image space in different dimensions MHA(Q,K,V):

[0103]

[0104] S2013: Residual connections are used to preserve the original features, and MLP layers are applied to achieve feature conversion and aggregation, see Figure 2 As shown, L refers to the number of stacked multi-layer autoencoders, i.e., the number of elliptical dashed boxes, to obtain the spatial domain feature F s , as shown in the following formula:

[0105] F′ s =MHA(Q,K,V)+X′ P

[0106] F s =MLP(LN(F′ s ))+F′ s .

[0107] S202: In the frequency domain, the frequency domain attention features of each image block are mined and fused with the spatial domain attention features to enhance the feature expression and obtain the frequency domain feature F f ;

[0108] S2021: Use the fully connected layer to transform the spatial features F of each image block mentioned above s Convert and perform Fourier transform to obtain the feature representation F in the corresponding frequency domain Q 、F K 、F V ;

[0109] Q s =LN(W Q ·F s ),K s =LN(W K ·F s ),V s =LN(W V ·F s )

[0110]

[0111] in is Fourier transform;

[0112] S2022: Apply the multi-head attention mechanism in the frequency domain to calculate the frequency domain attention feature A between all image blocks f ;

[0113] A f =MHA(F Q ,F K ,F V )

[0114] S2021: Frequency domain feature F f It is expressed as:

[0115]

[0116] Where ⊙ represents the dot product operation and σ is a transformation function.

[0117] S203: The frequency domain feature F f Perform inverse Fourier transform to obtain the deblurred image I deblur :

[0118]

[0119] Step S3: Using gradient loss, total variation loss and residual loss function to deblur the deblurred image I deblur Perform deblurring to obtain a clear image;

[0120] Without using a reference clear image, the image deblurring goal is achieved by comprehensively considering the gradient loss, total variation loss and residual loss function, including:

[0121] S301: Gradient loss focuses on the high-frequency information of the image and is used to measure the clarity of the image, expressed as L grad :

[0122]

[0123] in is the gradient at (i, j) in the deblurred image;

[0124] S302: Total variation loss promotes the sparsity of the image and enhances the edge information of the image, expressed as L VT :

[0125]

[0126] S303: The residual loss focuses on the difference between the image before and after deblurring, denoted as L res :

[0127]

[0128] S304: Considering the above loss functions comprehensively, construct a deblurring loss function L for the adipic acid reaction chamber image deblur It is expressed as follows to get a clear image:

[0129] L deblur =αL grad +βL VT +γL res

[0130] Among them, α is the adjustment function of gradient loss, β is the adjustment function of total variation loss, and r is the adjustment function of residual loss.

[0131] This method extracts the implicit semantic features in each image block by constructing an attention fusion model that combines the spatial domain and the frequency domain. It effectively reconstructs the deblurred image without the need to use a reference clear image, thereby reducing the adverse effects of colored gases in the adipic acid reaction chamber environment on image quality.

[0132] Through the above steps S1, S2 and S3, the image deblurring method of the adipic acid reaction chamber based on space-frequency attention fusion is realized. Since colored gas may exist in the adipic acid reaction chamber, the camera shooting field of view is relatively blurred, resulting in poor image quality and unclear targets, which affects the effect of the subsequent abnormal monitoring algorithm. Therefore, according to the characteristics of the special environment image of the adipic acid reaction chamber, the present invention adopts the adipic acid reaction chamber image deblurring algorithm based on space-frequency attention fusion, thereby improving the image clarity and other qualities, and promoting the subsequent adipic acid reaction chamber crystallization abnormality monitoring performance improvement.

[0133] Step S4: The clear image in step S3 is passed through a multi-head self-attention module to obtain clear image features, and at the same time, the relevant targets are passed through a multi-head self-attention module to extract target attention features; based on heterogeneous feature attention fusion, the clear image features and the target attention features are fused with different categories of features through a multi-head cross-attention module to obtain representative adipic acid reaction chamber abnormal image features F, thereby realizing heterogeneous feature attention fusion, thereby enhancing the representativeness of the features and keeping the network simple and efficient.

[0134] Specifically include:

[0135] S401: For the clear image block feature X, first embed a position feature matrix In this way, the original position information of each image block is retained, and the clear image block feature X is transformed into

[0136] Then, different linear layers are used to convert it into Q, K and V of the multi-head self-attention layer to perform attention calculation between image blocks, and then the MLP layer is used to transform the features;

[0137] X″=MHA(LN(X′),LN(X′),LN(X′))

[0138] F img =MLP(LN(X″+X′))+X″+X′;

[0139] S402: For the target feature Y, first embed a position feature matrix To retain the original information of the target in the image, the target feature Y is transformed into

[0140] Then, different linear layers are used to convert it into Q, K and V of the multi-head self-attention layer to perform attention calculation between image blocks, and then the MLP layer is used to transform the features to obtain the target attention feature F obj, used to represent the implicit information of the target; the Q, K and V of the multi-head self-attention layer are converted from the transformed target features through different linear layers;

[0141] Y″=MHA(LN(Y′),LN(Y′),LN(Y′))

[0142] F obj =MLP(LN(Y″+Y′))+Y″+Y′

[0143] S403: In order to fuse the multi-dimensional heterogeneous features related to the target, a cross-attention module is constructed to fuse the above target-related clear image features F img and the target attention feature F obj , where the clear image feature F img After the linear layer conversion, it is used as Q of the cross-attention module and the target attention feature F obj Transformed into K and V across attention modules through different linear layers,

[0144]

[0145] F = MLP(LN(F′+F img +F obj ))+F′+F img +F obj

[0146] in and Different linear layers are used for feature transformation to obtain the abnormal image feature F of the adipic acid reaction chamber that integrates different categories of information, so that the model can learn richer implicit target knowledge and thus have better performance.

[0147] S5: The abnormal image feature F of the adipic acid reaction chamber is passed through multiple fully connected network layers, and the probability of each category is obtained by using the softmax function to predict the category and bounding box of the target, and then the abnormal target detection model of the adipic acid reaction chamber is constructed by the bounding box loss, classification loss and confidence loss function to perform real-time monitoring of abnormal targets; wherein:

[0148] The bounding box loss uses GIoU loss to measure the difference between the predicted bounding box and the true bounding box. GIoU loss not only considers the overlapping area of ​​the predicted box and the true box, but also their minimum bounding rectangle.

[0149]

[0150] Among them, b pred and b gtare the predicted and true bounding boxes, respectively, I is the intersection area of ​​the predicted box and the true box, U is the union area of ​​the predicted box and the true box, and A c is the area of ​​the minimum enclosed region containing the predicted box and the true box;

[0151] The classification loss uses the cross entropy loss function to measure the accuracy of category prediction:

[0152]

[0153] Among them, C is the number of categories, y c is the one-hot encoding of the true label, p c The probability of the model detecting category c;

[0154] Confidence loss: Encourages the model to have high confidence in correct bounding boxes and low confidence in incorrect bounding boxes:

[0155]

[0156] in, is the model’s predicted probability of the target’s existence, p obj is the true probability of the target existing, is the model’s predicted probability for category c;

[0157] By comprehensively considering the above bounding box loss, classification loss and confidence loss function, a comprehensive loss function L is constructed zh The abnormal target detection model for the adipic acid reaction chamber is constructed as follows:

[0158] L zh =λ grad L grad +λ VT L VT +λ res L res

[0159] where λ grad , VT and λ res It is the weight parameter corresponding to the bounding box loss, classification loss and confidence loss, which is used to balance the contribution of different loss functions. The present invention fully considers the factors such as the target box, abnormal target classification and its probability involved in the abnormal monitoring process of the adipic acid reaction chamber, and accurately and efficiently realizes the prediction of abnormal targets by constructing the comprehensive loss of bounding box, classification and confidence.

[0160] Through the above steps S4 and S5, an abnormal target monitoring method based on heterogeneous feature attention fusion is constructed.

[0161] Possibly, it also includes S6: if an abnormality is detected, a target abnormality alarm is issued to facilitate timely reminders.

[0162] The present invention firstly proposes an image deblurring method for an adipic acid reaction chamber based on space-frequency attention fusion, which effectively improves the image clarity and other qualities, and promotes the subsequent improvement of the adipic acid reaction chamber crystallization abnormality monitoring performance. On this basis, a method for monitoring the abnormality of an adipic acid reaction chamber based on heterogeneous feature attention fusion is proposed for abnormal phenomena such as abnormal liquid level, shaking of stirring rod, crystallization of stirring rod contact area, boiling of liquid, crystallization of indoor wall, crystallization of siphon, and abnormal liquid level of siphon in the adipic acid reaction chamber. Based on the attention mechanism, image information and target information are efficiently fused, and representative target features are mined, thereby realizing intelligent monitoring of abnormal targets. Real-time monitoring of abnormal phenomena such as abnormal liquid level, shaking of stirring rod, crystallization of stirring rod contact area, boiling of liquid, crystallization of indoor wall, crystallization of siphon, and abnormal liquid level of siphon in the adipic acid reaction chamber is realized, and the safety level of the adipic acid reaction chamber production scene is improved.

[0163] Compared with the existing method that uses a cascaded transformer network to extract and fuse the spatiotemporal features of the image to identify the target, this method does not consider the quality improvement technology such as image deblurring based on space-frequency attention involved in the present invention, and the extraction and fusion of heterogeneous features and the target loss function are different, and the two are quite different. According to the characteristics of the special environment images of the adipic acid production process, the present invention constructs an integrated deep learning solution from image deblurring to abnormal monitoring. In special environments such as image blur, the method of the present invention has stronger accuracy and robustness. Specific embodiment 2:

[0165] See also Figure 3 and Figure 4 As shown, a system for monitoring adipic acid reaction chamber crystallization abnormality based on attention fusion includes: a data acquisition module 1, an image deblurring module 2 and an abnormal target detection module 3.

[0166] The data acquisition module 1 is used to collect the image I of the adipic acid reaction chamber. blur ;

[0167] The image deblurring module 2 is used to construct an image deblurring model based on space-frequency attention fusion, and divide the adipic acid reaction chamber image into N non-overlapping image blocks P, extract the implicit semantic features in each image block according to the image deblurring model, and reconstruct the deblurred image I deblur ; Using gradient loss, total variation loss and residual loss function to deblur the image I deblurDeblurring is performed to obtain a clear image. The specific processing method is the same as that in the specific embodiment 1, so it is omitted in this specific embodiment.

[0168] The abnormal target detection module 3 is used to extract clear image features from the clear image through a multi-head self-attention module, and then extract target attention features from related targets through the multi-head self-attention module. Based on heterogeneous feature attention fusion, the clear image features and the target attention features are fused with different category features through a multi-head cross-attention module to obtain representative abnormal image features F of the adipic acid reaction chamber, thereby realizing heterogeneous feature attention fusion; the abnormal image features of the adipic acid reaction chamber are passed through multiple fully connected network layers, and the probability of each category is obtained by using the softmax function to predict the category and bounding box of the target, and an abnormal target detection model for the adipic acid reaction chamber is constructed through the bounding box loss, classification loss and confidence loss function to perform real-time monitoring of abnormal targets. The specific processing method features are the same as those of specific embodiment 1, so this specific embodiment is omitted here.

[0169] Possibly, an alarm module 4 is further included, and the alarm module 4 is electrically connected or communicatively connected to the abnormal target detection module 3 .

[0170] The technical solution provided by the present invention is described in detail above. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principle of the present invention, the present invention can also be improved and modified in a number of ways, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

Claims

1. A method for monitoring abnormal crystallization of adipic acid reaction chamber based on attention fusion, characterized in that: The steps include: S1: Acquire the image I of the adipic acid reaction chamber blur , and the adipic acid reaction chamber image I blur Split into N non-overlapping image blocks, S2: construct an image deblurring model based on space-frequency attention fusion, extract the implicit semantic features in each image block according to the image deblurring model, and reconstruct the deblurred image I deblur ; include: S201: In the spatial domain, a spatial attention mechanism is used to obtain the representative spatial features F of each image block. s ; Specifically include: For each image patch feature X in the image P , S2011: image block feature X P Add a learnable matrix E SPos To retain its original spatial position information, the image block feature is transformed into X′ P =X P +E SPos , and then calculate the attention score Attention(Q,K,V) between it and all other image patches: Q=LN(W Q ·X′ P ),K=LN(W K ·X′ P ),V=LN(W V ·X′ P ) Among them, W Q , W K and W V is a learnable weight matrix, d is the feature dimension, and LN is the layer normalization layer: S2012: Use the softmax function to normalize the attention scores, obtain the attention weights, and use the multi-head attention mechanism to mine the implicit information of image space in different dimensions MHA(Q,K,V): S2013: Residual connection is used to retain the original features, and MLP layer is applied to realize feature conversion and aggregation to obtain spatial domain features F s , as shown in the following formula: F′ s =MHA(Q,K,V)+X′ P F s =MLP(LN(F′ s ))+F′ s ; S202: In the frequency domain, the frequency domain attention features of each image block are mined and fused with the spatial domain features to enhance the feature expression and obtain the frequency domain feature F f ; Specifically include: S2021: Use the fully connected layer to transform the spatial features F of each image block mentioned above s Convert and perform Fourier transform to obtain the feature representation F in the corresponding frequency domain Q 、F K 、F V ; Q s =LN(W Q ·F s ),K s =LN(W K ·F s ),V s =LN(W V ·F s ) in is Fourier transform; S2022: Apply the multi-head attention mechanism in the frequency domain to calculate the frequency domain attention feature A between all image blocks f ; A f =MHA(F Q ,F K ,F V ) S2023: Frequency domain feature F f It is expressed as: Where ⊙ represents the dot product operation, and σ is a transformation function; S203: The frequency domain feature F f Perform inverse Fourier transform to obtain the deblurred image I deblur ; S3: Deblurring the image I deblur Perform deblurring processing to obtain a clear image; S4: The clear image in step S3 is subjected to a multi-head self-attention module to extract clear image features, and the related targets are subjected to a multi-head self-attention module to extract target attention features. Based on heterogeneous feature attention fusion, the clear image features and the target attention features are subjected to a multi-head cross-attention module to perform fusion of different categories of features to obtain representative abnormal image features of the adipic acid reaction chamber; Specifically include: S401: For the clear image block feature X, first embed a position feature matrix In this way, the original position information of each clear image block is retained, and the clear image block feature X is transformed into Then, different linear layers are used to convert it into Q, K and V of the multi-head self-attention layer to perform attention calculation between image blocks, and then the MLP layer is used to transform the features to obtain the clear image feature F img ; X″=MHA(LN(X′),LN(X′),LN(X′)) F img =MLP(LN(X″+X′))+X″+X′; S402: For the target feature Y, first embed a position feature matrix To retain the original information of the target in the image, the target feature Y is transformed into Then, different linear layers are used to convert it into Q, K and V of the multi-head self-attention layer to perform attention calculation between image blocks, and then the MLP layer is used to transform the features to obtain the target attention feature F obj , used to represent the implicit information of the target; Y″=MHA(LN(Y′),LN(Y′),LN(Y′)) F obj =MLP(LN(Y″+Y′))+Y″+Y′ S402: In order to fuse the multi-dimensional heterogeneous features related to the target, a cross-attention module is constructed to fuse the above target-related clear image features F img and the target attention feature F obj , where the clear image feature F img After the linear layer conversion, it is used as Q of the cross-attention module and the target attention feature F obj Transformed into K and V across attention modules through different linear layers, F=MLP(LN(F′+F img +F obj )+F′+F img +F obj in and are different linear layers, used for feature conversion, and obtain the abnormal image feature F of the adipic acid reaction chamber that integrates different categories of information; S5: Predict the target category and bounding box of the abnormal image features of the adipic acid reaction chamber, and then build an abnormal target detection model for the adipic acid reaction chamber through bounding box loss, classification loss and confidence loss function to perform real-time monitoring of abnormal targets.

2. The method for monitoring abnormal crystallization of adipic acid reaction chamber based on attention fusion according to claim 1, characterized in that: The process also includes S6: if an abnormality is detected, issuing a target abnormality alarm.

3. The method for monitoring abnormal crystallization of adipic acid reaction chamber based on attention fusion according to claim 1, characterized in that: The deblurred image I deblur Deblurring is performed, including comprehensive processing of gradient loss, total variation loss and residual loss function.

4. The method for monitoring abnormal crystallization of adipic acid reaction chamber based on attention fusion according to claim 3, characterized in that: The comprehensive processing using gradient loss, total variation loss and residual loss function specifically includes: S301: Gradient loss focuses on the high-frequency information of the image and is used to measure the clarity of the image, expressed as L grad : in is the gradient at (i, j) in the deblurred image; S302: Total variation loss promotes the sparsity of the image and enhances the edge information of the image, expressed as L VT : S303: Residual loss focuses on the difference between the image before and after deblurring, denoted as L res : S304: Considering the above loss functions comprehensively, construct a deblurring loss function L for the adipic acid reaction chamber image deblur It is expressed as: L deblur =αL grad +βL VT +γL res 。 5. The method for monitoring abnormal crystallization of adipic acid reaction chamber based on attention fusion according to claim 1, characterized in that: In step S5, a bounding box loss, classification loss and confidence loss function are used to construct an abnormal target detection model for the adipic acid reaction chamber to perform real-time monitoring of abnormal targets, which specifically includes: The bounding box loss uses GIoU loss to measure the difference between the predicted bounding box and the true bounding box. GIoU loss not only considers the overlapping area of ​​the predicted box and the true box, but also their minimum bounding rectangle. Among them, b pred and b gt are the predicted and true bounding boxes, respectively, I is the intersection area of ​​the predicted box and the true box, U is the union area of ​​the predicted box and the true box, and A c is the area of ​​the minimum enclosed region containing the predicted box and the true box; The classification loss uses the cross entropy loss function to measure the accuracy of category prediction: Among them, C is the number of categories, y c is the one-hot encoding of the true label, p c The probability of the model detecting category c; Confidence loss: high confidence for correct bounding boxes, low confidence for incorrect bounding boxes: in, is the predicted probability of the target existing, p obj is the true probability of the target existing, is the predicted probability of category c; Through bounding box loss, classification loss and confidence loss function, a comprehensive loss function L is constructed zh The abnormal target detection model for the adipic acid reaction chamber is constructed as follows: L zh =λ grad L grad +λ VT L VT +λ res L res where λ grad , VT and λ res It is the weight parameter corresponding to the bounding box loss, classification loss and confidence loss.

6. A system for monitoring abnormal crystallization of adipic acid reaction chamber based on attention fusion, characterized in that: include: Data acquisition module, image deblurring module and abnormal target detection module, The data acquisition module is used to collect the image I of the adipic acid reaction chamber. blur ; The image deblurring module is used to construct an image deblurring model based on space-frequency attention fusion, and divide the adipic acid reaction chamber image into N non-overlapping image blocks P, extract the implicit semantic features in each image block according to the image deblurring model, and reconstruct the deblurred image I deblur ; include: S201: In the spatial domain, a spatial attention mechanism is used to obtain the representative spatial features F of each image block. s ; Specifically include: For each image patch feature X in the image P , S2011: image block feature X P Add a learnable matrix E SPos To retain its original spatial position information, the image block feature is transformed into X' P =X P +E SPos , and then calculate the attention score Attention(Q,K,V) between it and all other image patches: Q=LN(W Q ·X′ P ),K=LN(W K ·X' P ),V=LN(W V ·X' P ) Among them, W Q , W K and W V is a learnable weight matrix, d is the feature dimension, and LN is the layer normalization layer: S2012: Use the softmax function to normalize the attention scores, obtain the attention weights, and use the multi-head attention mechanism to mine the implicit information of image space in different dimensions MHA(Q,K,V): S2013: Residual connection is used to retain the original features, and MLP layer is applied to realize feature conversion and aggregation to obtain spatial domain features F s , as shown in the following formula: F′ s =MHA(Q,K,V)+X′ P F s =MLP(LN(F′ s ))+F′ s ; S202: In the frequency domain, the frequency domain attention features of each image block are mined and fused with the spatial domain features to enhance the feature expression and obtain the frequency domain feature F f ; Specifically include: S2021: Use the fully connected layer to transform the spatial features F of each image block mentioned above s Convert and perform Fourier transform to obtain the feature representation F in the corresponding frequency domain Q 、F K 、F V ; Q s =LN(W Q ·F s ),K s =LN(W K ·F s ),V s =LN(W V ·F s ) in is Fourier transform; S2022: Apply the multi-head attention mechanism in the frequency domain to calculate the frequency domain attention feature A between all image blocks f ; A f =MHA(F Q ,F K ,F V ) S2023: Frequency domain feature F f It is expressed as: Where ⊙ represents the dot product operation, and σ is a transformation function; S203: The frequency domain feature F f Perform inverse Fourier transform to obtain the deblurred image I deblur ; For the deblurred image I deblur Perform deblurring processing to obtain a clear image; The abnormal target detection module is used to extract clear image features from the clear image through a multi-head self-attention module, extract target attention features from related targets through a multi-head self-attention module, and fuse the clear image features and the target attention features with different categories through a multi-head cross-attention module based on heterogeneous feature attention fusion to obtain representative abnormal image features of the adipic acid reaction chamber; specifically includes: S401: For the clear image block feature X, first embed a position feature matrix In this way, the original position information of each clear image block is retained, and the clear image block feature X is transformed into Then, different linear layers are used to convert it into Q, K and V of the multi-head self-attention layer to perform attention calculation between image blocks, and then the MLP layer is used to transform the features to obtain the clear image feature F img ; X″=MHA(LN(X′),LN(X′),LN(X′)) F img =MLP(LN(X″+X′))+X″+X; S402: For the target feature Y, first embed a position feature matrix To retain the original information of the target in the image, the target feature Y is transformed into Then, different linear layers are used to convert it into Q, K and V of the multi-head self-attention layer to perform attention calculation between image blocks, and then the MLP layer is used to transform the features to obtain the target attention feature F obj , used to represent the implicit information of the target; Y″=MHA(LN(Y′),LN(Y′),LN(Y′)) F obj =MLP(LN(Y″+Y′))+Y″+Y′ S402: In order to fuse the multi-dimensional heterogeneous features related to the target, a cross-attention module is constructed to fuse the above target-related clear image features F img and the target attention feature F obj , where the clear image feature F img After the linear layer conversion, it is used as Q of the cross-attention module and the target attention feature F obj Transformed into K and V across attention modules through different linear layers, F=MLP(LN(F′+F img +F obj ))+F′+F img +F obj in and are different linear layers, used for feature conversion, and obtain the abnormal image feature F of the adipic acid reaction chamber that integrates different categories of information; The abnormal image features of the adipic acid reaction chamber are used to predict the category and bounding box of the target, and an abnormal target detection model for the adipic acid reaction chamber is constructed through bounding box loss, classification loss and confidence loss functions to perform real-time monitoring of abnormal targets.

Images