Text axis attention mechanism-based depression text detection method

Abstract

The invention provides a depression text detection method based on a text axis attention mechanism. The depression text detection method comprises the steps of obtaining a to-be-detected text; respectively extracting semantic features and psychological features of the to-be-detected text by utilizing the trained first model and second model; the semantic features and the psychological features are processed through a trained third model, a detection result is obtained, the third model adopts a double-branch architecture design to achieve extraction and fusion of global features and local features, and enhancement of the global features is achieved in one branch based on a text axis attention mechanism. Through the third model of the double-branch architecture, global features and local features of the text can be extracted at the same time, the two types of features are dynamically fused, and when the global features are extracted, the global features are enhanced in combination with a text axis self-attention mechanism and a multi-scale feature fusion technology, so that the accuracy and robustness of depression detection are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of text detection technology, and in particular to a method for detecting depression text based on a text axis attention mechanism. Background Art

[0002] With the widespread application of text data in mental health analysis, automated depression detection has become a research hotspot. Traditional methods rely on manual interviews or self-assessment scales, which suffer from low efficiency and high subjectivity. While existing natural language processing (NLP)-based technologies can achieve automated screening, they still face significant bottlenecks: First, global semantic modeling is insufficient. Traditional models (such as bag-of-words models or shallow neural networks) struggle to capture the contextual dependencies of long texts, leading to biased understanding of implicit semantics. Second, local sentiment analysis is crude. Methods based on sentiment lexicons or single convolutional kernels are unable to identify fine-grained emotional fluctuations in text (such as metaphorical expressions and ambivalence) and are prone to misjudgment in complex contexts. More importantly, existing methods often process global and local features independently, lacking dynamic fusion mechanisms and failing to exploit the complementarity of the two types of information. For example, pre-trained models such as BERT excel at global semantic modeling but overlook the importance of local sentiment words. Local feature extractors such as CNNs, while capable of capturing details, struggle to model long-range dependencies. This fragmented approach limits the model's generalization ability in real-world scenarios, particularly with unstructured data. Summary of the Invention

[0003] In view of the above defects of the prior art, the present invention provides a depression text detection method based on the text axis attention mechanism to solve the technical problem of low detection accuracy in the prior art.

[0004] To achieve the above-mentioned purpose and other related purposes, the present invention provides a method for detecting depression text based on a text-axis attention mechanism, comprising: obtaining a text to be detected; extracting the semantic features and psychological features of the text to be detected using a trained first model and a second model respectively; processing the semantic features and the psychological features using a trained third model to obtain a detection result, wherein the third model adopts a dual-branch architecture design to realize the extraction and fusion of global features and local features, and realizes the enhancement of global features based on the text-axis attention mechanism in one of the branches.

[0005] In one embodiment of the present invention, the first model is a BERT model, and the second model is a MentalRoBERTa model.

[0006] In one embodiment of the present invention, the BERT model extracts global contextual semantic features through a bidirectional Transformer encoding layer and outputs a contextual representation of each word or phrase; the Mental RoBERTa model is pre-trained with a psychology-related corpus to extract emotional features related to psychological states.

[0007] In one embodiment of the present invention, the third model includes a global feature encoding block, a multi-scale convolutional neural network, a gated fusion module and a multi-layer perceptual classifier; the trained third model is used to process the semantic features and the psychological features to obtain a detection result, including: using the global feature encoding block to perform self-attention encoding and cross-interaction on the semantic features and the psychological features to obtain a global feature; using the multi-scale convolutional neural network to extract the semantic features in parallel through convolution kernels of different scales to obtain a local feature; using the gated fusion module to fuse the global features and the local features to obtain a fused feature; using the multi-layer perceptual classifier to process the fused feature to obtain the detection result.

[0008] In one embodiment of the present invention, there are N stacks of global feature encoding blocks, where N∈{1,2,3,4,5}.

[0009] In one embodiment of the present invention, the global feature encoding block is used to perform self-attention encoding and cross-interaction on the semantic features and the psychological features to obtain global features, including: performing layer normalization and text-axis self-attention processing on the semantic features and the psychological features to obtain first features and second features; using the first feature and the second feature, cross-updating the semantic features and the psychological features to obtain third features and fourth features; performing layer normalization, nonlinear transformation and residual connection on the third feature and the fourth feature to obtain fifth features and sixth features; repeating the above steps N times, and taking the average of the fifth feature and the sixth feature obtained in the last time to obtain the global feature.

[0010] In one embodiment of the present invention, the semantic features are subjected to layer normalization and text-axis self-attention processing to obtain a first feature, including: performing layer normalization processing on the semantic features; performing linear mapping on the normalized semantic features to obtain a query matrix, a key matrix, and a value matrix; calculating the similarity score between the query matrix and the key matrix, and introducing a position attenuation factor to correct the similarity score; and weightedly summing the corrected similarity score with the value matrix to obtain the first feature.

[0011] In one embodiment of the present invention, the semantic feature and the psychological feature are cross-updated using the first feature and the second feature to obtain the third feature and the fourth feature, including: cross-updating using the following formula: X3=X BERT + Dropout(X2), X4=X Mental +Dropout(X1); where X BERT 、X Mental , X1, X2, X3, and X4 are respectively the semantic feature, the psychological feature, the first feature, the second feature, the third feature, and the fourth feature.

[0012] In one embodiment of the present invention, the third feature and the fourth feature are respectively subjected to layer normalization, nonlinear transformation and residual connection to obtain the fifth feature and the sixth feature, including: X5=X3+FFN(LayerNorm(X3)), X6=X4+FFN(LayerNorm(X4)); wherein X5 and X6 are the fifth feature and the sixth feature respectively, LayerNorm is a layer normalization operation, and FFN is a feedforward network for performing nonlinear transformation.

[0013] In one embodiment of the present invention, the multi-scale convolutional neural network is used to extract the semantic features in parallel through convolution kernels of different scales to obtain local features, including: transposing the semantic features to obtain transposed semantic features; using parallel one-dimensional convolution layers with convolution kernel sizes of 2, 3, and 4 to process the transposed semantic features respectively, and trimming or padding the convolution results to the original length to obtain multi-scale features; splicing the multi-scale features, and performing transposition and layer normalization processing to obtain the local features.

[0014] In one embodiment of the present invention, the gated fusion module is used to fuse the global features and the local features to obtain a fused feature, including: concatenating the global features and the local features; and generating a gating weight using the following formula:

[0015] gate=σ([X global ;X local ]W g +b g );

[0016] According to the gating weight, the fusion feature is calculated according to the following formula:

[0017] X fused =LayerNorm((gate·X global +(1-gate)·X local )W o +b o ); where Xlocal is the local feature, X global is the global feature, X fused is the fusion feature, W g 、W o 、b g 、b o are all learnable parameters, σ is the Sigmoid activation function, and LayerNorm is the layer normalization.

[0018] In one embodiment of the present invention, the multi-layer perceptron classifier is used to process the fused features to obtain the detection result, including: taking the average of the fused features along the sequence dimension, and mapping them to the first dimension through the first fully connected layer; performing ReLU activation and Dropout processing on the mapped features in sequence; mapping the processed features to two dimensions through the second fully connected layer, and performing normalization processing to obtain the probability distribution of normal categories and depression categories; and obtaining the detection result according to the category corresponding to the maximum probability in the probability distribution.

[0019] Beneficial effects of the present invention: The present invention proposes a method for detecting depression text based on a text-axis attention mechanism. This method can simultaneously extract global features and local features of the text through a third model of a dual-branch architecture, and dynamically fuse the two types of features. When extracting global features, the method combines the text-axis self-attention mechanism with multi-scale feature fusion technology to enhance global features, significantly improving the accuracy and robustness of depression detection. This method is applicable to a wide range of scenarios and provides an efficient automated solution for mental health screening. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 A general flow chart of a detection method provided in one embodiment of the present invention;

[0022] Figure 2 A detailed flow chart of step S300 provided in one embodiment of the present invention;

[0023] Figure 3 This is an architectural diagram of the entire detection model provided by one embodiment of the present invention;

[0024] Figure 4A detailed flowchart of step S310 provided in one embodiment of the present invention;

[0025] Figure 5 This is an architectural diagram of a global feature encoding block provided by one embodiment of the present invention;

[0026] Figure 6 A detailed flowchart of step S320 provided in one embodiment of the present invention;

[0027] Figure 7 An architectural diagram of a multi-scale convolutional neural network provided by one embodiment of the present invention;

[0028] Figure 8 A detailed flowchart of step S330 provided in one embodiment of the present invention;

[0029] Figure 9 This is a detailed flowchart of step S340 provided in one embodiment of the present invention. DETAILED DESCRIPTION

[0030] The following describes the embodiments of the present invention through specific embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. It should be noted that the following embodiments and the features in the embodiments can be combined with each other unless they conflict. In addition to the specific methods, equipment, and materials used in the embodiments, based on the understanding of the prior art by those skilled in the art and the description of the present invention, any methods, equipment, and materials of the prior art that are similar or equivalent to the methods, equipment, and materials in the embodiments of the present invention can also be used to implement the present invention.

[0031] It should be understood that the terms used in the examples of the present invention are for describing specific embodiments rather than for limiting the scope of protection of the present invention. Unless otherwise defined, all technical and scientific terms used in the present invention have the same meaning as those generally understood by those skilled in the art.

[0032] In the following description, numerous details are discussed to provide a more thorough explanation of the embodiments of the present invention. However, it is obvious to those skilled in the art that the embodiments of the present invention can be implemented without these specific details. In some of the embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring the embodiments of the present invention.

[0033] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functions and operations that may be implemented by the methods and computer program products of various embodiments disclosed in the present invention. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0034] See Figure 1 , Figure 1 An embodiment of the present invention provides a method for detecting depression text based on a text axis attention mechanism, including steps S100 to S300.

[0035] Step S100: Obtain the text to be detected. The present invention mainly detects depression based on text, so firstly, it is necessary to obtain the text to be detected, which can be, for example, a text written by a user or a text converted from a user's voice during communication.

[0036] Step S200: Using the trained first model and second model to extract the semantic features and psychological features of the text to be detected. In this step, the pre-trained existing model is used to detect the semantic features and psychological features of the text to be detected, so as to perform feature extraction and fusion in the future.

[0037] In one embodiment of the present invention, the first model is the BERT model and the second model is the Mental RoBERTa model. The BERT model is a public pre-trained language model that is widely used in natural language processing tasks. The BERT model extracts global contextual semantic features through a bidirectional Transformer encoding layer and outputs a contextual representation of each word or phrase. The Mental RoBERTa model is fine-tuned by the RoBERTa model (Facebook's improved BERT variant) using a psychology corpus to extract emotional features related to psychological states. The resulting semantic features can be denoted as X BERT , psychological characteristics can be recorded as X Mental , they have the same size, [B,L,768], where B is the batch size and L is the sequence length.

[0038] During the model training process, you first need to pre-train the BERT model and the Mental RoBERTa model, or you can directly use the weights of other people's trained models; then you can train the third model. When training the third model, you need to freeze the parameters of the BERT model and the Mental RoBERTa model and only update the weight parameters of the third model.

[0039] Step S300 uses the trained third model to process the semantic features and psychological features to obtain the detection results, wherein the third model adopts a dual-branch architecture design to realize the extraction and fusion of global features and local features, and in one of the branches, the global features are enhanced based on the text axis attention mechanism.

[0040] See Figure 2 In a specific embodiment of the present invention, the third model includes a global feature encoding block, a multi-scale convolutional neural network, a gated fusion module and a multi-layer perception classifier; step S300 includes steps S310 to S340. Figure 3 The diagram shows the architecture of the entire network model including the first model, the second model and the third model, where the blue circle represents the Bert feature and the orange circle represents the Mental feature.

[0041] Step S310: Use the global feature encoding block to perform self-attention encoding and cross-interaction on the semantic features and psychological features to obtain global features.

[0042] See Figure 4 In a specific embodiment of the present invention, step S310 includes steps S311 to S314, and the structure of the global feature encoding block is as follows: Figure 5 shown.

[0043] Step S311: Perform layer normalization and text-axis self-attention processing on the semantic features and psychological features to obtain the first feature and the second feature. In this step, the semantic features and psychological features are processed separately to obtain the first feature and the second feature. Their processing processes are the same.

[0044] In a specific implementation of the present invention, semantic features are taken as an example for illustration, and layer normalization and text axis self-attention processing are performed on the semantic features to obtain the first feature, which specifically includes the following steps.

[0045] (1) Perform layer normalization (LayerNorm) on semantic features.

[0046] (2) Perform linear mapping on the normalized semantic features to obtain the query matrix Q, key matrix K, and value matrix V, which can be expressed as follows:

[0047] Q=W qX BERT , K=W k X BERT , V=W v X BERT .

[0048] (3) Calculate the similarity scores of the query matrix and the key matrix, and introduce the position attenuation factor to correct the similarity scores, which can be expressed as follows:

[0049]

[0050] In the formula, scores is the corrected similarity score, and softplus is used instead of softmax to make the attention score smoother and avoid excessive sparsity. T is the transpose of the key matrix, L is the sequence length, and ζ is the position decay factor, which can be calculated as follows:

[0051] ζ=1 / log(ω·|δ|+c+∈),

[0052] Where ω and c are learnable parameters, and their initial values ​​are the natural constant e; |δ| is the absolute distance between two positions in the sequence (for example, the distance between the first and fifth words is 4); ∈ = 10 -8 , is a very small constant to prevent numerical overflow. As the distance |δ| increases, ζ decreases, reducing the attention weight of distant tokens to avoid noise interference. Compared with traditional fixed attenuation (such as relative position encoding), the use of a learnable dynamic position attenuation factor can more flexibly model long-distance dependencies.

[0053] (4) The modified similarity score is weighted and summed with the value matrix to obtain the first feature, which can be expressed as: X1 = scores·V.

[0054] It can be understood that in the above step (1), the psychological characteristics are input, and the final calculated result is the second characteristic, and the entire calculation process is the same.

[0055] Step S312: Using the first feature and the second feature, cross-update the semantic feature and the psychological feature to obtain the third feature and the fourth feature.

[0056] In a specific embodiment of the present invention, cross-update is performed using the following formula:

[0057] X3=X BERT + Dropout(X2), X4=X Mental +Dropout(X1);

[0058] Where, X BERT 、X Mental, X1, X2, X3, X4 are semantic features, psychological features, first features, second features, third features, and fourth features respectively; Dropout is a regularization technique that randomly "discards" (sets to zero) some neurons in the neural network to prevent overfitting. Let the semantic feature X BERT and psychological characteristics X Mental They complement each other, enhance the model's ability to capture implicit emotions (such as metaphorical expressions related to depression), and achieve two-way information transmission.

[0059] Step S313: perform layer normalization, nonlinear transformation, and residual connection on the third feature and the fourth feature respectively to obtain the fifth feature and the sixth feature.

[0060] In a specific embodiment of the present invention, step S313 includes:

[0061] X5=X3+FFN(LayerNorm(X3)), X6=X4+FFN(LayerNorm(X4));

[0062] Where X5 and X6 are the fifth and sixth features, respectively. LayerNorm is the layer normalization operation. FFN is a feedforward network for performing nonlinear transformations. The feedforward network may include a linear layer, a ReLU activation function, and a Dropout operation.

[0063] Step S314: Repeat the above steps N times, and take the average of the fifth feature and the sixth feature obtained in the last step to obtain a global feature.

[0064] In a specific embodiment of the present invention, the global feature encoding blocks can be stacked N, N∈{1,2,3,4,5}, and optimally, N=3. When N=1, the fifth feature and the sixth feature obtained in step S313 can be directly averaged to obtain the global feature X global When N>1, that is, the number of stacked global feature encoding blocks is greater than 1, it is necessary to repeat the above steps S311 to S313, that is, the fifth and sixth features obtained by the first execution of step S313 are used as the input of the next round to obtain new fifth and sixth features. Finally, the fifth and sixth features obtained for the Nth time are averaged to obtain the global feature X. global .

[0065] Step S320: Utilize a multi-scale convolutional neural network to extract semantic features in parallel through convolution kernels of different scales to obtain local features.

[0066] See Figure 6 In a specific embodiment of the present invention, step S320 includes steps S321 to S323. In this step, the structure of the multi-scale convolutional neural network is as follows: Figure 7shown.

[0067] Step S321: Transpose the semantic features to obtain the transposed semantic features. BERT For example, after transposition, its dimension becomes [B, 768, L].

[0068] Step S322: Use parallel one-dimensional convolution layers with convolution kernel sizes of 2, 3, and 4 to process the transposed semantic features respectively, and trim or pad the convolution results to the original length L to obtain multi-scale features.

[0069] Step S323: Splice multi-scale features, perform transposition and layer normalization processing to obtain local features X local .

[0070] Step S330: Using a gated fusion module, the global features and the local features are fused to obtain fused features.

[0071] See Figure 8 In a specific embodiment of the present invention, step S330 includes steps S331 to S333.

[0072] Step S331: Splice global features X global and local features X local , the concatenated features can be recorded as [X global ,X local ], the sizes of global features and local features are both [B, L, 768], and the size of the spliced ​​features becomes [B, L, 1536].

[0073] Step S332: Generate the gating weight using the following formula:

[0074] gate=σ([X global ;X local ]W g +b g ); In this step, the gate weight gate is generated by the full connection layer and the Sigmoid activation function, where W g and b g are all learnable parameters, After calculation, the size of the gating weight is [B,L,768].

[0075] Step S333: Calculate the fusion feature according to the following formula based on the gating weight:

[0076] X fused =LayerNorm((gate·X global +(1-gate)·X local )W o +bo ); where X fused is the fusion feature, W o and b o are all learnable parameters, LayerNorm is layer normalization. After this step, the final fusion feature X fused The size is also [B, L, 768].

[0077] Step S340: Use a multi-layer perception classifier to process the fusion features to obtain a detection result.

[0078] See Figure 9 In a specific embodiment of the present invention, step S340 includes steps S341 to S344.

[0079] Step S341: After averaging the fused features along the sequence dimension, the fused features are mapped to the first dimension through the first fully connected layer. The sequence dimension is L. After averaging the sequence dimension, the resulting feature size is B × 768. This feature is then mapped to the first dimension through the first fully connected layer. The first dimension can be 384, for example.

[0080] Step S342: Perform ReLU activation and Dropout processing on the mapped features in sequence. ReLU activation is used to introduce nonlinearity and enhance expressiveness; Dropout prevents overfitting.

[0081] Step S343: Map the processed features to two dimensions through the second fully connected layer and perform normalization to obtain the probability distribution of the normal and depression categories. The resulting two-dimensional vector can be recorded as [P1, P2] after normalization, where P1 represents the probability that the sample belongs to the normal category and P2 represents the probability that the sample belongs to the depression category.

[0082] Step S344: Obtain a test result based on the category corresponding to the maximum probability value in the probability distribution. For example, if a probability distribution is [0.7, 0.3], where the maximum probability value is 0.7, the corresponding category is normal, then the test result for the text to be tested is normal; or if a probability distribution is [0.2, 0.8], where the maximum probability value is 0.8, the corresponding category is depression, then the test result for the text to be tested is depression.

[0083] It should be noted that the step division of the various methods above is only for the purpose of clear description. During implementation, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they contain the same logical relationship, they are all within the scope of protection of this application; adding insignificant modifications to the algorithm or process or introducing insignificant designs without changing the core design of the algorithm and process are all within the scope of protection of this patent.

[0084] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the present invention. Anyone skilled in the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by one of ordinary skill in the art without departing from the spirit and technical principles disclosed herein are intended to be covered by the claims of the present invention.

Claims

1. A depression text detection method based on text axis attention mechanism, characterized by: include: Get the text to be detected; Extracting semantic features and psychological features of the text to be detected using the trained first model and the second model respectively; The semantic features and the psychological features are processed using the trained third model to obtain detection results, wherein the third model adopts a dual-branch architecture design to realize the extraction and fusion of global features and local features, and in one of the branches, the global features are enhanced based on the text axis attention mechanism.

2. The depression text detection method based on the text axis attention mechanism according to claim 1 is characterized in that The first model is a BERT model, and the second model is a Mental RoBERTa model.

3. The depression text detection method based on the text axis attention mechanism according to claim 1 is characterized in that The third model includes a global feature encoding block, a multi-scale convolutional neural network, a gated fusion module and a multi-layer perception classifier; The semantic features and the psychological features are processed using the trained third model to obtain a detection result, including: Using the global feature encoding block, self-attention encoding and cross-interaction are performed on the semantic features and the psychological features to obtain global features; Utilizing the multi-scale convolutional neural network, the semantic features are extracted in parallel by convolution kernels of different scales to obtain local features; Using the gated fusion module, the global feature and the local feature are fused to obtain a fused feature; The fusion features are processed using the multi-layer perceptron classifier to obtain the detection result.

4. The depression text detection method based on the text axis attention mechanism according to claim 3 is characterized in that There are N global feature encoding blocks stacked, N∈{1,2,3,4,5}.

5. The depression text detection method based on the text axis attention mechanism according to claim 3 is characterized in that The global feature encoding block is used to perform self-attention encoding and cross-interaction on the semantic features and the psychological features to obtain global features, including: performing layer normalization and text axis self-attention processing on the semantic feature and the psychological feature respectively to obtain a first feature and a second feature; cross-updating the semantic feature and the psychological feature using the first feature and the second feature to obtain a third feature and a fourth feature; performing layer normalization, nonlinear transformation, and residual connection on the third feature and the fourth feature respectively to obtain a fifth feature and a sixth feature; Repeat the above steps N times, and take the average of the fifth feature and the sixth feature obtained in the last step to obtain the global feature.

6. The depression text detection method based on the text axis attention mechanism according to claim 5 is characterized in that The semantic features are subjected to layer normalization and text axis self-attention processing to obtain a first feature, including: Performing layer normalization processing on the semantic features; Performing linear mapping on the normalized semantic features to obtain a query matrix, a key matrix, and a value matrix; Calculating a similarity score between the query matrix and the key matrix, and introducing a position attenuation factor to correct the similarity score; The first feature is obtained by performing a weighted summation of the modified similarity score and the value matrix.

7. The depression text detection method based on the text axis attention mechanism according to claim 5 is characterized in that Using the first feature and the second feature, cross-updating the semantic feature and the psychological feature to obtain a third feature and a fourth feature, including: Cross-update is performed using the following formula: X3=X BERT +Dropout(X2),X4=X Mental +Dropout(X1); Where, X BERT 、X Mental , X1, X2, X3, and X4 are respectively the semantic feature, the psychological feature, the first feature, the second feature, the third feature, and the fourth feature.

8. The depression text detection method based on the text axis attention mechanism according to claim 5 is characterized in that The third feature and the fourth feature are respectively subjected to layer normalization, nonlinear transformation, and residual connection to obtain a fifth feature and a sixth feature, including: X5=X3+FFN(LayerNorm(X3)), X6=X4+FFN(LayerNorm(X4)); In the formula, X5 and X6 are the fifth feature and the sixth feature respectively, LayerNorm is layer normalization, and FFN is a feedforward network for performing nonlinear transformation.

9. The depression text detection method based on the text axis attention mechanism according to claim 3 is characterized in that The multi-scale convolutional neural network is used to extract the semantic features in parallel through convolution kernels of different scales to obtain local features, including: performing transposition processing on the semantic features to obtain transposed semantic features; Parallel one-dimensional convolutional layers with kernel sizes of 2, 3, and 4 are used to process the transposed semantic features respectively, and the convolution results are cropped or padded to the original length to obtain multi-scale features; The multi-scale features are spliced, and transposed and layer normalized to obtain the local features.

10. The depression text detection method based on the text axis attention mechanism according to claim 3 is characterized in that The gated fusion module is used to fuse the global features and the local features to obtain fused features, including: splicing the global features and the local features; The gating weight is generated by the following formula: gate=σ([X global ;X local ]W g +b g ); According to the gating weight, the fusion feature is calculated according to the following formula: X fused =LayerNorm((gate·X global +(1-gate)·X local )W o +b o ); Among them, X local is the local feature, X global is the global feature, X fused is the fusion feature, W g 、W o 、b g 、b o All are learnable parameters, σ is the Sigmoid activation function, and LayerNorm is layer normalization.

Images