Biological tissue recognition method based on axial free attention network

Abstract

The invention relates to a biological tissue recognition method based on an axial free attention network, and the method comprises the steps: constructing an axial free attention converter which employs an encoder-decoder structure, and comprises a plurality of convolution blocks, axial free attention blocks, and an up-sampling block, and each axial free attention block comprises a plurality of convolution blocks, a plurality of axial free attention blocks, and a plurality of up-sampling blocks; comprising an input network, D axial free attention mechanism networks, a soft aggregation operation network, a 1 * 1 convolutional network and an output network, each axial free attention mechanism network processes the input image feature data according to different angle directions; training the axial free attention converter by using training sample data to obtain a biological tissue recognition processing training model; and identifying and processing the biological tissue image data by using the biological tissue identification processing training model. According to the method, non-local feature maps with long-distance clues are captured with relatively low computing resources, and diversified receptive fields are expanded.

Description

Technical Field

[0001] The present invention relates to a biological tissue recognition method based on an axial free attention network, and in particular to the construction and application of an axial free attention network for tissue recognition during the embryonic blastocyst period, and belongs to the technical field of image pattern recognition based on artificial neural networks and the technical field of artificial intelligence applied to medical treatment. Background Art

[0002] Since the implantation potential of IVF embryos is related to their morphology, the morphological structure including cell density, size and expansion level will determine the embryo quality grade, which makes the assessment of embryo quality dependent on the segmentation of blastocyst components. However, in the prior art, when selecting blastocysts with high implantation potential for transfer by visually evaluating their morphological characteristics, due to problems such as poor contrast of embryo images, noise and unclear boundaries of different tissue structures, it becomes a challenging problem to identify different tissue regions and their morphological structures in blastocyst images to achieve automatic blastocyst segmentation.

[0003] There are some technologies for automatic segmentation of blastocyst image recognition and segmentation in the existing technology. For example, the traditional computer vision method (level set) is used to segment the inner cell mass and trophectoderm; by combining text information (Gabor and DCT features) and the level set method to identify the ICM boundary, the feature expression can be enhanced to a certain extent; by introducing the clustering idea, the features of the same tissue can be better aggregated. With the development of deep learning, there is a method of inputting the extracted traditional features into a two-layer neural network to identify ZP, TE and ICM; there is also a pure end-to-end deep learning method, such as using a fully convolutional neural (FCN) network to segment the ICM. However, these methods only increase the perception field of a single pixel, but do not change the attribute relationship between pixels.

[0004] Ronneberger et al. proposed U-Net for the segmentation of biological microscope images. After that, various U-Net variants were proposed for medical image analysis, such as Attention U-Net, Unet++, DU-Net, PraNet, nnUnet, etc. These networks are not specifically designed for embryo data and only utilize effective locality at the expense of long-range clues.

[0005] There are also attempts to use CNN and Transformer-based methods for blastocyst image segmentation, but there are still two major challenges: (1) how to effectively capture non-local dependencies and alleviate local inconsistencies, such as Figure 1 (a) to address the complex representation of embryo morphology, especially issues regarding blurred edges and similar textures. (2) How to integrate global context features and obtain a wide receptive field to enhance the learning ability of the network. Summary of the invention

[0006] In order to solve the above technical problems, the present invention provides a biological tissue recognition method based on an axial free attention network, comprising the steps of: An axial free attention transformer is constructed. The axial free attention transformer adopts an encoder-decoder structure, including multiple convolution blocks, axial free attention blocks and upsampling blocks, wherein each axial free attention block includes an input network, D axial free attention mechanism networks, a soft aggregation operation network, a 1×1 convolution network and an output network; each axial free attention mechanism network processes the input image feature data according to different angular directions, and includes: a grid generator, a direction sampler, a one-dimensional attention module and a learnable sinusoidal relative position encoding module; wherein the grid generator is used to calculate the pixel position relationship between the input feature map and the generated direction feature map Z, the direction sampler generates the sampled direction feature map Z using a sampling grid, the one-dimensional attention module is used to perform one-dimensional self-attention mechanism calculation processing on the direction feature map Z, and the learnable sinusoidal relative position encoding is used to introduce continuous bias and adaptability; Acquire training sample data, where the training sample data is biological tissue image data, and the biological tissue image data is labeled data; Using the training sample data to train the axial free attention transformer to obtain a biological tissue recognition processing training model; Use the biological tissue recognition and processing training model to recognize and process biological tissue image data.

[0007] In the above technical solution, the number D of the axial-free attention mechanism networks is D × θ = π, where θ represents the angle between the attention axes processed by adjacent axial-free attention mechanism networks.

[0008] In the above technical scheme, the input network of each axial free attention block divides the input image feature data into D paths and inputs them into D axial free attention mechanism networks respectively. The D axial free attention mechanism networks process the input image feature data according to an angle direction respectively, and output the results to the soft aggregation operation network; the soft aggregation operation network soft-aggregates the output results of the D axial free attention mechanism networks and outputs them to the 1×1 convolution network; the output of the 1×1 convolution network is added to the input image feature data and output to the output network to obtain the output image feature data.

[0009] In the above technical solution, the pixel position relationship between the input feature map X and the directional feature map Z calculated by the grid generator is:

[0010] in, is the coordinate of the sample grid in the input feature map, θ is the angle of rotation of the processed attention axis relative to the coordinate axis, is the coordinate of the new pixel used to generate the directional feature map Z.

[0011] In the above technical solution, the kernel function of the sampling grid used by the direction sampler is:

[0012] in, and Defines the interpolation operation in the x and y directions, is the value of the (n, m) position on the feature map, and H and W are the height and width of the image data respectively.

[0013] In the above technical scheme, for a given pixel coordinate a=(i, j) and another pixel coordinate b=(l, n), the learnable sinusoidal relative position encoding (LSRPE) is defined as:

[0014] Among them, the learnable parameters , , is a relative position encoding.

[0015] In the above technical solution, the specific structure of the axial free attention converter includes: The first layer, including the first convolutional block; The second layer, includes the second convolution block and the first axis-free attention block; The third layer, includes the second axial free attention block; The fourth layer includes the third axis free attention block; The fifth layer, including the first upsampling block; The sixth layer, including the second upsampling block; The seventh layer, including the third upsampling block; The eighth layer, including the fourth upsampling block; Among them, the first layer is the input layer, and the eighth layer is the output layer. One of the two outputs of the first layer is input to the second layer, and the other is added to the output of the seventh layer and then input to the eighth layer; one of the two outputs of the second layer is input to the third layer, and the other is added to the output of the sixth layer and then input to the seventh layer; one of the two outputs of the third layer is input to the fourth layer, and the other is added to the output of the fifth layer and then input to the sixth layer; the output of the fourth layer is input to the fifth layer.

[0016] The present invention also provides a biological tissue recognition device based on an axial free attention network, which is characterized by comprising: A model building module is used to build an axial free attention transformer, which adopts an encoder-decoder structure, including multiple convolution blocks, axial free attention blocks and upsampling blocks, wherein each axial free attention block includes an input network, D axial free attention mechanism networks, a soft aggregation operation network, a 1×1 convolution network and an output network; each axial free attention mechanism network processes the input image feature data according to different angular directions, and includes: a grid generator, a direction sampler, a one-dimensional attention module and a learnable sinusoidal relative position encoding module; The grid generator is used to calculate the pixel position relationship between the input feature map and the generated directional feature map Z, the directional sampler generates the sampled directional feature map Z using the sampling grid, the one-dimensional attention module is used to perform one-dimensional self-attention mechanism calculation processing on the directional feature map Z, and the learnable sinusoidal relative position encoding is used to introduce continuous bias and adaptability; A training sample acquisition module is used to acquire training sample data, where the training sample data is biological tissue image data, and the biological tissue image data is labeled data; A model training module, using the training sample data to train the axial free attention transformer to obtain a biological tissue recognition processing training model; The data processing module uses the biological tissue recognition and processing training model to recognize and process the biological tissue image data.

[0017] The present invention also provides a biological tissue identification system based on an axial free attention network, comprising a processor and a storage medium; the storage medium stores a program code; the processor is used to call the program code stored in the storage medium to execute the biological tissue identification method based on the axial free attention network described in the above technical solution.

[0018] The present invention has achieved the following technical effects: First, the present invention proposes an axis-free attention mechanism, which uses lower computing resources to capture non-local feature maps with long-distance clues to reduce errors in local structures. Secondly, the present invention proposes an axis-free attention block with a soft aggregation operation, which embeds features extracted from different angles with axis-free attention features, which collect global clues and expand diverse receptive fields. By verifying on typical public datasets, the best segmentation performance was achieved, with accuracy, precision, recall, Dice coefficient and Jaccard index, with specific values ​​of 93.86%, 91.86%, 92.20%, 92.03% and 85.46% respectively; and extensive qualitative experimental results prove the effectiveness of this method. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is the detailed feature map of blastocyst tissue image, where Figure 1 (a) shows the local inconsistency in the blastocyst tissue image. Figure 1 (b) shows the rotational consistency of the blastocyst tissue image; Figure 2 This is the overall architecture diagram of BTFormer, where Figure 2 (a) shows the specific architecture of BTFormer. Figure 2 (b) shows the specific structure of the axial free attention block, where Figure 2 (c) shows the specific mode of soft aggregation operation; Figure 3 This is a structural diagram of axial free attention. DETAILED DESCRIPTION

[0020] In order to facilitate those skilled in the art to understand and implement the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0021] Local inconsistencies in blastocyst tissue images, such as Figure 1 As shown in (a), the blastocyst tissue image is captured using the Hoffman optical sampling system, which is well-known for its three-dimensional relief effect and transparent tissue properties. The overlapping structures present in the captured image will exacerbate this local inconsistency; in addition, blastocyst tissue images are usually captured under low light conditions. Although the entire embryo is transparent and has good brightness, the low contrast and noise of the captured image under low light conditions will further exacerbate this phenomenon. Figure 1 In (a), two small regions of interest (ROI 1 and ROI 2) are extracted from the embryo image, and their locations in the original image and its corresponding label image are marked. It is worth noting that ROI 1 and ROI 2 show similar colors and textures, although they contain / belong to different categories in the label image. This observation suggests that for blastocyst tissue images, the classification of a specific region is not only affected by its color and texture, but also by its spatial relationship with neighboring regions, which indicates the importance of contextual relationships in embryonic images. The main labels of human blastocysts, such as Figure 1 As shown in the label in (a), from inside to outside, it usually includes the inner cell mass (ICM), blastocoel, trophectoderm (TE), zona pellucida (ZP), and background. In particular, TE is usually surrounded by ZP and blastocoel from both sides, while ICM tends to be located within the boundaries of TE, closer to the center of the embryo. Figure 1In local views such as ROI 2 in (a), the combination of local inconsistency and blurred edges requires the integration of contextual and non-local information for accurate analysis. Figure 1 (b) shows the rotation consistency of the blastocyst tissue image and its semantic information is not lost when it is rotated at any angle around the center.

[0022] In order to cope with the above technical challenges in the prior art, the present invention proposes a biological tissue recognition method based on an axial free attention network, which is based on a new Transformer (Blast Transformer, BTFormer) architecture of an axial free attention layer and an axial free attention block. By introducing an axial free attention mechanism, non-local information can be captured from any direction at a reduced computational cost. This axial free attention block can utilize the rotation consistency principle of the blastocyst image to effectively extract global significant information. For simplicity, the BTFormer in the present invention can be referred to as an "axial free attention Transformer" or "axial free attention transformer".

[0023] The present invention proposes an axial free attention transformer (BTFormer), such as Figure 2 As shown in FIG. 1 , it adopts an encoder-decoder structure and combines multi-scale fusion. The overall architecture of the BTFormer of the present invention is as follows: Figure 2 As shown in (a), it uses a transformer-based backbone as an encoder to extract global features. The decoder consists of an upsampling block, including an upsampling operator and some simple convolutional layers (Conv-BN-ReLU), which retains a new independent Transformer base encoder by replacing the convolutional blocks in the standard decoder structure with the axial free attention block proposed by the present invention. The typical structure of the BTFormer of the present invention is as follows Figure 2 As shown in (a), the first layer is the input layer, which includes the first convolution block; the second layer includes the second convolution block and the first axial free attention block; the third layer includes the second axial free attention block; the fourth layer includes the third axial free attention block; the fifth layer includes the first upsampling block; the sixth layer includes the second upsampling block; the seventh layer includes the third upsampling block; the eighth layer is the output layer, including the fourth upsampling block; one of the two outputs of the first layer is input to the second layer, and the other is added to the output of the seventh layer and then input to the eighth layer; one of the two outputs of the second layer is input to the third layer, and the other is added to the output of the sixth layer and then input to the seventh layer; one of the two outputs of the third layer is input to the fourth layer, and the other is added to the output of the fifth layer and then input to the sixth layer; the output of the fourth layer is input to the fifth layer.

[0024] The specific structure of the axial free attention block in BTFormer proposed in this invention is as follows: Figure 2 As shown in (b), it is implemented as an independent convolution module, rather than on a high-level convolution position (bottleneck position) similar to Unet, which is suitable for capturing long-distance dependencies. In order to promote a wider receptive field, the present invention collects information from a wide range of angles by applying rotation consistency. The theoretical basis is that the embryo image maintains physiological significance when rotated at any angle (such as Figure 1 That is, semantic clues in any direction are used for segmentation. Therefore, the present invention evenly arranges multiple axial free attention layers in 2D space: D × =π (1) Where D is the number of axial free attention layers used, and θ represents the angle between two adjacent layers. Preferably, the present invention selects a parallel structure, D = 4.

[0025] like Figure 2 As shown in (b), the typical structure of each axis-free attention block includes an input network, D axis-free attention mechanism networks, a soft aggregation operation network (SAO), a 1×1 convolution network and an output network; the input network converts the input image feature data into It is divided into D paths and input into D axial free attention mechanism networks, where are the number of color channels, height, and width of the input image feature data respectively; the D axial free attention mechanism networks process the input image feature data in an angular direction respectively, and output the results to the soft aggregation operation network; the soft aggregation operation network soft aggregates the output results of the D axial free attention mechanism networks and outputs them to the 1×1 convolution network; the output of the 1×1 convolution network is added to the input image feature data and output to the output network to obtain the output image feature data ,in They are respectively the number of color channels, height, and width of the output image feature data.

[0026] Typically, the present invention applies four axis-free attention mechanism networks to obtain semantic information in four directions: According to formula (1), we can know that .

[0027] The soft aggregation operation network (SAO) of the axis-free attention block of the present invention, as in Figure 2 As shown in (c), it is used to collect semantic information in D directions (0, 45, 90 and 135 degrees are shown in the figure). By adjusting the value of D, you can expand or reduce This design enriches the contextual information in each direction and promotes the global receptive field. Correspondingly, this method also obtains rotation invariance. In addition, when the input is rotated When the value is a multiple of , the outputs share global connections from the same direction.

[0028] In the above operations that combine semantic information, the required semantic information is not always the same for each point in the feature map; therefore, we cannot simply add the four axis-free attention results. Figure 2 As shown in (c), the present invention assumes that the axial free attention block is composed of the outputs of D axial free attention mechanism networks, namely: The final result is obtained by the sum of two branches of the soft aggregation operation network (SAO), namely, and , with the same input .

[0029] for Figure 2 The left branch of the soft aggregation operation network (SAO) in (c) includes a two-layer structure connected in sequence, wherein the first layer includes a 1×1 convolutional network, a BN network (Batch Normalization) and a ReLU network, and the second layer includes a SoftMax operation network; the right branch of the soft aggregation operation network (SAO) includes a two-layer structure connected in sequence, wherein the first layer includes a connection operation network (Concat), and the second layer includes a 1×1 convolutional network. The output of each of the D axis-free attention mechanism networks is divided into three paths, wherein the first path of the D group is added to become a group of inputs to the left branch for processing, the second path of the D group is input to the right branch for processing, and the third path of the D group is multiplied with the D group outputs of the left branch and then added to become a group of outputs added to the right branch to obtain the output result.

[0030] Specifically, the left branch of the soft aggregation operation network (SAO) processes D groups of context features, and then aggregates them into a set using the aggregation operation; the right branch of the soft aggregation operation network (SAO) mainly implements the connection operation Cat. Therefore, the processing operation of the soft aggregation operation network (SAO) can be described by the following formula:

[0031]

[0032]

[0033]

[0034] in, represents the output of the d-th axis-free attention mechanism network, is the processing result of the first layer output of the left branch, represents the convolution operation, express Function operation; The processing result of the left branch compresses the biased information into one output, keeps the same size, and generates weighted and diversified features; It is the processing result of the right branch, which is processed through a simple connection operation to promote feature fusion; It is the processing result of the soft aggregation operation network.

[0035] In the Soft Aggregation Operation (SAO) network, the weights of each axis-free attention layer are learned through convolution, and the sum of the results of each axis-free attention layer is calculated. The final aggregation result of D axis-free attention layers Through the weighted and aggregated processing of each axis-free attention layer, four representative axis features from different angles (shown in the figure are features representing 0, 45, 90 and 135 degrees, respectively) are aggregated and combined with the original features to form an output feature containing global information.

[0036] Due to the correlation between high resolution and high cost of Transformer, the first and second layers of BTFormer in the present invention include traditional convolution blocks to extract image features. Generally speaking, in the Transformer architecture, the role of the convolution block is to reduce the calculation and resolution of the feature map; in the present invention, there is no pooling operation at the end of the second convolution block of BTFormer, so it does not change the resolution of the feature map; all attention blocks of BTFormer in the present invention add an additional 2×2 maximum pooling operation; the resolution of the feature map will be changed after being processed by each axis-free attention block.

[0037] Specifically, the present invention realizes the extraction of non-local correlation in any given direction at low cost through an axis-free attention mechanism network. Figure 3 As shown in Figure 1, the axial free attention mechanism network consists of four main components: a grid generator, a direction sampler, a 1D attention module, and a learnable sinusoidal relative position encoding module. For simplicity, the following mainly describes the specific implementation of the axial free attention mechanism network in a single-head attention manner, but it can be extended to multi-head attention. Figure 2 (b) and Figure 2(c) It can be seen that the axis-free attention block of the present invention uses multi-head attention for each axis, the results of each head are connected as the output, the entire block adopts a residual structure, and the two 1×1 convolutions in the residual branch maintain the mixture of features.

[0038] like Figure 3 As shown, the grid generator in the axial free attention mechanism network of the present invention is used to generate a given input feature map. To calculate the pixel position used to generate the directional feature map Z, is the number of color channels of the input image, H is the height of the input image, and W is the width of the input image. In the present invention, the input feature map X is projected along the width or height axis, and its size (excluding channels) is limited to the width W×1 or height H×1 of the input feature map X; those skilled in the art can expect that this operation can reduce the amount of calculation and improve robustness.

[0039] Generally speaking, the new pixel a used to generate the directional feature map Z is the original grid of the input feature map X The conversion is obtained, where In this rotation case, the projection relationship of the pixels is (3) in, The coordinates of the sample grid in the input feature map X are defined, and θ represents the angle of rotation. According to the above formula, the directional information contained in the directional feature map Z can be controlled by setting the original input grid.

[0040] The directional sampler in the present invention uses a sampling grid to generate a sampled directional feature map Z for the input feature map X, which is achieved by sampling pixels. This is achieved by applying the sampling kernel: (4) in, and Defines interpolation operations in the x and y directions (such as bilinear interpolation and bicubic interpolation). is the value at position (n,m) on the feature map.

[0041] Since the sampling in the above formula is performed independently for each channel, the spatial invariance between channels is maintained. Specifically, if bilinear interpolation is used, for: (5) The 1D attention module in the present invention performs a one-dimensional self-attention mechanism calculation on the input directional feature map Z. For each element in the input sequence, the model calculates a query vector, a key vector, and a value vector; when the model processes an element, it generates a query (Query); then, the model compares this query (Query) with the keys (Key) of all other elements to determine which elements are related to the currently processed element. Finally, the model updates the representation of the currently processed element based on the values ​​(Value) of these related elements. In the multi-head attention mechanism, the input data is first divided into multiple "heads", each of which has its own query (Query), key (Key), and value (Value) vectors. Each head performs self-attention calculations independently to obtain its own attention output, and finally splices these outputs together to form the final output.

[0042] Specifically, in Figure 3 In , the one-dimensional axis-free attention along the angle θ with a given parameter is defined as follows: (6)

[0043] Where θ is the rotation direction, that is, the angle between the attention axis and the width axis. Here, the query ,key ,value is a linear transformation of pixels in the orientation map, where are learnable weights, the same as the settings for self-attention; , is the element value in the directional feature map Z. In addition, , and is a learnable sinusoidal relative position encoding. Considering that the width W of the input is not always the same as the height H, the direction is projected onto the height axis. To obtain multi-range dependencies, the formula is as follows: (7) in, , It is the coverage area of ​​axial free attention.

[0044] In particular, when θ = 0, Equation (6) is the same as the width axis attention, and the axial free attention turns to the width axis attention (8) When θ becomes When the axial free attention is converted into high axial attention, (9) By performing the above axis-free attention, the computational cost can be reduced to At the same time, it can capture long-range dependencies instead of focusing on local interactions.

[0045] This paper proposes a learnable sinusoidal relative position encoding (LSRPE) that aims to introduce continuous bias and adaptability. In the learnable sinusoidal relative position encoding, the position encoding provides information about spatial relationships. Additional position bias terms are usually added to the values, which are usually used to make the model sensitive to position information.

[0046] Given a pixel coordinate a=(i, j) and another pixel coordinate b=(l, n), the learnable sinusoidal relative position encoding (LSRPE) is defined as follows: (10) Among them, the learnable parameters , , is the relative position encoding. Here all position encodings for keys, queries, and values ​​are random and independent.

[0047] In the above model, the weighted soft Jaccard index approximation (WSJA) loss is used. The calculation for category i is as follows: (11) in is a smoothing constant, are the true label and predicted label of category i respectively. Loss function This is done by summing the SJA over the five categories and applying weighted attention on the two categories with the smallest SJA, namely: (12) Preferably, the weight Set them to 1.0 and 0.8 respectively.

[0048] The invention was tested and evaluated on a public blastocyst dataset, which contains 249 blastocyst images and their masks. There are five categories in the dataset: ZP, TE, ICM, blastocyst cavity and background. 199 blastocyst images were randomly selected as our training set, and the remaining 50 images constituted the test set. Five evaluation indicators were used: accuracy, precision, recall, Dice coefficient (Dice) and Jaccard index (Jaccard), which are defined as follows:

[0049]

[0050]

[0051]

[0052]

[0053] These metrics are defined based on four parameters: TP (true positive), FP (false positive), TN (true negative), and FN (false negative).

[0054] In the test, 2000 epochs of training were performed, and specific hyperparameters were used: the initial learning rate was 0.0008, the momentum was set to 0.9, and the weight decay was 0.0005. In order to adaptively adjust the learning rate, the Poly learning rate strategy was used. The Adam optimizer was used to update the network parameters. The specific implementation used PyTorch (version ≥ 1.10), and both training and testing were performed on a 32GB NVIDIA Tesla V100 GPU. Before the images were input into the model for training and testing, their size was normalized to 256×256. In addition, during the training phase, data augmentation was performed on each sample, including random scaling, rotation, cropping, and horizontal flipping.

[0055] The baselines for evaluation and comparison include two types: CNN-based methods such as U-Net, AttUNet, Unet++, Pranet, nnUnet, Deeplabv3+, Unet3+, FCN, PSPNet, DANet, CCNet, and Blast-Net; and Transformer-based methods such as SETR, TransUNet, Segtran, Swin-Unet, UCTransnet, MedT, PCS-Net, and SPC-Net. The baselines are trained using the same binary cross entropy (CE) loss between the predicted and true labels.

[0056] The test results show that the present invention achieves the best performance, with accuracy, precision, recall, Dice and Jaccard of 93.86%, 91.81%, 92.25%, 92.02% and 85.45% respectively.

[0057] The test results also show that the choice of the number of D in formula (1) is crucial for forming the axial free attention block. First, the present invention attempts to explore the impact of different D on the receptive field; when D is equal to 1, the model only obtains a narrow effective receptive field along the width axis; when D is equal to 2, the axial free attention layer will be crossed, in which case BTFormer collects dense attention from the width axis and the height axis; as the number of D increases, the dense attention becomes scattered and fills the entire image, which means that a larger D can obtain a global representation and a wide receptive field. Then, the comparison of Dice and Jaccard with different D is also compared; BTFormer with a larger D takes up more computation, and a larger number of D does not always ensure good performance. Therefore, considering computational efficiency and performance, D cannot be too large. Specifically, BTFormer (D = 4) is the best choice, with the highest Dice and Jaccard and lower computational cost.

[0058] Through the detailed description of the specific implementation methods and embodiments of the present invention above, ordinary technicians in the field can understand that various changes, modifications, substitutions and variations can be made to these implementation methods and embodiments without departing from the principles and basic concepts of the present invention, and the scope of the present invention is defined by the attached claims and their equivalents. Contents not described in detail in this specification, such as the details of the specific structure and implementation of a typical neural network and its functional layer, are within the scope of what can be achieved by professional and technical personnel in the field based on the known prior art and the technical teachings of the present invention.

Claims

1. A biological tissue recognition method based on axial free attention network, characterized in that Includes steps: An axial free attention transformer is constructed. The axial free attention transformer adopts an encoder-decoder structure, including multiple convolution blocks, axial free attention blocks and upsampling blocks, wherein each axial free attention block includes an input network, D axial free attention mechanism networks, a soft aggregation operation network, a 1×1 convolution network and an output network; each axial free attention mechanism network processes the input image feature data according to different angular directions, and includes: a grid generator, a direction sampler, a one-dimensional attention module and a learnable sinusoidal relative position encoding module; wherein the grid generator is used to calculate the pixel position relationship between the input feature map and the generated direction feature map Z, the direction sampler generates the sampled direction feature map Z using a sampling grid, the one-dimensional attention module is used to perform one-dimensional self-attention mechanism calculation processing on the direction feature map Z, and the learnable sinusoidal relative position encoding is used to introduce continuous bias and adaptability; Acquire training sample data, where the training sample data is biological tissue image data, and the biological tissue image data is labeled data; Using the training sample data to train the axial free attention transformer to obtain a biological tissue recognition processing training model; Use the biological tissue recognition and processing training model to recognize and process biological tissue image data.

2. The biological tissue recognition method based on axial free attention network as claimed in claim 1, characterized in that: For the number D of axis-free attention mechanism networks, there is D × θ = π, where θ represents the angle between the attention axes processed by adjacent axis-free attention mechanism networks.

3. The biological tissue recognition method based on axial free attention network as claimed in claim 2, characterized in that: The number D of the axial free attention mechanism networks is 4 or more.

4. The biological tissue recognition method based on axial free attention network as claimed in claim 3, characterized in that: The input network of each axial free attention block divides the input image feature data into D paths and inputs them into D axial free attention mechanism networks respectively. The D axial free attention mechanism networks process the input image feature data according to an angle direction respectively and output the results to the soft aggregation operation network; the soft aggregation operation network soft-aggregates the output results of the D axial free attention mechanism networks and outputs them to the 1×1 convolution network; the output of the 1×1 convolution network is added to the input image feature data and output to the output network to obtain the output image feature data.

5. The biological tissue recognition method based on axial free attention network as claimed in claim 4, characterized in that: The pixel position relationship between the input feature map X and the directional feature map Z calculated by the grid generator is: ； in, is the coordinate of the sample grid in the input feature map, θ is the angle of rotation of the processed attention axis relative to the coordinate axis, is the coordinate of the new pixel used to generate the directional feature map Z.

6. The biological tissue recognition method based on axial free attention network as claimed in claim 5, characterized in that: The kernel function of the sampling grid used by the direction sampler is: ； in, and Defines the interpolation operation in the x and y directions, is the value of the (n, m) position on the feature map, and H and W are the height and width of the image data respectively.

7. The biological tissue recognition method based on axial free attention network as claimed in claim 6, characterized in that: For a given pixel coordinate a = (i, j) and another pixel coordinate b = (l, n), the learnable sinusoidal relative position encoding (LSRPE) is defined as: ； Among them, the learnable parameters , , is a relative position encoding.

8. The biological tissue recognition method based on axial free attention network as claimed in claim 7, characterized in that: The specific structure of the axial free attention converter includes: The first layer, including the first convolutional block; The second layer, includes the second convolution block and the first axis-free attention block; The third layer, includes the second axial free attention block; The fourth layer includes the third axis free attention block; The fifth layer, including the first upsampling block; The sixth layer, including the second upsampling block; The seventh layer, including the third upsampling block; The eighth layer, including the fourth upsampling block; Among them, the first layer is the input layer, and the eighth layer is the output layer. One of the two outputs of the first layer is input to the second layer, and the other is added to the output of the seventh layer and then input to the eighth layer; one of the two outputs of the second layer is input to the third layer, and the other is added to the output of the sixth layer and then input to the seventh layer; one of the two outputs of the third layer is input to the fourth layer, and the other is added to the output of the fifth layer and then input to the sixth layer; the output of the fourth layer is input to the fifth layer.

Images