A virus identification method and system based on instance segmentation

Abstract

The application discloses a virus identification method and system based on instance segmentation, and the method comprises the following steps: acquiring basic characteristics of virus particles based on at least one preset morphologically similar virus image; dividing the virus image into area graphs, generating first feature maps corresponding to each area graph based on the area graphs; acquiring multi-scale virus characteristics of the virus particles based on the first feature maps and the basic characteristics, and generating second feature maps; decomposing the second feature maps into first attention feature maps, second attention feature maps and third attention feature maps; and performing fusion processing on the attention feature maps to generate an identification result. The method can obtain more accurate virus identification results and improve the virus identification accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of microorganism detection, in particular to a virus identification method and system based on instance segmentation. BACKGROUND

[0002] Respiratory viruses are a class of viruses that use the respiratory tract as an entry portal, causing local infection of the respiratory tract or disease of tissues and organs other than the respiratory tract. Influenza virus, new coronavirus and respiratory syncytial virus are common respiratory viruses. Common methods for identifying respiratory viruses include virus isolation and culture, electron microscope detection, immunoserological detection and molecular biological detection, but all have problems such as low efficiency and low accuracy. The classic virus identification method has high accuracy, but it mainly relies on the screening personnel to find virus particles in the sample and identify the virus category through morphological characteristics, which not only requires the screening personnel to have higher virology knowledge, but also consumes time and effort, greatly affecting the detection efficiency of the disease.

[0003] With the development of artificial intelligence technology, a large number of literatures have published methods for classifying and detecting viruses using traditional machine learning methods. However, these methods rely on manual design for feature extraction of viruses, which has great limitations, and in addition, these methods are only suitable for identifying simple and specific viruses, and have low accuracy for identifying morphologically similar respiratory viruses. SUMMARY

[0004] The present application provides a virus identification method and system based on instance segmentation to overcome at least one technical problem in the prior art.

[0005] According to one aspect of the present application, a virus identification method based on instance segmentation is provided, comprising: based on at least one preset morphologically similar virus image, obtaining the basic features of virus particles in the virus image; dividing the virus image into non-overlapping region maps, and performing a preset self-attention calculation in each region map to generate a first feature map corresponding to each region map; based on the first feature map and the basic features, obtaining multi-scale virus features of the virus particles, and generating a second feature map based on the first feature map, the basic features and the multi-scale virus features; decomposing the second feature map into a first attention feature map along the channel direction, a second attention feature map along the horizontal direction and a third attention feature map along the vertical direction; and performing fusion processing on the first attention feature map, the second attention feature map and the third attention feature map to generate an identification result.

[0006] Optionally, the second feature map comprises a first feature sub-map, a second feature sub-map and a third feature sub-map; the step of obtaining the multi-scale virus feature of the virus particle based on the first feature map and the basic feature to generate the second feature map comprises: dividing the first feature map into C1 level, C2 level, C3 level and C4 level according to the size of the first feature map; performing convolution calculation and linear addition calculation based on the first feature map of C2 level, C3 level and C4 level to generate the first feature sub-map of C2' level; performing convolution calculation and linear addition calculation based on the first feature map of C2 level and C3 level to generate the second feature sub-map of C3' level; performing twice convolution calculation based on the first feature map of C4 level to generate the third feature sub-map of C4' level.

[0007] Optionally, the step of decomposing the second feature map into a first attention feature map along a channel direction, a second attention feature map along a horizontal direction and a third attention feature map along a vertical direction comprises: performing information screening processing on the second feature map to generate a third feature map; performing compression on the third feature map in the channel direction of the third feature map based on the third feature map to generate the first attention feature map; performing compression on the third feature map in the horizontal direction of the third feature map based on the third feature map to generate the second attention feature map; performing compression on the third feature map in the vertical direction of the third feature map based on the third feature map to generate the third attention feature map.

[0008] Optionally, the step of performing information screening processing on the second feature map to generate a third feature map comprises: performing identity mapping processing on the second feature map to generate a third feature map.

[0009] Optionally, the step of performing compression on the third feature map in the channel direction of the third feature map based on the third feature map to generate the first attention feature map comprises: assigning the channel number of the third feature map to a channel average value based on the third feature map to generate a first spatial attention feature map; wherein the channel average value is an average value of the channel number of the third feature map corresponding to each of the region maps; assigning the channel number of the third feature map to a channel maximum value based on the third feature map to generate a second spatial attention feature map; wherein the channel maximum value is a maximum value of the channel number of the third feature map corresponding to each of the region maps; performing superposition processing and activation function calculation on the first spatial attention feature map and the second spatial attention feature map to generate the first attention feature map.

[0010] Optionally, the step of compressing the third feature map in the horizontal direction to generate a second attention feature map comprises: assigning a width average value to the width of the third feature map based on the third feature map to generate a preliminary horizontal direction attention feature map; wherein the width average value is the average of the width of the third feature map corresponding to each of the region maps; and performing convolution calculation and activation function calculation on the preliminary horizontal direction attention feature map to generate a second attention feature map.

[0011] Optionally, the step of compressing the third feature map in the vertical direction to generate a third attention feature map comprises: assigning a height average value to the height of the third feature map based on the third feature map to generate a preliminary vertical direction attention feature map; wherein the height average value is the average of the height of the third feature map corresponding to each of the region maps; and performing convolution calculation and activation function calculation on the preliminary vertical direction attention feature map to generate a third attention feature map.

[0012] According to another aspect of the present application, a virus identification system based on instance segmentation is provided, comprising: an acquisition module configured to acquire basic features of virus particles in a virus image based on a preset virus image of at least one morphologically similar virus; a division module configured to divide the virus image into non-overlapping region maps, and perform a preset self-attention calculation in each of the region maps to generate a first feature map corresponding to each of the region maps; an extraction module configured to acquire multi-scale virus features of the virus particles based on the first feature map and the basic features, and generate a second feature map based on the first feature map, the basic features and the multi-scale virus features; a decomposition module configured to decompose the second feature map into a first attention feature map along a channel direction, a second attention feature map along a horizontal direction, and a third attention feature map along a vertical direction; and a generation module configured to perform fusion processing on the first attention feature map, the second attention feature map and the third attention feature map to generate an identification result.

[0013] Optionally, the extraction module specifically comprises a first extraction submodule, a second extraction submodule, a third extraction submodule and a fourth extraction submodule; the first extraction submodule is configured to divide the first feature map into C1 level, C2 level, C3 level and C4 level according to the size of the first feature map; the second extraction submodule is configured to perform convolution calculation and linear addition calculation based on the first feature map of C2 level, C3 level and C4 level, and generate a first feature submap of C2' level; the third extraction submodule is configured to perform convolution calculation and linear addition calculation based on the first feature map of C2 level and C3 level, and generate a second feature submap of C3' level; and the fourth extraction submodule is configured to perform twice convolution calculation based on the first feature map of C4 level, and generate a third feature submap of C4' level.

[0014] Optionally, the decomposition module specifically comprises a first decomposition submodule, a second decomposition submodule, a third decomposition submodule and a fourth decomposition submodule; the first decomposition submodule is configured to perform information screening processing on the second feature map, and generate a third feature map; the second decomposition submodule is configured to perform compression on the third feature map in the channel direction of the third feature map based on the third feature map, and generate a first attention feature map; the third decomposition submodule is configured to perform compression on the third feature map in the horizontal direction of the third feature map based on the third feature map, and generate a second attention feature map; and the fourth decomposition submodule is configured to perform compression on the third feature map in the vertical direction of the third feature map based on the third feature map, and generate a third attention feature map.

[0015] The innovation points of the embodiments of the present application include:

[0016] 1. In the present application, first, the basic features of the virus are extracted based on the image of the virus, and a first feature map is generated, then, based on the first feature map and the basic features, multi-scale virus features are obtained by using a dense feature pyramid network, and a second feature map is generated; next, the second feature map is decomposed into three attention feature maps along the channel direction, the horizontal direction and the vertical direction, and the attention features in the three attention feature maps along the channel direction, the horizontal direction and the vertical direction are fused, so as to obtain more accurate virus recognition results by fusing the attention information of the virus target in the multi-dimensional spatial domain, and the virus recognition accuracy is improved. This is one of the innovation points of the embodiments of the present application.

[0017] 2、The application can obtain multi-scale virus features according to the resolution of the first feature map through the dense feature pyramid, can effectively improve the transmission efficiency of the virus feature map and gradient in the network, further fuse feature map information of different resolutions, thereby extracting high-resolution, rich, multi-scale virus feature information, achieving the purpose of quickly identifying viruses through the feature information, and improving detection efficiency. It is one of the innovations of the embodiments of the application. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0019] Figure 1 The processing flow diagram of the virus identification method based on instance segmentation of the present application;

[0020] Figure 2 The processing flow diagram of the second virus identification method based on instance segmentation of the present application;

[0021] Figure 3 The attention module diagram of the present application;

[0022] Figure 4 The processing flow diagram of the second virus identification system based on instance segmentation of the present application;

[0023] Figure 5 The structure diagram of the extraction module proposed by the present application;

[0024] Figure 6 The structure diagram of the decomposition module proposed by the present application. DETAILED DESCRIPTION

[0025] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0026] It should be noted that the terms "comprising" and "having" and any variations thereof in the embodiments of the present application and the accompanying drawings are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units not listed, or can optionally include other steps or units inherent to the process, method, product or device.

[0027] Embodiment 1

[0028] The embodiments of the present application disclose a first virus identification method based on instance segmentation. The following are described in detail respectively. Reference is made to Figure 1 , Figure 1 is a processing flow diagram of a virus identification method based on instance segmentation of the present application. As shown in Figure 1 , the virus identification method based on instance segmentation includes the following steps:

[0029] Step 101, based on at least one preset morphologically similar virus image, obtaining the basic features of virus particles in the virus image.

[0030] In this step, the preset virus image is an image of at least one highly morphologically similar respiratory infectious virus. In specific implementation, three viruses, H1N1 influenza virus (FLUAV), respiratory syncytial virus (RSV) and new coronavirus, can be selected.

[0031] Specifically, the three viruses can be cultured, extracted and dyed, and then detected under a transmission electron microscope (FEITecnai12TEM), and a camera is equipped to obtain images of the three viruses. It should be noted that the Swin Transformer framework can be introduced to extract the basic features of the virus particles. The basic features include shape features, gray level features and texture features of the virus particles.

[0032] Step 102, dividing the virus image into non-overlapping region maps, and performing a preset self-attention calculation in each region map to generate a first feature map corresponding to each region map.

[0033] In this step, the virus image can be divided into non-overlapping region maps by using a window multi-head self-attention layer (W-MSA), and then self-attention calculation is performed within the region maps. Since there is a lack of information exchange between non-overlapping region maps, a shifted-window multi-head self-attention layer (SW-MSA) can be introduced to solve the information exchange problem between different windows, and finally a first feature map corresponding to each region map is generated.

[0034] It should be further noted that the window multi-head self-attention layer and the shifted-window multi-head self-attention layer are sub-components of the SwinTransformer Block, and the Swin Transformer Block is a sub-module of the Swin Transformer framework.

[0035] In step 103, based on the first feature map and the basic feature, a multi-scale virus feature of the virus particle is obtained, and a second feature map is generated based on the first feature map, the basic feature and the multi-scale virus feature.

[0036] Specifically, since the first feature map contains virus targets of different sizes, and different targets have different basic features representing different phenomena; the basic features in the shallow layer of the first feature map can distinguish simple targets; and the basic features in the deep layer of the first feature map can distinguish complex targets. Therefore, the multi-scale virus feature can be obtained, and the first feature map is subjected to convolution calculation and then linearly added to generate a second feature map.

[0037] It can be understood that the multi-scale virus feature is the basic feature exhibited by the first feature map at different scales.

[0038] In step 104, the second feature map is decomposed into a first attention feature map along the channel direction, a second attention feature map along the horizontal direction, and a third attention feature map along the vertical direction.

[0039] In this step, the second feature map is compressed in the channel direction to obtain two one-dimensional vectors, and a first attention feature map is generated through a multi-layer perception operation.

[0040] Specifically, a channel average value of the channel number of the second feature map is assigned to generate a first spatial attention feature map; wherein the channel average value is an average value of the channel number of the second feature map corresponding to each of the region maps; a channel maximum value of the channel number of the second feature map is assigned to generate a second spatial attention feature map; wherein the channel maximum value is a maximum value of the channel number of the second feature map corresponding to each of the region maps. The first spatial attention feature map and the second spatial attention feature map are superimposed and calculated through an activation function to generate a first attention feature map.

[0041] Similarly, the second feature map is compressed in the horizontal direction and the vertical direction respectively, the width of the second feature map is assigned to a width average value to generate a preliminary horizontal attention feature map; the height of the second feature map is assigned to a height average value to generate a preliminary vertical attention feature map; and the second attention feature map and the third attention feature map are generated through activation function calculation.

[0042] Step 105, the first attention feature map, the second attention feature map and the third attention feature map are fused to generate a recognition result.

[0043] The first attention feature map, the second attention feature map and the third attention feature map are point multiplied to obtain a weight coefficient matrix with scales in width, height and channel number, and then the weight coefficient matrix is point multiplied with the third feature map to generate a fourth feature map.

[0044] In a specific implementation, after generating the fourth feature map, the virus profile in the fourth feature map needs to be segmented to finally generate a virus recognition result.

[0045] It can be seen that the virus recognition method based on instance segmentation provided in the embodiment of the application can first extract basic features of a virus based on an image of the virus and generate a first feature map, then obtain multi-scale virus features based on the first feature map and the basic features using a dense feature pyramid network and generate a second feature map; next, the second feature map is decomposed into three attention feature maps along the channel direction, the horizontal direction and the vertical direction, and the attention features in the three attention feature maps along the channel direction, the horizontal direction and the vertical direction are fused to realize the fusion of attention information of a virus target in a multi-dimensional spatial domain to obtain a more accurate virus recognition result and improve the virus recognition accuracy.

[0046] Embodiment 2

[0047] The embodiment of the application discloses a second virus recognition method based on instance segmentation. The following will be described in detail. Reference Figure 2 ,Figure 2 The processing flow diagram of the second virus identification method based on instance segmentation of the present application is shown. As shown in the figure, the virus identification method based on instance segmentation comprises the following steps: Figure 2

[0048] Step 201, according to the preset H1N1 influenza virus (FLUAV), respiratory syncytial virus (RSV) and new coronavirus, a plurality of images of the three viruses are obtained, and the virus regions in the virus images are drawn.

[0049] Specifically, the preset H1N1 influenza virus (FLUAV), respiratory syncytial virus (RSV) and new coronavirus are cultured in vero, MDCK and Hep2 cells in BSL-3 or BSL-2 laboratory. When the three virus cells produce pathological effects, the culture supernatant is collected. After complete inactivation of the supernatant with 2% formaldehyde, the supernatant is concentrated by ultracentrifugation to obtain the three viruses. As described above, the emitting copper grid and the phosphotungstic acid with a mass concentration of 1% are used for negative staining of the three viruses, and finally the images of the three viruses are obtained.

[0050] It should be noted that the copper grid contains 400 grids, and the surface of the copper grid is covered with carbon reinforced plastic film; the PH value of phosphotungstic acid is 6.8.

[0051] It should also be noted that the virus is detected under a transmission electron microscope (FEI Tecnai12TEM), and a camera is provided to obtain images of the three viruses; the image format is 8-bit grayscale PNG image. It can be understood that the more images of the three viruses, the more virus particles in the images, and the higher the recognition accuracy; in specific implementation, the images of the three viruses can be 318, and the 318 images can contain 1908 virus particles.

[0052] Step 202, based on the virus image, the basic characteristics of the virus particles in the virus are obtained.

[0053] Specifically, the Swin Transformer framework can be introduced to extract the basic characteristics of the virus particles. The basic characteristics include shape characteristics, gray scale characteristics and texture characteristics of the virus particles.

[0054] Further, the shape characteristics include the shape and size of the virus particles; the gray scale characteristics include the brightness and darkness of the region where the virus particles are located; the texture characteristics include the surface properties of the region where the virus particles are located, such as the thickness and density of the texture, etc. The texture characteristics can be represented by the gray scale distribution of the pixels and their surrounding spatial neighborhood.

[0055] ​Step 203, dividing the virus image into non-overlapping region maps, performing preset self-attention calculation in each region map respectively to generate a first feature map corresponding to each region map.

[0056] Specifically, the virus image is divided into non-overlapping region maps by a window multi-head self-attention layer (W-MSA), and then self-attention calculation is performed in the region maps.

[0057] It should be noted that since each region map contains a fixed size patch, and the number of small maps is much smaller than the number of patches, the computational complexity of W-MSA is linearly related to the image size. It can be understood that within a certain range, the smaller the image size, the lower the calculation amount. Since there is a lack of information exchange between non-overlapping small maps, a shifted-window multi-head self-attention layer (SW-MSA) can be introduced to solve the information exchange problem between different windows, and finally generate a first feature map corresponding to each region map.

[0058] Step 204, based on the first feature map and the basic feature, obtaining a multi-scale virus feature of the virus particle, and generating a second feature map based on the first feature map, the basic feature and the multi-scale virus feature.

[0059] Among them, the second feature map includes a first feature sub-map, a second feature sub-map and a third feature sub-map.

[0060] Specifically, since there is a difference in resolution between the first feature maps, there are virus targets of different sizes in the first feature maps, and different basic features of different targets represent different phenomena; some simple targets can be distinguished by using the basic features of the shallow layer; and the basic features of the deep layer can be used to distinguish other complex targets. Therefore, multi-scale virus features can be obtained by using a dense feature pyramid structure, and the first feature maps are convolved and then linearly added to generate a second feature map.

[0061] Further, the first feature maps can be divided into different levels according to the size of the first feature maps, and the first feature maps of different levels are convolved and then linearly added to generate second feature maps of different levels.

[0062] In a specific implementation, the first feature maps can be divided into C1 level, C2 level, C3 level and C4 level, and it can be understood that the higher the level, the smaller the size.

[0063] Based on the first feature maps at levels C2, C3, and C4, convolution and linear addition are performed to generate a first feature sub-map at level C2'.

[0064] Based on the first feature map at levels C2 and C3, convolution and linear addition are performed to generate a second feature sub-map at level C3'.

[0065] Based on the first feature map at C4 level, two convolution calculations are performed to generate a third feature sub-map at C4' level.

[0066] Step 205: Perform information filtering processing on the second feature map to generate a third feature map.

[0067] Specifically, neural network depth is a core factor affecting network performance. However, excessive depth can also lead to the following problems: Firstly, as the network deepens, a large number of parameters are added, resulting in a significant decrease in computational performance; secondly, during network optimization, phenomena such as vanishing or exploding gradients can occur. Although normalization can largely solve this problem, it can also lead to network degradation, where accuracy tends to saturate and then rapidly declines as depth increases.

[0068] The second feature map can be processed by identity mapping to filter out regions that do not belong to virus particles and noise information, thereby generating a third feature map.

[0069] Furthermore, a Coor-Spatial attention module is introduced based on the aforementioned dense feature pyramid network; such as... Figure 3 As shown, Figure 3 This is a schematic diagram of the attention module of the present invention; the second feature map is input into the residual module in the attention module, and by superimposing a layer of y=x on a shallow network, the second feature map is subjected to identity mapping processing, so that the network does not degenerate as the depth increases.

[0070] Step 206: Based on the third feature map, compress the channel direction of the third feature map to generate a first attention feature map.

[0071] In this step, the number of channels in the third feature map can be assigned an average channel value to generate a first spatial attention feature map.

[0072] The average channel value is the average number of channels in the third feature map corresponding to each of the region maps.

[0073] The channel number of the third feature map is assigned a channel maximum value based on the third feature map to generate a second spatial attention feature map.

[0074] The channel maximum value is the maximum value of the channel number of the third feature map corresponding to each region map.

[0075] The first spatial attention feature map and the second spatial attention feature map are superimposed and processed by an activation function to generate a first attention feature map.

[0076] Specifically, the third feature map is input into a spatial attention module in the attention module, and global average pooling and global maximum pooling in the spatial attention module are used to compress the third feature map along the channel direction to obtain a first spatial attention feature map and a second spatial attention feature map with a channel scale of 1. Then, the first spatial attention feature map and the second spatial attention feature map are superimposed and processed by a sigmoid activation function to generate a first attention feature map.

[0077] In step 207, based on the third feature map, the third feature map is compressed in the horizontal direction to generate a second attention feature map.

[0078] In this step, based on the third feature map, the width of the third feature map is assigned a width average value to generate a preliminary horizontal direction attention feature map.

[0079] The width average value is the average value of the width of the third feature map corresponding to each region map.

[0080] The preliminary horizontal direction attention feature map is subjected to convolution calculation and activation function calculation to generate a second attention feature map.

[0081] Specifically, the third feature map is input into a coordinate attention module in the attention module, and a global average pooling layer in the coordinate attention module is used to compress and encode the third feature map along the horizontal direction X to obtain a preliminary horizontal direction attention feature map with a width scale of 1. Then, the preliminary horizontal direction attention feature map is subjected to convolution calculation and sigmoid activation function calculation to generate a second attention feature map.

[0082] In step 208, based on the third feature map, the third feature map is compressed in the vertical direction to generate a third attention feature map.

[0083] In this step, based on the third feature map, the width of the third feature map is assigned a width average value to generate a preliminary vertical direction attention feature map.

[0084] The width average value is the average value of the height of the third feature map corresponding to each region map.

[0085] The preliminary vertical direction attention feature map is subjected to convolution calculation and activation function calculation to generate a third attention feature map.

[0086] Specifically, the third feature map is input into a coordinate attention module in the attention module, and a global average pooling layer in the coordinate attention module is used to compress and encode the third feature map along the vertical direction Y to obtain a preliminary horizontal direction attention feature map with a width scale of 1 and a preliminary vertical direction attention feature map with a height scale of 1. Then, the preliminary horizontal direction attention feature map and the preliminary vertical direction attention feature map are subjected to convolution calculation and sigmoid activation function calculation to generate a third attention feature map.

[0087] In step 209, the first attention feature map, the second attention feature map and the third attention feature map are subjected to fusion processing to generate a recognition result.

[0088] Specifically, the first attention feature map, the second attention feature map and the third attention feature map are subjected to point multiplication processing to obtain a weight coefficient matrix with scales in width, height and channel number, and then the weight coefficient matrix is subjected to point multiplication processing with the third feature map and is taken as a fourth feature map as the output of the attention module.

[0089] The fourth feature map is input into a preset Protonet module to segment the virus outline in the fourth feature map to generate a first segmentation result. Next, the first segmentation result is input into a preset PredictionHead module for scoring. Then, the first segmentation result is linearly weighted with the score to obtain a second segmentation result, and then the bounding box of the segmented outline is intercepted to generate a detection result. It should be noted that the second segmentation result and the detection result are the recognition result.

[0090] It should be noted that the size of the size value of the first attention feature map, the second attention feature map and the third attention feature map in this step is between [0, 1].

[0091] It can be seen that the virus recognition method based on instance segmentation provided in the embodiment of the present application can extract the basic features of the virus based on the image of the virus, and generate a first feature map, then based on the first feature map and the basic features, a dense feature pyramid network is used to obtain multi-scale virus features and generate a second feature map; next, the second feature map is decomposed into three attention feature maps along the channel direction, the horizontal direction and the vertical direction, and the attention features in the channel direction, the horizontal direction and the vertical direction of the above three attention feature maps are fused, so as to realize the fusion of the attention information of the virus target in the multi-dimensional spatial domain, so as to obtain more accurate virus recognition results and improve the virus recognition accuracy. The dense feature pyramid is used to obtain multi-scale virus features according to the resolution of the first feature map, which can effectively improve the transmission efficiency of the virus feature map and the gradient in the network, further fuse the feature map information of different resolutions, so as to extract high-resolution, rich and multi-scale virus feature information, and achieve the purpose of quickly identifying viruses through the feature information, and improve the detection efficiency.

[0092] Embodiment 3

[0093] The embodiment of the present application discloses the experimental test results of the virus recognition method based on instance segmentation. The following will be described in detail.

[0094] The embodiment of the present application will perform quantitative analysis on the segmentation and detection of the present application to evaluate the effect of the present application.

[0095] For the quantitative analysis of the segmentation, Jaccard index and Dice coefficient can be used for evaluation; for the quantitative analysis of the detection, the number of true positives (TP), false positives (FP) and false negatives (FN) of virus detection can be calculated, and the precision, recall and F1 score of virus detection can be calculated. The specific description of the above evaluation indexes is as follows:

[0096] Jaccard coefficient is used to measure the similarity between network prediction segmentation graph and real label graph, as shown in formula (1):

[0097]

[0098] Wherein, intersection(x, y) represents the intersection of the real label graph and the network prediction segmentation graph, and union(x, y) represents the union of the real label graph and the network prediction segmentation graph.

[0099] Dice coefficient is a set similarity measure index, which is usually used to calculate the similarity coefficient of two samples, as shown in formula (2):

[0100]

[0101] Wherein, |x| represents a real label graph, |y| represents a network prediction segmentation graph, and intersection(x, y) represents an intersection of two sets.

[0102] The precision rate (precision) represents a ratio of the number of real viruses detected to the total number of viruses detected, and is used to measure the accuracy of the method, as shown in formula (3), wherein TP (True positive) represents the number of viruses correctly predicted, and FP (False positive) represents the number of viruses predicted incorrectly.

[0103]

[0104] The recall rate (recall) represents a ratio of the number of real viruses detected to the total number of real viruses, and is used to measure the completeness of the method, as shown in formula (4), wherein FN (False Negative) represents the number of real viruses predicted as non-viruses.

[0105]

[0106] The F1 score can also be used to comprehensively evaluate the detection result. The F1 score is calculated by calculating the harmonic mean of the precision rate and the recall rate, and is used to comprehensively reflect the overall performance of the method. As shown in formula (5):

[0107]

[0108] The virus instance segmentation method provided by the present application is compared with the commonly used instance segmentation methods of Solov2, YOLACT, YOLACT++, and Swin-YOLACT, as shown in Table 1, and Table 1 represents the experimental results of different methods in the TEM virus database. In the segmentation result, the Jaccard evaluation score of the present application is 0.8746, and the Dice index is 0.9328, which is greatly improved compared with other methods; in the detection evaluation, the accuracy, recall rate and F1 value of the present application are 0.9404, 0.9332 and 0.9368 respectively, which are all better than those of other methods.

[0109] Table 1

[0110]

[0111] Embodiment 4

[0112] The present application discloses a virus identification system based on instance segmentation. The following are described in detail. Reference Figure 4 , Figure 4 is a processing flow diagram of a virus identification system based on instance segmentation of the present application. As Figure 4As shown, a virus identification system based on instance segmentation includes:

[0113] An acquisition module 40 is configured to acquire basic features of virus particles in a virus image based on preset virus images similar in shape;

[0114] A division module 50 is configured to divide the virus image into non-overlapping region maps, and perform preset self-attention calculation in each region map to generate a first feature map corresponding to each region map;

[0115] An extraction module 60 is configured to acquire multi-scale virus features of the virus particles based on the first feature map and the basic features, and generate a second feature map based on the first feature map, the basic features and the multi-scale virus features;

[0116] A decomposition module 70 is configured to decompose the second feature map into a first attention feature map along a channel direction, a second attention feature map along a horizontal direction and a third attention feature map along a vertical direction;

[0117] A generation module 80 is configured to perform fusion processing on the first attention feature map, the second attention feature map and the third attention feature map to generate an identification result.

[0118] Optionally, as shown, the extraction module 60 specifically includes a first extraction submodule 61, a second extraction submodule 62, a third extraction submodule 63 and a fourth extraction submodule 64; Figure 5

[0119] The first extraction submodule 61 is configured to divide the first feature map into C1 level, C2 level, C3 level and C4 level according to the size of the first feature map;

[0120] The second extraction submodule 62 is configured to perform convolution calculation and linear addition calculation based on the first feature map of C2 level, C3 level and C4 level to generate a first feature submap of C2' level;

[0121] The third extraction submodule 63 is configured to perform convolution calculation and linear addition calculation based on the first feature map of C2 level and C3 level to generate a second feature submap of C3' level;

[0122] The fourth extraction submodule 64 is configured to perform twice convolution calculation based on the first feature map of C4 level to generate a third feature submap of C4' level.

[0123] Optionally, as shown, Figure 6 ​As shown, the decomposition module 70 specifically comprises a first decomposition submodule 71, a second decomposition submodule 72, a third decomposition submodule 73 and a fourth decomposition submodule 74.

[0124] The first decomposition submodule 71 is configured to perform information screening processing on the second feature map to generate a third feature map.

[0125] The second decomposition submodule 72 is configured to compress the third feature map in a channel direction of the third feature map based on the third feature map to generate a first attention feature map.

[0126] The third decomposition submodule 73 is configured to compress the third feature map in a horizontal direction of the third feature map based on the third feature map to generate a second attention feature map.

[0127] The fourth decomposition submodule 74 is configured to compress the third feature map in a vertical direction of the third feature map based on the third feature map to generate a third attention feature map.

[0128] It can be seen that the virus recognition system based on instance segmentation provided in the embodiment can extract basic features of a virus based on an image of the virus, generate a first feature map, then generate a second feature map based on the first feature map and the basic features, and then decompose the second feature map into three attention feature maps along a channel direction, a horizontal direction and a vertical direction, and fuse the attention features in the three attention feature maps along the channel direction, the horizontal direction and the vertical direction, so as to obtain more accurate virus recognition results by fusing the attention information of the virus target in the multi-dimensional spatial domain, and improve the virus recognition accuracy. The multi-scale virus features are obtained according to the resolution of the first feature map, which can effectively improve the transmission efficiency of the virus feature map and the gradient in the network, further fuse the feature map information of different resolutions, so as to extract high-resolution, rich and multi-scale virus feature information, achieve the purpose of quickly recognizing the virus through the feature information, and improve the detection efficiency.

[0129] Those skilled in the art can understand that the modules or flows in the drawings are not necessarily necessary for implementing the present application.

[0130] Those skilled in the art can understand that the modules in the device in the embodiment can be distributed in the device in the embodiment according to the description of the embodiment, or can be located in one or more devices different from the embodiment. The modules in the above embodiment can be combined into one module, or can be further split into multiple sub-modules.

[0131] Finally, it should be noted that the above examples are only used to illustrate the technical solutions of the present application, and are not intended to limit the same; although the present application has been described in detail with reference to the foregoing examples, those of ordinary skill in the art should understand that the technical solutions recorded in the foregoing examples can still be modified, or some of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A virus identification method based on instance segmentation, characterized in that, The method comprises the following steps: Based on the preset at least one morphologically similar virus image, the basic characteristics of the virus particles in the virus image are obtained, including the shape characteristics, gray scale characteristics and texture characteristics of the virus particles; The virus image is divided into non-overlapping region maps, and preset self-attention calculation is performed in each region map to generate a first feature map corresponding to each region map; Based on the first feature map and the basic characteristics, a dense feature pyramid network is used to obtain multi-scale virus characteristics of the virus particles, and the multi-scale virus characteristics of the first feature map are subjected to convolution operation and then linearly added to obtain a second feature map; The second feature map is decomposed into a first attention feature map along the channel direction, a second attention feature map along the horizontal direction and a third attention feature map along the vertical direction; The first attention feature map, the second attention feature map and the third attention feature map are fused to generate a recognition result.

2. The method of claim 1, wherein, The second feature map includes a first feature sub-map, a second feature sub-map and a third feature sub-map; The step of generating a second feature map based on the first feature map and the basic characteristics includes: According to the size of the first feature map, the first feature map is divided into C1 level, C2 level, C3 level and C4 level; Based on the first feature map of C2 level, C3 level and C4 level, convolution calculation and linear addition calculation are performed to generate a first feature sub-map of C2' level; Based on the first feature map of C2 level and C3 level, convolution calculation and linear addition calculation are performed to generate a second feature sub-map of C3' level; Based on the first feature map of C4 level, two convolution calculations are performed to generate a third feature sub-map of C4' level.

3. The method of claim 1, wherein, The step of decomposing the second feature map into a first attention feature map along the channel direction, a second attention feature map along the horizontal direction and a third attention feature map along the vertical direction includes: Information screening processing is performed on the second feature map to generate a third feature map; Based on the third feature map, compression is performed in the channel direction of the third feature map to generate a first attention feature map; Based on the third feature map, compression is performed in the horizontal direction of the third feature map to generate a second attention feature map; Based on the third feature map, compression is performed in the vertical direction of the third feature map to generate a third attention feature map.

4. The method of claim 3, wherein, The step of performing information screening processing on the second feature map to generate a third feature map includes: Identity mapping processing is performed on the second feature map to generate a third feature map.

5. The method of claim 3, wherein, The step of performing compression on the third feature map in the channel direction of the third feature map to generate a first attention feature map includes: Based on the third feature map, the channel number of the third feature map is assigned to the channel average value to generate a first spatial attention feature map; The channel average value is the average value of the channel number of the third feature map corresponding to each region map. Based on the third feature map, the number of channels in the third feature map is assigned the maximum value of the channels to generate a second spatial attention feature map; Wherein, the maximum channel value is the maximum number of channels in the third feature map corresponding to each of the region maps; The first spatial attention feature map and the second spatial attention feature map are superimposed and activation functions are calculated to generate the first attention feature map.

6. The method of claim 3, wherein, The step of compressing the third feature map in the horizontal direction to generate a second attention feature map based on the third feature map includes: Based on the third feature map, the width of the third feature map is assigned an average width to generate a preliminary horizontal attention feature map; Wherein, the average width is the average width of the third feature map corresponding to each of the region maps; The initial horizontal attention feature map is subjected to convolution and activation function calculations to generate a second attention feature map.

7. The method of claim 3, wherein, The step of compressing the third feature map in the vertical direction to generate a third attention feature map based on the third feature map includes: Based on the third feature map, the height of the third feature map is assigned an average height value to generate a preliminary vertical attention feature map; Wherein, the average height is the average height of the third feature map corresponding to each of the region maps; The initial vertical attention feature map is subjected to convolution and activation function calculations to generate a third attention feature map.

8. A virus identification system based on instance segmentation, characterized by, include: The acquisition module is used to acquire the basic features of virus particles in the virus image based on at least one preset virus image with similar morphology. The segmentation module is used to divide the virus image into non-overlapping region maps, perform preset self-attention calculations in each region map, and generate a first feature map corresponding to each region map; An extraction module is used to obtain multi-scale viral features of the virus particle based on the first feature map and the basic features, and to generate a second feature map based on the first feature map, the basic features and the multi-scale viral features. The decomposition module is used to decompose the second feature map into a first attention feature map along the channel direction, a second attention feature map along the horizontal direction, and a third attention feature map along the vertical direction. The generation module is used to fuse the first attention feature map, the second attention feature map, and the third attention feature map to generate a recognition result.

9. The system according to claim 8, characterized in that, The extraction module specifically includes a first extraction submodule, a second extraction submodule, a third extraction submodule, and a fourth extraction submodule; The first extraction submodule is used to divide the first feature map into C1 level, C2 level, C3 level and C4 level according to the size of the first feature map; The second extraction submodule is used to perform convolution calculation and linear addition calculation based on the first feature map at levels C2, C3 and C4 to generate a first feature submap at level C2'. The third extraction submodule is configured to perform convolution calculation and linear addition calculation based on the first feature map at the C2 level and the C3 level, to generate a second feature submap at a C3' level. The fourth extraction submodule is configured to perform twice convolution calculation based on the first feature map at the C4 level, to generate a third feature submap at a C4' level. 10.The system of claim 8, wherein, The decomposition module specifically comprises a first decomposition submodule, a second decomposition submodule, a third decomposition submodule, and a fourth decomposition submodule. The first decomposition submodule is configured to perform information screening processing on the second feature map, to generate a third feature map. The second decomposition submodule is configured to perform compression on the third feature map in a channel direction of the third feature map based on the third feature map, to generate a first attention feature map. The third decomposition submodule is configured to perform compression on the third feature map in a horizontal direction of the third feature map based on the third feature map, to generate a second attention feature map. The fourth decomposition submodule is configured to perform compression on the third feature map in a vertical direction of the third feature map based on the third feature map, to generate a third attention feature map.

Images