Method, system and equipment for identifying strains of biological organic fertilizer based on neural network

Through the multimodal fusion method based on neural network, the problem of low recognition speed and accuracy in biological organic fertilizer species recognition is solved, and fast and accurate species recognition is achieved, using microscopic images, spectroscopy and biochemical indicators, etc., etc. Data improves the accuracy and scope of identification.

CN118568677BActive Publication Date: 2024-11-22BAOJI DADI AGRI TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411063306.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-05
Publication Date
2024-11-22
Estimated Expiration
2044-08-05

AI Technical Summary

Technical Problem

The existing biological organic fertilizer strain recognition technology has the problem of low recognition speed and accuracy, strong subjectivity of morphological observation, long time-consuming physiological and biochemical tests, high cost of molecular biological methods and requires professional equipment and technology.

Method used

Using a neural network-based biological organic fertilizer strain recognition method, data preprocessing and feature extraction are performed by collecting multimodal information (micro images, near-infrared spectroscopy, biochemical indexes), and cross-modal attention mechanism and dynamic weight fusion are used. The module performs feature fusion to realize the classification of bacterial species.

Benefits of technology

The speed and accuracy of bacterial species recognition are improved, and the complementary information of multimodal data is fully utilized, complex correlations between modes are captured, the importance of adaptively adjusting different modes, and the model performance and generalization capabilities are improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118568677B_ABST
    Figure CN118568677B_ABST
Patent Text Reader

Abstract

The present application discloses a biological organic fertilizer strain identification method, system and equipment based on a neural network, the method comprising: collecting multimodal information of biological organic fertilizer, and performing data preprocessing on the multimodal information to obtain a multimodal data set; extracting features from each modality in the multimodal data set respectively; mapping the features of each modality to a target semantic space, and updating the features of each modality according to the mutual relationship; calculating the global features of each modality respectively, and performing a splicing operation on all global features; performing feature fusion based on the updated features of each modality and the spliced ​​global features to obtain fused features; and classifying the fused features to obtain the strain category of the biological organic fertilizer. Through the scheme of the present application, the strain identification speed and recognition accuracy can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image recognition, and in particular to a method, system and device for identifying strains of biological organic fertilizer based on a neural network. Background Art

[0002] Bio-organic fertilizers play an increasingly important role in modern agricultural production, among which the identification and quality control of bacterial strains are the key links to ensure the effectiveness of bio-organic fertilizers. The background of the bio-organic fertilizer strain identification method stems from the demand for sustainable agricultural development and the pressure of environmental protection, aiming to improve fertilizer utilization efficiency, reduce the use of chemical fertilizers, and improve soil health. The usefulness of this identification method is mainly reflected in quality control, product development, and application guidance. By accurately identifying bacterial strains, it can be ensured that the bio-organic fertilizer contains the expected beneficial microorganisms and avoid contamination by harmful strains, thereby ensuring product quality and use effect. At the same time, strain identification is of great significance for the research and development of new bio-organic fertilizers, optimizing formulas, and guiding farmers to apply them correctly.

[0003] At present, the existing technologies for identifying strains of biological organic fertilizer mainly include traditional morphological observation, physiological and biochemical tests, molecular biology methods (such as PCR, DNA sequencing), etc. The working principles of these methods are different: morphological observation relies on professionals to observe the morphological characteristics of bacteria through a microscope; physiological and biochemical tests are used to identify strains by testing the growth response of strains to different culture media; molecular biology methods determine the identity of strains by analyzing the DNA or RNA sequence of strains. However, there are some problems with these existing technologies: morphological observation is highly subjective and easily affected by the experience of the observer; physiological and biochemical tests are time-consuming, and some strains may show similar physiological characteristics; although molecular biology methods are highly accurate, they are expensive and require professional laboratory equipment and technicians. In order to solve the above problems, researchers began to explore new strain identification methods. Among them, methods based on image analysis have attracted attention due to their rapid and non-invasive characteristics. These methods usually use computer vision technology to identify strains by analyzing features such as colony morphology and color. Another method is to use spectral analysis technology, such as near-infrared spectroscopy or Raman spectroscopy, to identify strains by analyzing the spectral characteristics of strains. These methods have the advantages of being easy to operate and fast compared to traditional technologies, but they are still limited in accuracy and scope of application. For example, image analysis methods may have difficulty distinguishing morphologically similar bacterial species, while spectral analysis methods may be affected by sample preparation and environmental factors.

[0004] Therefore, there is an urgent need for a technical solution that can improve the speed and accuracy of strain identification. Summary of the invention

[0005] In order to solve the deficiencies of the prior art, the embodiments of the present application provide a method, system and device for identifying bacterial strains of biological organic fertilizer based on neural network. The present application solves the technical problems of low recognition speed and low recognition accuracy in the prior art.

[0006] The embodiment of the present application provides a biological organic fertilizer strain identification method based on a neural network, comprising: collecting multimodal information of the biological organic fertilizer, and performing data preprocessing on the multimodal information to obtain a multimodal data set; respectively extracting features of each modality in the multimodal data set; mapping the features of each modality to a target semantic space, and updating the features of each modality according to the mutual relationship; respectively calculating the global features of each modality, and performing a splicing operation on all global features; performing feature fusion based on the updated features of each modality and the spliced ​​global features to obtain fused features; and classifying the fused features to obtain the strain category of the biological organic fertilizer.

[0007] In a possible implementation, multimodal information of bio-organic fertilizer is collected, and data preprocessing is performed on the multimodal information to obtain a multimodal data set, including: collecting a microscopic image of the bio-organic fertilizer, and removing salt and pepper noise through adaptive median filtering; applying a contrast-limited adaptive histogram equalization method to enhance the details of the microscopic image after noise reduction; normalizing the pixel values ​​of the microscopic image to a target range to obtain first modal data; collecting near-infrared spectral data of the bio-organic fertilizer, and using multivariate scattering correction to remove baseline drift; applying Savitzky-Golay filtering to reduce noise, and using a standard normal variable method to standardize to obtain second modal data; collecting multiple biochemical indicators, and using an improved Z-score method to process outliers; performing standardized mapping on the processed biochemical indicators to obtain third modal data; splicing the first modal data, the second modal data, and the third modal data to obtain a multimodal data set.

[0008] In a possible implementation, feature extraction is performed on each modality in a multimodal data set, including: using an improved EfficientNet-B0 as a backbone network, and introducing a spatial attention mechanism to perform feature extraction on the first modality data to obtain image features; performing feature extraction on the second modality data based on one-dimensional convolution and self-attention mechanism to obtain spectral features; using a multilayer perceptron to extract features of the third modality data to obtain biochemical indicator features.

[0009] In a possible implementation, the features of each modality are mapped to a target semantic space, and the features of each modality are updated based on their mutual relationships, including: mapping the features of each modality to the same semantic space, and calculating a cross-modal attention matrix; constructing a cross-modal mutual relationship matrix based on the cross-modal attention matrix to update the features of each modality.

[0010] In a possible implementation, a cross-modal relationship matrix is ​​constructed based on the cross-modal attention matrix to update the features of each modality, including:

[0011] ,in, Indicates the updated The characteristics of a mode, Represents the original The characteristics of a mode, represents the learnable feature update strength parameter, represents the total number of modes, represents the learnable balance parameter, Indicates the mode to be calculated and modal The learnable weight matrix of the similarity between Indicates The norm of the modal features, Indicates The query matrix of each modality, Indicates The key-value matrix of each modality, represents the dimension of the key value, represents the learnable weight matrix for feature transformation, Indicates the target modal index of the feature currently being updated. represents the modality index for interaction and information fusion with the target modality, Indicates the index variable for traversing all modes. Indicates traversal and accumulation of all modes. Indicates The transpose of the modal eigenvectors, Indicates that the traversal index is The eigenvectors of the modes, Indicates The norm of the modal features, Indicates The eigenvector of the mode.

[0012] In one possible implementation, the global features of each modality are calculated separately, including:

[0013] ,in, Representing modality The global characteristics of Representing modality The number of features, Representing modality No. Features, represents the learnable modality-specific parameters, represents the number of attention heads, Indicates The weight of the attention head, represents the activation function, represents element-wise multiplication, represents the sigmoid activation function, Represents the layer normalization operation.

[0014] In a possible implementation, feature fusion is performed based on the updated features of each modality and the spliced ​​global features to obtain fused features, including:

[0015] ,in, represents the fusion feature, represents the balance factor, represents a multi-layer perceptron, represents the global features after splicing, represents the dynamic weight, represents the attention mechanism, Represents the updated features.

[0016] In a possible implementation, before extracting features from each modality in a multimodal data set, the method further includes: separately training a neural network for extracting features from each modality data and freezing other parts of the neural network; freezing the trained neural network for feature extraction and training other parts of the neural network; and after the training is completed, fine-tuning the parameters of all neural networks using a small learning rate.

[0017] The embodiment of the present application also provides a biological organic fertilizer strain identification system based on a neural network, comprising: a collection unit, a preprocessing unit, a feature unit and an identification unit; wherein the collection unit is used to collect multimodal information of the biological organic fertilizer; the preprocessing unit is used to perform data preprocessing on the multimodal information to obtain a multimodal data set; the feature unit is used to extract features from each modality in the multimodal data set respectively; the features of each modality are mapped to a target semantic space, and the features of each modality are updated according to their mutual relationships; the global features of each modality are calculated respectively, and all global features are spliced; feature fusion is performed based on the updated features of each modality and the spliced ​​global features to obtain fused features; the identification unit is used to classify the fused features to obtain the strain category of the biological organic fertilizer.

[0018] An embodiment of the present application also provides a biological organic fertilizer strain identification device based on a neural network, characterized in that it includes: a processor, a memory, and a system bus; wherein the processor and the memory are connected via the system bus; the memory is used to store one or more programs, and the one or more programs include instructions, and when the instructions are executed by the processor, the processor executes the method described in the above embodiment.

[0019] In the biological organic fertilizer strain identification method, system and equipment based on neural network provided above, the embodiment of the present application performs automatic identification through neural network, which can effectively improve the recognition speed and accuracy; through the multimodal data fusion strategy, it can make full use of the complementary information of different types of data; through the cross-modal attention mechanism, it can capture the complex correlation between modalities; through the dynamic weight fusion module, it can adaptively adjust the importance of different modalities; through the improved loss function and training strategy, it can improve the model performance and generalization ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 A schematic diagram of a process for identifying strains of biological organic fertilizer based on a neural network provided in an embodiment of the present application; Figure 2 A schematic block diagram of a biological organic fertilizer strain identification system based on a neural network provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. It should be noted that unless otherwise specifically stated, the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present application.

[0023] Those skilled in the art will appreciate that the terms "first", "second" and the like in the embodiments of the present application are only used to distinguish different steps, devices or modules, etc., and do not represent any specific technical meaning, nor do they represent the necessary logical order between them. It should also be understood that in the embodiments of the present application, "multiple" may refer to two or more, and "at least one" may refer to one, two or more. It should also be understood that for any component, data or structure mentioned in the embodiments of the present application, in the absence of explicit limitation or contrary revelation given in the context, it can generally be understood as one or more. In addition, the term "and / or" in the present application is only a kind of association relationship describing the associated objects, indicating that there can be three kinds of relationships, for example, A and / or B, which can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present application generally indicates that the associated objects before and after are an "or" relationship. It should also be understood that the description of each embodiment in the present application emphasizes the differences between the embodiments, and the same or similar parts can refer to each other. For the sake of brevity, they will not be repeated one by one.

[0024] At the same time, it should be understood that, for ease of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship. The following description of at least one exemplary embodiment is actually only illustrative and is by no means intended to limit the present application and its application or use. The techniques, methods and devices known to ordinary technicians in the relevant fields may not be discussed in detail, but where appropriate, the techniques, methods and devices should be considered as part of the specification. It should be noted that similar numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further discussed in subsequent drawings.

[0025] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0026] Figure 1 A schematic flow chart of a method for identifying strains of biological organic fertilizer based on a neural network is provided in an embodiment of the present application.

[0027] like Figure 1As shown, at step S101, multimodal information of biological organic fertilizer is collected, and data preprocessing of the multimodal information is performed to obtain a multimodal data set. Specifically, microscopic images of biological organic fertilizers are collected, and samples are randomly collected from this batch of biological organic fertilizers, each sample weighing 10g. A stratified sampling method is used to ensure that the samples are representative. The collected samples are diluted with sterile water at a dilution ratio of 1:10000. The dilution process adopts a continuous dilution method, and the specific steps are as follows: Take 1g of sample and add it to 9ml of sterile water, mix it thoroughly, and obtain a 10^-1 dilution. Draw 1ml from the 10^-1 dilution and add it to 9ml of sterile water to obtain a 10^-2 dilution. Repeat the above steps until a 10^-4 dilution is obtained.

[0028] Then select a medium suitable for the growth of the target bacteria, such as LB agar medium. Spread the diluted sample evenly on the culture dish and culture it in a 28°C constant temperature incubator for 48-72 hours. Use a high-resolution digital microscope to photograph the colonies in the culture dish. The shooting parameters are set as follows:

[0029] Resolution: 4000 × 3000 pixels;

[0030] Magnification: 40×;

[0031] Lighting: LED ring light source, color temperature 5500K,

[0032] Five images were taken for each dish to ensure that the entire dish area was covered, and then salt and pepper noise was removed by adaptive median filtering. The window size was adaptively adjusted according to the noise level, ranging from 3×3 to 7×7. Contrast-limited adaptive histogram equalization was applied to enhance the details of the denoised microscopic image, with the block size set to 8×8 and the contrast limit threshold set to 0.01. The pixel values ​​of the microscopic image were normalized to the target range to obtain the first modality data. The pixel values ​​can be normalized to the range [0, 1].

[0033] The near-infrared spectrum data of bio-organic fertilizer was collected, with a wavelength range of 900-2500nm and an interval of 2nm. Each sample was scanned 3 times and the average value was taken to improve data reliability. Multivariate scattering correction was used to remove baseline drift, and the formula was:

[0034] ,in, Represents the near-infrared spectral data of bio-organic fertilizer, a and b are calculated by the least squares method.

[0035] Savitzky-Golay filtering was applied to reduce noise, with the window size set to 15 and the polynomial order set to 3. The standard normal variable method was used for standardization to obtain the second modal data, with the formula: ,in, is the mean, is the standard deviation.

[0036] Collect multiple biochemical indicators, including but not limited to pH value, water content, organic matter content, total nitrogen, total phosphorus, total potassium and other key biochemical indicators. Use standardized experimental methods, repeat the measurement of each indicator 3 times, and take the average value. And use the improved Z-score method to handle outliers, with the threshold set to 3.5. Standardize and map the processed biochemical indicators to obtain the third modal data. Min-Max normalization can be used to map the values ​​to the [0, 1] interval.

[0037] The first modal data, the second modal data and the third modal data are spliced ​​to obtain a multimodal data set. Next, at step S102, feature extraction is performed on each modality in the multimodal data set. The embodiment of the present application designs a new type of multimodal attention fusion neural network (Multi-modal Attention Fusion Network, MAFNet), which includes the following key modules: modality-specific feature extraction module, cross-modal attention mechanism, dynamic weight fusion module and strain classification module.

[0038] For image feature extraction, the improved EfficientNet-B0 is used as the backbone network, and the spatial attention mechanism is introduced to extract features from the first modality data to obtain image features. The output feature map of the last MBConv block of EfficientNet-B0 is recorded as , design the spatial attention module:

[0039] Channel Compression:

[0040] ;

[0041] Feature Fusion:

[0042] ;

[0043] Attention map generation:

[0044] ,in is the Sigmoid activation function;

[0045] Feature weighting:

[0046] , Represents a tensor product.

[0047] For spectral feature extraction, the second modality data is extracted based on one-dimensional convolution and self-attention mechanism to obtain spectral features. Among them, the one-dimensional convolution layer:

[0048] , Self-attention layer: compute query, key and value:

[0049] ; Calculate attention weights:

[0050] ; Output: O = AV; Network structure: Conv1D -> BatchNorm1D -> ReLU -> Self-Attention -> Conv1D -> GlobalAvgPool1D.

[0051] For biochemical index feature extraction, a multi-layer perceptron is used to extract the features of the third modality data to obtain biochemical index features. Input layer: ; Hidden layer:

[0052] ,in ; Output layer:

[0053] .

[0054] At step S103, the features of each modality are mapped to the target semantic space, and the features of each modality are updated according to the mutual relationship. Specifically, the features of each modality are mapped to the same semantic space, and a cross-modal attention matrix is ​​calculated; a cross-modal mutual relationship matrix is ​​constructed based on the cross-modal attention matrix to update the features of each modality. In one embodiment, it includes:

[0055] ,

[0056] in, Indicates the updated The characteristics of a mode, Represents the original The characteristics of a mode, represents the learnable feature update strength parameter, represents the total number of modes, represents the learnable balance parameter, Indicates the mode used to calculate and modal The learnable weight matrix of the similarity between Indicates The norm of the modal features, Indicates The query matrix of each modality, Indicates The key-value matrix of each modality, represents the dimension of the key value, represents the learnable weight matrix for feature transformation, Indicates the target modal index of the feature currently being updated, that is, the left side of the formula The corresponding mode, Represents the modality index that interacts and fuses information with the target modality, which is used to calculate the similarity between modality i and modality j and update the features of target modality i. It represents the index variable that traverses all modes. It appears below the summation symbol ∑, indicating that all modes are summed as part of the normalized denominator. Indicates traversal and accumulation of all modalities. In the formula, it is used to calculate the normalized denominator of the similarity between modality i and all other modalities. Indicates The transpose of the modal eigenvectors, Indicates that the traversal index is The eigenvectors of the modes, Indicates The norm of the modal features, Indicates The eigenvector of the mode.

[0057] Through cross-modal calculation, information from different modes can be effectively integrated. In organic fertilizer strain identification, this means that multiple data such as microscopic images, spectral characteristics, biochemical indicators, etc. can be used simultaneously to fully grasp the characteristics of strains and improve the accuracy and reliability of identification. Through a learnable weight matrix and attention mechanism, the method can adaptively learn the most relevant feature combination. This is particularly important in organic fertilizer strain identification because different strains may show significant differences in different features. The data types involved in organic fertilizer strain identification are diverse, including images, sequences, numerical values, etc. The method can effectively process these heterogeneous data, so that the embodiments of the present application can make full use of all available information. Through the cross-modal attention mechanism, the method can capture the complex nonlinear relationship between different modes. This is crucial for understanding the complex biological characteristics of organic fertilizer strains.

[0058] At the same time, attention weights can provide a certain degree of interpretability, which can help understand which features contribute most to the recognition results. This can provide valuable biological insights in the study of organic fertilizer strains.

[0059] At step S104, the global features of each modality are calculated respectively, and all global features are concatenated. In one embodiment, the process includes:

[0060] ,in, Representing modality The global characteristics of Representing modality The number of features, Representing modality No. Features, represents the learnable modality-specific parameters, represents the number of attention heads, Indicates The weight of the attention head, represents the activation function, represents element-wise multiplication, represents the sigmoid activation function, Represents the layer normalization operation.

[0061] Calculating the global features of each mode can capture the overall information and long-range dependencies within the mode. Specifically: Features By performing weighted averaging, we can obtain a global representation that integrates all local features of the modality. . This global feature can reflect the overall statistical characteristics and distribution of the modal data, rather than being limited to local features. For the image modality, the global feature can capture the semantic information of the entire image; for the spectral modality, the global feature can reflect the trend of the entire spectral curve; for the biochemical index modality, the global feature can summarize the comprehensive situation of multiple indicators. This design enables the calculation process of the global feature to take into account the characteristics and interrelationships of different modalities, thereby obtaining a richer and more representative feature representation.

[0062] In addition, although the global features are calculated, the method still retains the specific information of each modality: each modality has its own independent global features. , rather than mixing all the modes together. Learnable modality-specific parameters This allows the model to adjust the feature extraction strategy according to the characteristics of different modalities. The calculation of global features is based on the original features of the modality. This design not only realizes the information interaction between modalities, but also retains the uniqueness of each modality, which is beneficial to the subsequent multimodal fusion and classification tasks.

[0063] Multiple features of each mode Compressed into a global feature , can significantly reduce the dimension of features: For high-dimensional original features (such as image features), the calculation of global features is equivalent to a dimensionality reduction operation. The reduced feature dimension can reduce the amount of computation for subsequent processing and improve the training and reasoning efficiency of the model. Low-dimensional global features are easier to visualize and interpret, which helps to understand the working principle of the model. This dimensionality reduction operation improves the computational efficiency and interpretability of the model while retaining key information. The characteristics of different modal data are also taken into account: for the number of features For more modalities (such as images), the calculation of global features can effectively capture the statistical information of a large number of local features. For modalities with fewer features (such as biochemical indicators), the calculation of global features can enhance the expressiveness of features. By adjusting the learnable parameters, the model can adaptively adjust the calculation strategy of global features according to the importance of different modalities. This flexible design enables this method to adapt to multimodal data of different types and sizes, and has good versatility.

[0064] In multimodal learning, the number and quality of features of different modalities may vary greatly, which can easily lead to the model favoring certain modalities and ignoring other modalities. This problem can be alleviated to a certain extent by calculating global features: each modality obtains a global feature of a fixed dimension, balancing the status of different modalities in subsequent fusion. The introduction of learnable parameters allows the model to dynamically adjust the importance of different modalities instead of simply relying on the number of features. The LayerNorm operation can eliminate the differences in the distribution of features of different modalities, making subsequent fusion more fair. This design helps the model make full use of the information of each modality and improve the effect of multimodal learning.

[0065] At step S105, feature fusion is performed based on the updated features of each modality and the spliced ​​global features to obtain fused features. In one embodiment, the method includes:

[0066] ,in, represents the fusion feature, represents the balance factor, represents a multi-layer perceptron, represents the global features after splicing, represents the dynamic weight, represents the attention mechanism, Represents the updated features.

[0067] The concatenation of the global features of each modality contains the overall information of each modality. Through nonlinear transformation of MLP, high-level interactions between modalities can be learned. This helps to identify the overall correlation and complementarity between different modalities. Local features capture fine-grained interactions between different modal features through the attention mechanism. This can discover the association between local regions between modalities, which is conducive to extracting more refined discriminative information. By combining global and local features, this method can fully utilize the information of multimodal data and improve the richness and discriminability of feature representation.

[0068] At step S106, the fused features are classified to obtain the bacterial species of the biological organic fertilizer. The final bacterial species classification can be performed using a multi-head attention mechanism and a feedforward neural network:

[0069] Multi-head attention:

[0070] ,in,

[0071] ; Feedforward Neural Network:

[0072] ;

[0073] Classifier:

[0074] .

[0075] In addition, before extracting features from each modality in the multimodal dataset separately, the method also includes: separately training the neural network for feature extraction from each modality data and freezing the other parts of the neural network; freezing the neural network for feature extraction after training and training the other parts of the neural network; after the training is completed, fine-tuning the parameters of all neural networks using a small learning rate.

[0076] Specifically, a phased training strategy is adopted:

[0077] Stage 1: Train each modality feature extraction module separately and freeze other modules;

[0078] Stage 2: Freeze the feature extraction module and train the cross-modal attention and dynamic weight fusion modules;

[0079] Stage 3: Fine-tune the entire network, using a small learning rate.

[0080] In phase 3, the Cosine Annealing with Warm Restarts learning rate scheduling strategy is used:

[0081] ,in, is the current learning rate, and are the minimum and maximum learning rates, respectively, is the current epoch number, Initial setting , , .

[0082] During the training process, mixed precision training is adopted, using FP16 for forward propagation and gradient calculation, and FP32 for parameter update to improve training efficiency and reduce memory usage.

[0083] After the final bacterial species classification is performed through the above steps, the method of the present application can accurately classify the bacterial species in the organic fertilizer, such as Bacillus, lactic acid bacteria, yeast, actinomycetes, etc. The specific classification results include information such as the name, quantity and proportion of each bacteria. In addition, it should be noted that the technical solution of the present application replaces the detection method based on spectrum and the like in the prior art by fast identification of organic fertilizer bacterial species by multimodal fusion based on neural network, and has the advantages of fast identification speed, high accuracy and wide application range. The present solution is not limited to the identification of certain specific bacterial species, but can flexibly set the number and type of classification according to actual needs, and perform model training based on the corresponding data set. For example, if the goal is to identify the main dominant bacterial species in organic fertilizer, a smaller number of classifications can be set; if the goal is to comprehensively analyze the bacterial species composition of organic fertilizer, a larger number of classifications can be set to identify more bacterial species. In short, the present solution can set the number and type of classifications according to the scope of the identification target, and has strong flexibility and applicability.

[0084] Figure 2 A schematic block diagram of a biological organic fertilizer strain identification system based on a neural network provided for an embodiment of the present application. It should be understood that the system shown in the figure is exemplary and non-restrictive. This means that the system architecture involved is not limited to a specific form or design, but is presented as an example. In other words, the architecture shown in the figure can be regarded as a way of expression, in order to clearly describe related concepts and relationships, and does not exclude other forms of architecture. Therefore, when explaining the architecture in the picture, it should be understood that the model has flexibility and diversity, and its purpose is to provide an exemplary description rather than a restrictive provision to a specific form.

[0085] In the embodiment of the present application, it includes: a collection unit 201, a preprocessing unit 202, a feature unit 203 and an identification unit 204; wherein the collection unit 201 is used to collect multimodal information of biological organic fertilizer; the preprocessing unit 202 is used to perform data preprocessing on the multimodal information to obtain a multimodal data set; the feature unit 203 is used to extract features from each modality in the multimodal data set respectively; the features of each modality are mapped to the target semantic space, and the features of each modality are updated according to the mutual relationship; the global features of each modality are calculated respectively, and all global features are spliced; feature fusion is performed based on the updated features of each modality and the spliced ​​global features to obtain fused features; the identification unit 204 is used to classify the fused features to obtain the strain category of the biological organic fertilizer.

[0086] Furthermore, an embodiment of the present application also provides a biological organic fertilizer strain identification device based on a neural network, comprising: a processor, a memory, and a system bus; the processor and the memory are connected via the system bus; the memory is used to store one or more programs, and the one or more programs include instructions, which, when executed by the processor, enable the processor to execute any of the above methods.

[0087] Furthermore, an embodiment of the present application also provides a computer program product, which, when executed on a terminal device, enables the terminal device to execute any of the above methods.

[0088] It can be known from the description of the above implementation mode that those skilled in the art can clearly understand that all or part of the steps in the above-mentioned embodiment method can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., including several instructions for enabling a computer device (which can be a personal computer, a server, or a network communication device such as a media gateway, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0089] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0090] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0091] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A biological organic fertilizer strain identification method based on neural network, characterized in that: include: Collecting multimodal information of bio-organic fertilizer, and performing data preprocessing on the multimodal information to obtain a multimodal data set; Perform feature extraction on each modality in the multimodal dataset separately; Map the features of each modality to the target semantic space and update the features of each modality according to their mutual relationships; Map the features of each modality to the target semantic space and update the features of each modality according to their relationships, including: Map the features of each modality to the same semantic space and calculate the cross-modal attention matrix; Based on the cross-modal attention matrix, a cross-modal relationship matrix is ​​constructed to update the features of each modality; Based on the cross-modal attention matrix, a cross-modal relationship matrix is ​​constructed to update the features of each modality, including: in, represents the updated features of the ith modality, X i represents the feature vector of the i-th mode, β represents the learnable feature update strength parameter, M represents the total number of modes, α represents the learnable balance parameter, and W ij represents the learnable weight matrix used to calculate the similarity between modality i and modality j, ||X i || represents the norm of the i-th modal feature, Q i represents the query matrix of the i-th modality, K j represents the key-value matrix of the jth mode, d k Indicates the dimension of the key value, W j represents the learnable weight matrix for feature transformation, i represents the target modality index of the feature currently being updated, j represents the modality index that interacts and fuses information with the target modality, and k represents the index variable that traverses all modalities. Indicates traversal and accumulation of all modes. represents the transpose of the ith modal eigenvector, X k represents the eigenvector of the kth mode, ||X k || represents the norm of the kth modal feature, X j represents the eigenvector of the jth mode; Calculate the global features of each modality separately and concatenate all global features; Calculate the global features of each mode separately, including: Among them, G m represents the global characteristics of mode m, N m represents the number of features of mode m, represents the i-th feature of mode m, α m , β m , γ m represents the learnable modality-specific parameters, K represents the number of attention heads, and w j represents the weight of the j-th attention head, ReLU represents the activation function, ⊙ represents element-wise multiplication, σ represents the sigmoid activation function, and LayerNorm represents the layer normalization operation; Perform feature fusion based on the updated features of each modality and the concatenated global features to obtain fused features; Based on the updated features of each modality and the concatenated global features, feature fusion is performed to obtain fused features, including: Among them, F fused represents fusion features, λ represents the balance factor, MLP represents multi-layer perceptron, represents the global features after concatenation, γ ij represents dynamic weight, Attention represents attention mechanism, and represents the updated features; The fused features are classified to obtain the strain category of biological organic fertilizer.

2. The method for identifying bacterial species of biological organic fertilizer according to claim 1, characterized in that: in, Collecting multimodal information of bio-organic fertilizer and performing data preprocessing on the multimodal information to obtain a multimodal data set, including: Collect microscopic images of bio-organic fertilizer and remove salt and pepper noise through adaptive median filtering; The contrast-limited adaptive histogram equalization method is used to enhance the details of the microscopic image after noise reduction. Normalizing pixel values ​​of the microscopic image to a target range to obtain first modality data; Collect near-infrared spectral data of bio-organic fertilizer and use multivariate scattering correction to remove baseline drift; Savitzky-Golay filtering was applied to reduce noise and the standard normal variate method was used for standardization to obtain the second modality data; Multiple biochemical indices were collected, and the improved Z-score method was used to handle outliers; The processed biochemical indicators are standardized and mapped to obtain the third modality data; The first modality data, the second modality data, and the third modality data are concatenated to obtain a multimodal data set.

3. The method for identifying bacterial species of biological organic fertilizer according to claim 2, characterized in that: in, Feature extraction is performed on each modality in the multimodal dataset, including: The improved EfficientNet-B0 is used as the backbone network, and the spatial attention mechanism is introduced to extract features from the first modality data to obtain image features; Feature extraction is performed on the second modality data based on one-dimensional convolution and self-attention mechanism to obtain spectral features; A multi-layer perceptron is used to extract the features of the third modality data to obtain biochemical indicator features.

4. The method for identifying bacterial species of biological organic fertilizer according to claim 1, characterized in that: in, Before extracting features for each modality in the multimodal dataset, the following steps are also included: Train the neural network for feature extraction of each modality data separately, and freeze the neural network of other parts; Freeze the neural network for feature extraction after training, and train the neural network for other parts; After training is complete, fine-tune all neural network parameters using a small learning rate.

5. A biological organic fertilizer strain identification system based on neural network, characterized in that: include: Acquisition unit, preprocessing unit, feature unit and recognition unit; wherein, The collection unit is used to collect multimodal information of the bio-organic fertilizer; The preprocessing unit is used to perform data preprocessing on the multimodal information to obtain a multimodal data set; The feature unit is used to extract features from each modality in the multimodal data set respectively; Map the features of each modality to the target semantic space and update the features of each modality according to their mutual relationships; Map the features of each modality to the target semantic space and update the features of each modality according to their relationships, including: Map the features of each modality to the same semantic space and calculate the cross-modal attention matrix; Based on the cross-modal attention matrix, a cross-modal relationship matrix is ​​constructed to update the features of each modality; Based on the cross-modal attention matrix, a cross-modal relationship matrix is ​​constructed to update the features of each modality, including: in, represents the updated features of the ith modality, X i represents the feature vector of the i-th mode, β represents the learnable feature update strength parameter, M represents the total number of modes, α represents the learnable balance parameter, and W ij represents the learnable weight matrix used to calculate the similarity between modality i and modality j, ||X i || represents the norm of the i-th modal feature, Q i represents the query matrix of the i-th modality, K j represents the key-value matrix of the jth mode, d k Indicates the dimension of the key value, W j represents the learnable weight matrix for feature transformation, i represents the target modality index of the feature currently being updated, j represents the modality index that interacts and fuses information with the target modality, and k represents the index variable that traverses all modalities. Indicates traversal and accumulation of all modes. represents the transpose of the ith modal eigenvector, X k represents the eigenvector of the kth mode, ||X k || represents the norm of the kth modal feature, X j represents the eigenvector of the jth mode; Calculate the global features of each modality separately and concatenate all global features; Calculate the global features of each mode separately, including: Among them, G m represents the global characteristics of mode m, N m represents the number of features of mode m, represents the i-th feature of mode m, α m , β m , γ m represents the learnable modality-specific parameters, K represents the number of attention heads, and w j represents the weight of the j-th attention head, ReLU represents the activation function, ⊙ represents element-wise multiplication, σ represents the sigmoid activation function, and LayerNorm represents the layer normalization operation; Perform feature fusion based on the updated features of each modality and the concatenated global features to obtain fused features; Based on the updated features of each modality and the concatenated global features, feature fusion is performed to obtain fused features, including: Among them, F fused represents fusion features, λ represents the balance factor, MLP represents multi-layer perceptron, represents the global features after concatenation, γ ij represents dynamic weight, Attention represents attention mechanism, and represents the updated features; The identification unit is used to classify the fusion features to obtain the bacterial species category of the biological organic fertilizer.

6. A biological organic fertilizer strain identification device based on neural network, characterized in that: include: A processor, a memory, and a system bus; wherein the processor and the memory are connected via the system bus; The memory is used to store one or more programs, wherein the one or more programs include instructions, and when the instructions are executed by the processor, the processor executes the method according to any one of claims 1 to 4.