Intelligent regulation and control system and method for fermentation temperature of microbial fertilizer based on Internet of Things

Abstract

The invention relates to the field of intelligent fermentation temperature regulation and control, in particular to a microbial fertilizer fermentation temperature intelligent regulation and control system and method based on the Internet of Things. An artificial intelligence technology based on deep learning is adopted to carry out image analysis on a fermentation state image of the microbial fertilizer, visual feature representation of the fermentation state of the fertilizer is extracted, fermentation state information of the fertilizer is mined, and meanwhile, real-time temperature data in the fermentation process are combined to determine the fermentation state of the microbial fertilizer. And performing feedback control on the fermentation temperature based on the associated interactive fusion feature between the fermentation temperature and the fermentation state. Therefore, the method can dynamically adapt to activity changes of different batches of saccharomycetes to provide a more suitable temperature control instruction, so that the stability of the fermentation process of the microbial fertilizer is improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent fermentation temperature control, and specifically to a system and method for intelligent temperature control of microbial fertilizer fermentation based on the Internet of Things. Background Art

[0002] Microbial fertilizer is a kind of fertilizer that utilizes the biological activity of microorganisms to improve soil fertility and promote plant growth through the metabolic activities of microorganisms. In the production process of microbial fertilizers, fermentation temperature is a key factor affecting microbial activity and fertilizer quality. The regulation of fermentation temperature in the traditional compound microbial fertilizer fermentation process is mainly based on the manual experience of the operator. However, this experience-based fermentation temperature regulation method has poor accuracy and can easily lead to high or low fermentation temperatures, thereby affecting the quality of the final compound microbial fertilizer.

[0003] In this regard, China's patent application CN116661517B discloses an intelligent control system for the fermentation temperature of a compound microbial fertilizer based on the Internet of Things. The system simply divides the fermentation process of the compound microbial fertilizer into multiple growth stages, and controls the fermentation temperature according to the growth stages and fermentation environment parameters. However, due to the differences in the activity of different batches of yeast, this fixed-stage-based control strategy is prone to deviations, and there is still a problem of insufficient control accuracy, which makes it impossible to achieve reasonable control of the fermentation temperature, thereby affecting the fermentation effect and quality stability of the fertilizer.

[0004] Therefore, a system and method for intelligent temperature control of microbial fertilizer fermentation based on the Internet of Things is expected. Summary of the invention

[0005] This application is made in consideration of the above problems. One purpose of this application is to provide a microbial fertilizer fermentation temperature intelligent control system and method based on the Internet of Things.

[0006] The embodiment of the present application provides a microbial fertilizer fermentation temperature intelligent control system based on the Internet of Things, which includes:

[0007] A fermentation status monitoring module is used to obtain fermentation status images collected by a camera;

[0008] A fermentation state feature extraction module, used for performing fermentation state feature extraction and feature foreground salient processing on the fermentation state image to obtain a fermentation growth state foreground salient semantic feature map;

[0009] A fermentation temperature monitoring module, used to obtain a time series of fermentation temperature values ​​collected by a temperature sensor for a predetermined time period;

[0010] A fermentation temperature time series feature extraction module, used for performing time series encoding on the time series of the fermentation temperature values ​​to obtain an implicit feature vector associated with the fermentation temperature time series;

[0011] The fermentation temperature control module is used to determine the control instructions of the fermentation temperature based on the multimodal cross-domain joint features of the fermentation growth state prospect significant semantic feature map and the fermentation temperature time series associated implicit feature vector.

[0012] For example, according to the microbial fertilizer fermentation temperature intelligent control system based on the Internet of Things according to the embodiment of the present application, the fermentation state feature extraction module includes:

[0013] An image feature extraction unit, used for extracting image semantic features from the fermentation state image to obtain a fermentation growth state semantic feature map;

[0014] The feature foreground salient unit is used to input the fermentation growth state semantic feature map into the feature mask salient device to obtain the fermentation growth state foreground salient semantic feature map.

[0015] For example, according to the microbial fertilizer fermentation temperature intelligent control system based on the Internet of Things in an embodiment of the present application, the image feature extraction unit is used to:

[0016] The fermentation state image is input into a growth state feature extractor based on a hole convolutional neural network model to obtain the fermentation growth state semantic feature map.

[0017] For example, according to the microbial fertilizer fermentation temperature intelligent control system based on the Internet of Things according to the embodiment of the present application, the feature prospect salient unit includes:

[0018] A layer normalization subunit, used for performing layer normalization processing on the fermentation growth state semantic feature map to obtain a normalized fermentation growth state semantic feature map;

[0019] A deep convolutional coding subunit, used for performing channel expansion based on point convolution and deep convolutional coding based on a hole convolution layer on the normalized fermentation growth state semantic feature map to obtain a deep convolutional backup semantic feature map of the fermentation growth state and a deep convolutional original semantic feature map of the fermentation growth state;

[0020] A mask generation subunit is used to input the fermentation growth state deep convolution original semantic feature map into a foreground gated mask module based on the Gelu function to obtain a fermentation growth state deep convolution gated mask weight feature map;

[0021] A mask applying subunit, used for calculating the position point multiplication between the fermentation growth state deep convolution gated mask weight feature map and the fermentation growth state deep convolution backup semantic feature map to obtain a gated mask foreground highlighting fermentation growth state semantic feature map;

[0022] The channel modulation subunit is used to perform point convolution-based channel contraction on the gated mask foreground-prominent fermentation growth state semantic feature map to obtain the fermentation growth state foreground-salient semantic feature map.

[0023] For example, according to the microbial fertilizer fermentation temperature intelligent control system based on the Internet of Things according to the embodiment of the present application, the fermentation temperature time series feature extraction module is used to:

[0024] The time series of the fermentation temperature values ​​is input into a fermentation temperature sequence encoder based on a Bi-LSTM model to obtain an implicit feature vector associated with the fermentation temperature time series.

[0025] For example, according to the microbial fertilizer fermentation temperature intelligent control system based on the Internet of Things according to the embodiment of the present application, the fermentation temperature control module includes:

[0026] A cross-domain joint encoding unit, used for performing cross-domain joint encoding on the fermentation growth state foreground significant semantic feature map and the fermentation temperature time series associated implicit feature vector to obtain a fermentation temperature-growth state cross-domain joint representation feature map;

[0027] A control instruction generating unit is used to input the fermentation temperature-growth state cross-domain joint representation feature map into a classifier-based fermentation temperature control module to obtain a control instruction, wherein the control instruction is used to indicate whether the fermentation temperature value at the next time point should increase, decrease or remain unchanged.

[0028] For example, according to the microbial fertilizer fermentation temperature intelligent control system based on the Internet of Things according to an embodiment of the present application, the cross-domain joint encoding unit is used to:

[0029] The fermentation growth state foreground significant semantic feature map and the fermentation temperature time series associated implicit feature vector are input into a cross-domain joint encoder based on auxiliary modal information to obtain the fermentation temperature-growth state cross-domain joint representation feature map.

[0030] For example, according to the microbial fertilizer fermentation temperature intelligent control system based on the Internet of Things according to an embodiment of the present application, the cross-domain joint encoding unit is used to:

[0031] Performing linear transformation and characteristic shape reshaping on the fermentation temperature time series associated implicit feature vector to obtain a fermentation temperature auxiliary modal modulation matrix;

[0032] Performing a first linear transformation and a second linear transformation on the fermentation growth state prospect significant semantic feature map to obtain a first linear transformation fermentation growth state prospect significant semantic feature map and a second linear transformation fermentation growth state prospect significant semantic feature map;

[0033] Reshaping the first linear transformation fermentation growth state foreground significant semantic feature map and the second linear transformation fermentation growth state foreground significant semantic feature map to obtain a first linear transformation fermentation growth state foreground significant semantic feature matrix and a second linear transformation fermentation growth state foreground significant semantic feature matrix;

[0034] Taking the first linear transformation fermentation growth state prospect significant semantic feature matrix and the second linear transformation fermentation growth state prospect significant semantic feature matrix as a key matrix and a value matrix and taking the fermentation temperature auxiliary modal modulation matrix as a query matrix, inputting the query matrix, the key matrix and the value matrix into a Transformer structure to obtain an auxiliary modal attention query result matrix;

[0035] Reshape and perform point convolution encoding on the auxiliary modality attention query result matrix to obtain an auxiliary modality attention query result feature map;

[0036] The auxiliary modal attention query result feature map and the fermentation growth state foreground significant semantic feature map are calculated and multiplied by the position points to obtain the fermentation temperature-growth state cross-domain joint representation feature map.

[0037] The embodiment of the present application also provides a method for intelligently controlling the fermentation temperature of microbial fertilizer based on the Internet of Things, which comprises:

[0038] Acquire a fermentation state image captured by a camera;

[0039] Performing fermentation state feature extraction and feature foreground salient processing on the fermentation state image to obtain a fermentation growth state foreground salient semantic feature map;

[0040] Acquire a time series of fermentation temperature values ​​collected by a temperature sensor for a predetermined time period;

[0041] Performing time series encoding on the time series of the fermentation temperature values ​​to obtain an implicit feature vector associated with the fermentation temperature time series;

[0042] Based on the multimodal cross-domain joint features of the fermentation growth state prospect salient semantic feature graph and the fermentation temperature time series associated implicit feature vector, the control instruction of the fermentation temperature is determined.

[0043] For example, according to the method for intelligently controlling the fermentation temperature of microbial fertilizer based on the Internet of Things according to the embodiment of the present application, the fermentation state image is subjected to fermentation state feature extraction and feature foreground salient processing to obtain a fermentation growth state foreground salient semantic feature map, including:

[0044] Extracting image semantic features from the fermentation state image to obtain a fermentation growth state semantic feature map;

[0045] The fermentation growth state semantic feature map is input into a feature mask salient device to obtain the fermentation growth state foreground salient semantic feature map.

[0046] According to the microbial fertilizer fermentation temperature intelligent control system and method based on the Internet of Things of the embodiment of the present application, it uses artificial intelligence technology based on deep learning to analyze the fermentation state image of the microbial fertilizer, extract the visual feature representation of the fertilizer fermentation state, and mine the fermentation state information of the fertilizer. At the same time, combined with the real-time temperature data during the fermentation process, the fermentation temperature is feedback controlled based on the correlation interaction fusion characteristics between the fermentation temperature and the fermentation state. In this way, it can dynamically adapt to the activity changes of different batches of yeast to provide more suitable temperature control instructions, thereby improving the stability of the microbial fertilizer fermentation process. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings of the embodiments of the present application are briefly introduced below. Obviously, the drawings described below only relate to some embodiments of the present application, and are not intended to limit the present application.

[0048] Figure 1 The schematic diagram of the application architecture of the microbial fertilizer fermentation temperature intelligent control system based on the Internet of Things in the embodiment of the present application is shown;

[0049] Figure 2 The schematic diagram of the structure of the intelligent temperature control system for microbial fertilizer fermentation based on the Internet of Things in the embodiment of the present application is shown;

[0050] Figure 3 The structure diagram of the fermentation state feature extraction module of the microbial fertilizer fermentation temperature intelligent control system based on the Internet of Things in the embodiment of the present application is shown;

[0051] Figure 4 The structure diagram of the fermentation temperature control module of the microbial fertilizer fermentation temperature intelligent control system based on the Internet of Things in the embodiment of the present application is shown;

[0052] Figure 5 A flow chart showing a method for intelligently controlling the fermentation temperature of microbial fertilizer based on the Internet of Things in an embodiment of the present application is shown; and

[0053] Figure 6 An application scenario diagram of the microbial fertilizer fermentation temperature intelligent control system based on the Internet of Things in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0054] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without creative work also fall within the scope of protection of the present application.

[0055] The terms used in this specification are those common terms currently widely used in the art in consideration of the functions of the present application, but these terms may vary according to the intentions of those skilled in the art, precedents, or new technologies in the art. In addition, specific terms may be selected, and in this case, their detailed meanings will be described in the detailed description of the present application. Therefore, the terms used in the specification should not be understood as simple names, but rather as a general description based on the meaning of the terms and the present application.

[0056] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0057] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. At the same time, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.

[0058] Figure 1 A schematic diagram of the application architecture of the intelligent temperature control system for microbial fertilizer fermentation based on the Internet of Things in an embodiment of the present application is shown, including a server 100 and a terminal device 200.

[0059] The terminal device 200 and the server 100 can be connected via the Internet to achieve mutual communication. Optionally, the above-mentioned Internet uses standard communication technology and / or protocol. The Internet is usually the Internet, but it can also be any network, including but not limited to any combination of a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a dedicated network or a virtual private network. In some embodiments, the data exchanged through the network is represented by technologies and / or formats including Hyper Text Markup Language (HTML), Extensible Markup Language (XML), etc. In addition, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPsec) can also be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technologies can also be used to replace or supplement the above-mentioned data communication technologies.

[0060] The server 100 can provide various network services for the terminal device 200, wherein the server 100 can be a single server, a server cluster consisting of several servers, or a cloud computing center. Specifically, the server 100 may include a processor 110 (Center Processing Unit, CPU), a memory 120, an input device 130, and an output device 140, etc. The input device 130 may include a keyboard, a mouse, a touch screen, etc., and the output device 140 may include a display device, such as a liquid crystal display (Liquid Crystal Display, LCD), a cathode ray tube (Cathode Ray Tube, CRT), etc.

[0061] The memory 120 may include a read-only memory (ROM) and a random access memory (RAM), and provides the processor 110 with program instructions and data stored in the memory 120. In the embodiment of the present application, the memory 120 may be used to store the corresponding method of the intelligent temperature control system for microbial fertilizer fermentation based on the Internet of Things in the embodiment of the present application.

[0062] The processor 110 calls the program instructions stored in the memory 120, and the processor 110 is used to execute the steps of any one of the methods for intelligent temperature control of microbial fertilizer fermentation based on the Internet of Things in the embodiments of the present application according to the obtained program instructions.

[0063] In addition, the application architecture diagram in the embodiment of the present application is intended to more clearly illustrate the technical solution in the embodiment of the present application, and does not constitute a limitation on the technical solution provided in the embodiment of the present application. Of course, for other application architectures and business applications, the technical solution provided in the embodiment of the present application is also applicable to similar problems.

[0064] The following is a non-restrictive description of the IoT-based intelligent temperature control system for microbial fertilizer fermentation provided according to at least one embodiment of the present application through several examples or embodiments. As described below, different features in these specific examples or embodiments can be combined with each other without conflicting with each other to obtain new examples or embodiments, and these new examples or embodiments also fall within the scope of protection of the present application.

[0065] In response to the above technical problems, the technical concept of this application is to use artificial intelligence technology based on deep learning to analyze the fermentation state image of microbial fertilizer, extract the visual feature representation of the fertilizer fermentation state, and mine the fermentation state information of the fertilizer. At the same time, combined with the real-time temperature data during the fermentation process, the fermentation temperature is feedback controlled based on the correlation and interaction fusion characteristics between the fermentation temperature and the fermentation state. In this way, it is possible to dynamically adapt to the activity changes of different batches of yeast to provide more suitable temperature control instructions, thereby improving the stability of the microbial fertilizer fermentation process.

[0066] Based on this, Figure 2 The schematic diagram of the structure of the microbial fertilizer fermentation temperature intelligent control system 800 based on the Internet of Things in the embodiment of the present application is shown. The microbial fertilizer fermentation temperature intelligent control system 800 based on the Internet of Things includes: a fermentation state monitoring module 810, which is used to obtain a fermentation state image collected by a camera; a fermentation state feature extraction module 820, which is used to extract fermentation state features and perform feature foreground salient processing on the fermentation state image to obtain a fermentation growth state foreground significant semantic feature map; a fermentation temperature monitoring module 830, which is used to obtain a time series of fermentation temperature values ​​collected by a temperature sensor in a predetermined time period; a fermentation temperature time series feature extraction module 840, which is used to perform time series encoding on the time series of the fermentation temperature values ​​to obtain a fermentation temperature time series associated implicit feature vector; a fermentation temperature control module 850, which is used to determine the control instruction of the fermentation temperature based on the multimodal cross-domain joint features of the fermentation growth state foreground significant semantic feature map and the fermentation temperature time series associated implicit feature vector.

[0067] Specifically, in the technical solution of the present application, first, a fermentation state image collected by a camera is obtained. It should be understood that the camera can capture and record in real time the visual changes in the fermentation process of the microbial fertilizer, such as the color and transparency of the fermentation liquid, bubble generation, the morphology, size, color, etc. of the bacterial colonies. In other words, the fermentation state image can directly reflect the activity of the fermentation bacteria and the growth status of the microorganisms, providing an important basis for the intelligent regulation of the fermentation temperature.

[0068] Next, in order to extract key fermentation state visual information from the fermentation state image, the present application uses a growth state feature extractor based on a void convolutional neural network model to process the fermentation state image, so as to utilize the special void structure of the void convolutional neural network to increase the network's receptive field while maintaining a high spatial resolution, thereby enhancing the ability to recognize the fermentation state, thereby effectively capturing a wider range of spatial contextual information in the fermentation state image, mining deeper fermentation state features, and generating a fermentation growth state semantic feature map.

[0069] Secondly, considering that there is usually a large amount of background information in the fermentation state image, the fermentation growth state semantic feature map may contain more noise interference or non-critical information. To this end, the present application further introduces a feature mask salient device to perform foreground highlighting on the fermentation growth state semantic feature map. Specifically, the feature foreground mask salient device first performs layer normalization on the fermentation growth state semantic feature map to eliminate the scale difference between layers. Subsequently, deep convolution coding is performed on it to provide rich spatial and depth information. Next, based on the gating mechanism, adaptive feature learning is performed on the semantic features of the fermentation growth state after deep convolution coding, and a feature mask is generated to weight and emphasize the key information to highlight the foreground area with the highest correlation with the fermentation state, effectively suppressing background noise. Finally, a point convolution operation is performed on the masked weighted feature map to achieve channel contraction, so that its feature dimension is consistent with the original feature map, so as to obtain a foreground salient semantic feature map of the fermentation growth state.

[0070] Accordingly, if Figure 3 As shown, the fermentation state feature extraction module 820 includes: an image feature extraction unit 821, which is used to extract image semantic features from the fermentation state image to obtain a fermentation growth state semantic feature map; a feature foreground salient unit 822, which is used to input the fermentation growth state semantic feature map into a feature mask salient device to obtain the fermentation growth state foreground salient semantic feature map.

[0071] Wherein, the image feature extraction unit 821 is used to: input the fermentation state image into a growth state feature extractor based on a hole convolutional neural network model to obtain the fermentation growth state semantic feature map.

[0072] The feature foreground saliency unit 822 includes: a layer normalization subunit, which is used to perform layer normalization processing on the fermentation growth state semantic feature map to obtain a normalized fermentation growth state semantic feature map; a deep convolution encoding subunit, which is used to perform channel expansion based on point convolution and deep convolution encoding based on a hole convolution layer on the normalized fermentation growth state semantic feature map to obtain a fermentation growth state deep convolution backup semantic feature map and a fermentation growth state deep convolution original semantic feature map; a mask generation subunit, which is used to convert the fermentation growth state deep convolution original semantic feature map into a mask. The feature map is input into a foreground gated mask module based on the Gelu function to obtain a fermentation growth state deep convolution gated mask weight feature map; a mask application subunit is used to calculate the position point multiplication between the fermentation growth state deep convolution gated mask weight feature map and the fermentation growth state deep convolution backup semantic feature map to obtain a gated mask foreground highlighting fermentation growth state semantic feature map; a channel modulation subunit is used to perform point convolution-based channel contraction on the gated mask foreground highlighting fermentation growth state semantic feature map to obtain the fermentation growth state foreground salient semantic feature map.

[0073] In a specific example, the feature foreground salient unit 822 is used to process the fermentation growth state semantic feature map using the following foreground mask highlighting formula to obtain the fermentation growth state foreground salient semantic feature map, wherein the foreground mask highlighting formula is:

[0074] F ln =Layer Normalization(F)

[0075] F m =Conv 3×3,2 (Conv 1×1 (F ln ))

[0076] F r =Conv 3×3,2 (Conv 1×1 (F ln ))

[0077] F a =mask[Gelu(F m )]

[0078]

[0079] F t =F a ⊙F r

[0080] F i =Conv1×1 (F t )

[0081] Wherein, F represents the semantic feature map of the fermentation growth state, Layer Normalizeation(·) represents the layer normalization operation, and F ln Represents the normalized fermentation growth state semantic feature map, Conv 1×1 Represents point convolution, Conv 3×3,2 represents a 3×3 dilated convolution with a dilation number of 2, F m Represents the deep convolution original semantic feature map of the fermentation growth state, F r represents the deep convolutional semantic feature map of the fermentation growth state, Gelu(·) represents the GELU activation function, mask(·) represents the gated mask processing, θ is the preset gate threshold, F a represents the deep convolution gated mask weight feature map of the fermentation growth state, ⊙ represents the position point multiplication operation, F t The gated mask foreground highlights the semantic feature map of the fermentation growth state, F i A foreground significant semantic feature map representing the fermentation growth state.

[0082] During the fermentation process, maintaining a suitable temperature is crucial for the growth of microorganisms and the fermentation effect of fertilizers. Based on this, in the technical solution of the present application, a time series of fermentation temperature values ​​collected by a temperature sensor in a predetermined time period is further obtained to monitor the change of the fermentation temperature in real time, and adjust the temperature control strategy according to the actual situation to ensure that the fermentation process is carried out under suitable temperature conditions.

[0083] Then, in order to capture the temporal variation trend of the fermentation temperature, the present application adopts a fermentation temperature sequence encoder based on the Bi-LSTM model to process the time series of the fermentation temperature values. The Bi-LSTM model can effectively learn the long-term dependencies by performing recursive operations on the time series of the fermentation temperature values, while maintaining sensitivity to short-term temperature changes, and fully understanding the temporal dynamic characteristics of temperature changes, so as to generate implicit feature vectors associated with the fermentation temperature time series.

[0084] Correspondingly, the fermentation temperature time series feature extraction module 840 is used to: input the time series of the fermentation temperature values ​​into a fermentation temperature sequence encoder based on a Bi-LSTM model to obtain an implicit feature vector associated with the fermentation temperature time series.

[0085] Next, considering that the fermentation growth state prospect significant semantic feature map provides intuitive visual information of the fermentation process, and the fermentation temperature time series associated implicit feature vector reflects the dynamic change of temperature during the fermentation process, the two are complementary in content. In order to more comprehensively describe the actual situation of the fermentation process, the present application introduces a cross-domain joint encoder based on auxiliary modal information to perform cross-modal association fusion on the fermentation growth state prospect significant semantic feature map and the fermentation temperature time series associated implicit feature vector. Specifically, the cross-domain joint encoder linearly transforms and reshapes the fermentation growth state prospect significant semantic feature map and the fermentation temperature time series associated implicit feature vector to adapt to the input requirements of the Transformer structure, and uses the fermentation temperature feature as a query, the fermentation state feature as a key and value, and the fermentation temperature feature as auxiliary feature modal information to guide the cross-domain union between the fermentation state feature and the fermentation temperature feature. Through the self-attention mechanism of the Transformer, interactive learning and information complementarity between the two are achieved, effectively enhancing the expression ability of the feature, and forming a deep understanding of the fermentation process to generate a fermentation temperature-growth state cross-domain joint representation feature map.

[0086] Then, the fermentation temperature-growth state cross-domain joint representation feature map is input into the classifier-based fermentation temperature control module to obtain a control instruction, and the control instruction is used to indicate whether the fermentation temperature value at the next time point should be increased, decreased, or kept unchanged. Specifically, the classifier performs dimensionality reduction processing and full connection encoding on the fermentation temperature-growth state cross-domain joint representation feature map to capture the complex interaction pattern between temperature and growth state during the fermentation process, and probabilistically normalizes it through a softmax function to predict the optimal temperature adjustment direction under the current fermentation state and temperature conditions, and generates probability distributions corresponding to whether the fermentation temperature should be increased, decreased, or kept unchanged, respectively, and then determines the fermentation temperature control instruction at the next time point according to the probability distribution to guide the actual temperature control system to make corresponding adjustments.

[0087] Accordingly, if Figure 4 As shown, the fermentation temperature control module 850 includes: a cross-domain joint encoding unit 851, which is used to perform cross-domain joint encoding on the fermentation growth state foreground significant semantic feature map and the fermentation temperature time series associated implicit feature vector to obtain a fermentation temperature-growth state cross-domain joint representation feature map; a control instruction generation unit 852, which is used to input the fermentation temperature-growth state cross-domain joint representation feature map into a classifier-based fermentation temperature control module to obtain a control instruction, and the control instruction is used to indicate that the fermentation temperature value at the next time point should increase, decrease or remain unchanged.

[0088] Among them, the cross-domain joint encoding unit 851 is used to: input the fermentation growth state foreground significant semantic feature map and the fermentation temperature time series associated implicit feature vector into a cross-domain joint encoder based on auxiliary modal information to obtain the fermentation temperature-growth state cross-domain joint representation feature map.

[0089] Among them, the cross-domain joint encoding unit 851 is used to: perform linear transformation and feature shape reshaping on the fermentation temperature time series associated implicit feature vector to obtain a fermentation temperature auxiliary modal modulation matrix; perform a first linear transformation and a second linear transformation on the fermentation growth state foreground significant semantic feature map to obtain a first linear transformation fermentation growth state foreground significant semantic feature map and a second linear transformation fermentation growth state foreground significant semantic feature map; perform feature shape reshaping on the first linear transformation fermentation growth state foreground significant semantic feature map and the second linear transformation fermentation growth state foreground significant semantic feature map to obtain a first linear transformation fermentation growth state foreground significant semantic feature matrix and a second linear transformation fermentation growth state foreground significant semantic feature map. feature matrix; using the first linear transformation fermentation growth state foreground significant semantic feature matrix and the second linear transformation fermentation growth state foreground significant semantic feature matrix as the key matrix and the value matrix and using the fermentation temperature auxiliary modal modulation matrix as the query matrix, inputting the query matrix, the key matrix and the value matrix into the Transformer structure to obtain the auxiliary modal attention query result matrix; reshaping and point convolution encoding are performed on the auxiliary modal attention query result matrix to obtain the auxiliary modal attention query result feature map; calculating the position point multiplication of the auxiliary modal attention query result feature map and the fermentation growth state foreground significant semantic feature map to obtain the fermentation temperature-growth state cross-domain joint representation feature map.

[0090] In a specific example, the cross-domain joint coding unit 851 is used to process the fermentation growth state foreground significant semantic feature map and the fermentation temperature time series associated implicit feature vector using the following cross-domain joint coding formula to obtain the fermentation temperature-growth state cross-domain joint representation feature map, wherein the cross-domain joint coding formula is:

[0091] M s = reshape(σ(W s V s +B s ))

[0092] F1=σ(W1F i +B1)

[0093] F2=σ(W2F i +B2)

[0094] M1=reshape(F1)

[0095] M2=reshape(F2)

[0096]

[0097] F w =Conv 1×1 [reshape(M t )]

[0098] F s =F w ⊙F i

[0099] Among them, V s represents the fermentation temperature time series associated implicit feature vector, F i represents the foreground significant semantic feature map of the fermentation growth state, reshape(·) represents the feature shape reshaping, W s , W1 and W2 represent different linear transformation weight matrices, B s , B1 and B2 represent different bias matrices, σ represents the Sigmoid activation function, F1 represents the first linear transformation fermentation growth state foreground significant semantic feature map, F2 represents the second linear transformation fermentation growth state foreground significant semantic feature map, M s represents the fermentation temperature auxiliary modal modulation matrix, M1 represents the first linear transformation fermentation growth state prospect significant semantic feature matrix, M2 represents the second linear transformation fermentation growth state prospect significant semantic feature matrix, (·) T represents the transpose of a matrix, represents matrix multiplication, softmax represents the normalized exponential function, M t Represents the auxiliary modality attention query result matrix, Conv 1×1 represents point convolution, F w represents the auxiliary modality attention query result feature map, ⊙ represents the position point multiplication, F s A cross-domain joint representation feature graph of the fermentation temperature-growth state is represented.

[0100] Preferably, in a preferred example, the control instruction generating unit 852 includes: an optimization subunit, used to optimize the fermentation temperature-growth state cross-domain joint representation feature graph to obtain the optimized fermentation temperature-growth state cross-domain joint representation feature graph; a classification subunit, used to input the optimized fermentation temperature-growth state cross-domain joint representation feature graph into the classifier-based fermentation temperature control module to obtain the control instruction.

[0101] In the technical solution of the present application, considering that the fermentation growth state foreground significant semantic feature map and the fermentation temperature time series associated implicit feature vector respectively represent the feature foreground significant image semantic features of the fermentation state image and the fermentation temperature local time series associated features, when the fermentation growth state foreground significant semantic feature map and the fermentation temperature time series associated implicit feature vector are input into the heterogeneous Transformer cross-domain joint encoder based on the auxiliary modality, the fermentation temperature time series associated implicit feature vector is used as the auxiliary feature modality information to guide the cross-domain union between the fermentation growth state foreground significant semantic feature map and the fermentation temperature time series associated implicit feature vector. However, due to the difference in feature modalities, the fermentation temperature-growth state cross-domain joint representation feature map will have insufficient cross-modal feature coverage relative to the fermentation growth state foreground significant semantic feature map, resulting in interference components in the fermentation temperature-growth state cross-domain joint representation feature map.

[0102] Accordingly, the interference components of the fermentation temperature-growth status cross-domain joint representation feature graph will ensure the existence of fine-grained feature structure holes in the overall feature manifold of the fermentation temperature-growth status cross-domain joint representation feature graph in the high-dimensional feature space. The existence of fine-grained feature structure holes will not only make the features of the fermentation temperature-growth status cross-domain joint representation feature graph corresponding to the semantic coverage of the class probability labels insufficient, but will also cause the outlier class regression inference mapping to be off-target, affecting the accuracy of the recognition result obtained by the fermentation temperature-growth status cross-domain joint representation feature graph input into the classifier-based state recognizer.

[0103] In the technical solution of the present application, in a preferred example, the fermentation temperature-growth state cross-domain joint representation feature map is input into a state recognizer based on a classifier to obtain a recognition result, including:

[0104] Expanding the fermentation temperature-growth state cross-domain joint representation feature graph into a fermentation temperature-growth state cross-domain joint representation feature vector;

[0105] Inputting the fermentation temperature-growth state cross-domain joint representation feature vector into a classifier-based state identifier based on a Softmax function to obtain a fermentation temperature-growth state cross-domain joint class probability label vector;

[0106] Calculating the product between the fermentation temperature-growth state cross-domain joint class probability label vector and its transposed vector to obtain a fermentation temperature-growth state cross-domain joint full label domain modulation matrix;

[0107] Dividing the eigenvalues ​​of each position in the fermentation temperature-growth state cross-domain joint full label domain modulation matrix by the square root of the scale value of the fermentation temperature-growth state cross-domain joint representation eigenvector to obtain the fermentation temperature-growth state cross-domain joint full label domain scale modulation matrix;

[0108] Inputting the fermentation temperature-growth state cross-domain joint full-label domain scale modulation matrix into a quasi-probability domain attention sparse module based on a multi-level gating function to obtain a fermentation temperature-growth state cross-domain joint full-label domain scale sparse modulation matrix;

[0109] Taking the fermentation temperature-growth state cross-domain joint representation feature vector as a query feature vector, matrix multiplying the fermentation temperature-growth state cross-domain joint full-label domain scale sparse modulation matrix with the fermentation temperature-growth state cross-domain joint representation feature vector to obtain an optimized fermentation temperature-growth state cross-domain joint representation feature vector;

[0110] The optimized fermentation temperature-growth state cross-domain joint representation feature vector is input into the classifier-based state identifier to obtain the identification result.

[0111] The fermentation temperature-growth state cross-domain joint representation feature vector optimization process is expressed by the formula:

[0112]

[0113]

[0114] λ≥1

[0115]

[0116] Among them, v i represents the eigenvalue of the ith position of the fermentation temperature-growth state cross-domain joint representation eigenvector, L represents the scale value of the fermentation temperature-growth state cross-domain joint representation eigenvector, Represents the eigenvalue of the ith position of the fermentation temperature-growth state cross-domain joint class probability label vector, V p represents the fermentation temperature-growth state cross-domain joint class probability label vector, represents matrix multiplication, T represents the transpose of the vector, M p represents the fermentation temperature-growth state cross-domain joint full-label domain scale modulation matrix, θ represents the predetermined threshold, represents the eigenvalue of the ith position of the fermentation temperature-growth state cross-domain joint full-label domain scale modulation matrix, λ represents a predetermined hyperparameter, mask represents a mask function, represents the eigenvalue of the ith position of the fermentation temperature-growth state cross-domain joint full-label domain scale sparse modulation matrix, M p ' represents the fermentation temperature-growth state cross-domain joint full-label domain scale sparse modulation matrix, and V' represents the optimized fermentation temperature-growth state cross-domain joint representation feature vector.

[0117] Therefore, by adding a class probability domain-level optimizable disturbance to the original feature distribution to strengthen the dependency entanglement between different variables in the original feature vector based on label domain modulation, the source domain feature vector can be optimized based on class probability query-based scale modulation. In this way, the significant inherent modal information in the source domain features can be more effectively retained, and the interference components in the feature distribution can be effectively masked, thereby improving the adversarial robustness of the manifold expression of the feature distribution, so as to improve the accuracy of the recognition results obtained by the classifier-based state recognizer.

[0118] Based on the above embodiments, see Figure 5 , which is a flow chart of a method for intelligently controlling the fermentation temperature of microbial fertilizer based on the Internet of Things in an embodiment of the present application. For example, the method for intelligently controlling the fermentation temperature of microbial fertilizer based on the Internet of Things can be executed by a server, which can be Figure 1 The server 100 shown in FIG. Figure 5 As shown, according to the embodiment of the present application, the method for intelligent control of the fermentation temperature of a microbial fertilizer based on the Internet of Things includes the following steps: S510, obtaining a fermentation state image collected by a camera; S520, performing fermentation state feature extraction and feature foreground salient processing on the fermentation state image to obtain a fermentation growth state foreground significant semantic feature map; S530, obtaining a time series of fermentation temperature values ​​of a predetermined time period collected by a temperature sensor; S540, performing time series encoding on the time series of the fermentation temperature values ​​to obtain a fermentation temperature time series associated implicit feature vector; S550, determining the control instruction of the fermentation temperature based on the multimodal cross-domain joint features of the fermentation growth state foreground significant semantic feature map and the fermentation temperature time series associated implicit feature vector.

[0119] In one example, the fermentation state image is subjected to fermentation state feature extraction and feature foreground salient processing to obtain a fermentation growth state foreground salient semantic feature map, including: performing image semantic feature extraction on the fermentation state image to obtain a fermentation growth state semantic feature map; inputting the fermentation growth state semantic feature map into a feature mask salientator to obtain the fermentation growth state foreground salient semantic feature map.

[0120] Here, those skilled in the art can understand that the specific operations of each step in the above-mentioned method for intelligent temperature control of microbial fertilizer fermentation based on the Internet of Things have been referred to above. Figures 2 to 4The description of the IoT-based microbial fertilizer fermentation temperature intelligent control system 800 has been introduced in detail, and therefore, its repeated description will be omitted.

[0121] Figure 6 FIG. 1 is an application scenario diagram of the microbial fertilizer fermentation temperature intelligent control system based on the Internet of Things according to an embodiment of the present application. Figure 6 As shown, in this application scenario, first, a fermentation state image captured by a camera is obtained (for example, Figure 6 D1 as shown in FIG. 1 ) and a time series of fermentation temperature values ​​collected by a temperature sensor for a predetermined period of time (e.g., Figure 6 Then, the fermentation state image and the time series of the fermentation temperature value are input to a server (for example, Figure 6 In S) shown in , the server is able to use the IoT-based microbial fertilizer fermentation temperature intelligent control algorithm to process the fermentation status image and the time series of the fermentation temperature value to obtain a control instruction indicating that the fermentation temperature value at the next time point should increase, decrease or remain unchanged.

[0122] Based on the above embodiments, another exemplary embodiment of an electronic device is also provided in the embodiments of the present application. In some possible embodiments, the electronic device in the embodiments of the present application may include a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor can implement the steps of the method for intelligently controlling the fermentation temperature of microbial fertilizer based on the Internet of Things in the above embodiments when executing the program.

[0123] For example, in the case of electronic equipment Figure 1 Taking the server 100 in the example as an example, the processor in the electronic device is the processor 110 in the server 100, and the memory in the electronic device is the memory 120 in the server 100.

[0124] The embodiment of the present application also provides a computer-readable storage medium, on which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, the method for intelligently controlling the temperature of microbial fertilizer fermentation based on the Internet of Things according to the embodiment of the present application described with reference to the above drawings can be executed. The computer-readable storage medium includes, but is not limited to, for example, volatile memory and / or non-volatile memory. The volatile memory may, for example, include random access memory (RAM) and / or cache memory (cache), etc. The non-volatile memory may, for example, include read-only memory (ROM), hard disk, flash memory, etc.

[0125] The embodiments of the present application also provide a computer program product or a computer program, which includes computer executable instructions, and the computer executable instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer executable instructions from the computer-readable storage medium, and the processor executes the computer executable instructions, so that the computer device executes the microbial fertilizer fermentation temperature intelligent control method based on the Internet of Things according to the embodiment of the present application.

[0126] Those skilled in the art will appreciate that the contents disclosed in this application may be subject to various variations and improvements. For example, the various devices or components described above may be implemented by hardware, or by software, firmware, or a combination of some or all of the three.

[0127] In addition, although the present application makes various references to certain units in the system according to embodiments of the present application, any number of different units can be used and run on the client and / or server. The units are only illustrative, and different aspects of the system and method can use different units.

[0128] Those skilled in the art will appreciate that all or part of the steps in the above method can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, such as a read-only memory, a disk or an optical disk. Optionally, all or part of the steps in the above embodiment can also be implemented using one or more integrated circuits. Accordingly, each module / unit in the above embodiment can be implemented in the form of hardware or in the form of software function modules. The present application is not limited to any particular form of combination of hardware and software.

[0129] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those of ordinary skill in the art to which this application belongs. It should also be understood that terms such as those defined in common dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an idealized or extremely formal sense, unless explicitly defined as such herein.

[0130] The above is an explanation of the present application and should not be considered as limiting thereof. Although several exemplary embodiments of the present application have been described, those skilled in the art will readily appreciate that many modifications may be made to the exemplary embodiments without departing from the novel teachings and advantages of the present application.

Claims

1. A microbial fertilizer fermentation temperature intelligent control system based on the Internet of Things, characterized in that: include: A fermentation status monitoring module is used to obtain fermentation status images collected by a camera; A fermentation state feature extraction module, used for performing fermentation state feature extraction and feature foreground salient processing on the fermentation state image to obtain a fermentation growth state foreground salient semantic feature map; A fermentation temperature monitoring module, used to obtain a time series of fermentation temperature values ​​collected by a temperature sensor for a predetermined time period; A fermentation temperature time series feature extraction module, used for performing time series encoding on the time series of the fermentation temperature values ​​to obtain an implicit feature vector associated with the fermentation temperature time series; The fermentation temperature control module is used to determine the control instructions of the fermentation temperature based on the multimodal cross-domain joint features of the fermentation growth state prospect significant semantic feature map and the fermentation temperature time series associated implicit feature vector.

2. The microbial fertilizer fermentation temperature intelligent control system based on the Internet of Things according to claim 1 is characterized in that: The fermentation state feature extraction module comprises: An image feature extraction unit, used for extracting image semantic features from the fermentation state image to obtain a fermentation growth state semantic feature map; The feature foreground salient unit is used to input the fermentation growth state semantic feature map into the feature mask salient device to obtain the fermentation growth state foreground salient semantic feature map.

3. The microbial fertilizer fermentation temperature intelligent control system based on the Internet of Things according to claim 2 is characterized in that: The image feature extraction unit is used to: The fermentation state image is input into a growth state feature extractor based on a hole convolutional neural network model to obtain the fermentation growth state semantic feature map.

4. The microbial fertilizer fermentation temperature intelligent control system based on the Internet of Things according to claim 3 is characterized in that: The feature foreground salient unit comprises: A layer normalization subunit, used for performing layer normalization processing on the fermentation growth state semantic feature map to obtain a normalized fermentation growth state semantic feature map; A deep convolutional coding subunit, used for performing channel expansion based on point convolution and deep convolutional coding based on a hole convolution layer on the normalized fermentation growth state semantic feature map to obtain a deep convolutional backup semantic feature map of the fermentation growth state and a deep convolutional original semantic feature map of the fermentation growth state; A mask generation subunit is used to input the fermentation growth state deep convolution original semantic feature map into a foreground gated mask module based on the Gelu function to obtain a fermentation growth state deep convolution gated mask weight feature map; A mask applying subunit, used for calculating the position point multiplication between the fermentation growth state deep convolution gated mask weight feature map and the fermentation growth state deep convolution backup semantic feature map to obtain a gated mask foreground highlighting fermentation growth state semantic feature map; The channel modulation subunit is used to perform point convolution-based channel contraction on the gated mask foreground-prominent fermentation growth state semantic feature map to obtain the fermentation growth state foreground-salient semantic feature map.

5. The microbial fertilizer fermentation temperature intelligent control system based on the Internet of Things according to claim 4 is characterized in that: The fermentation temperature time series feature extraction module is used to: The time series of the fermentation temperature values ​​is input into a fermentation temperature sequence encoder based on a Bi-LSTM model to obtain an implicit feature vector associated with the fermentation temperature time series.

6. The microbial fertilizer fermentation temperature intelligent control system based on the Internet of Things according to claim 5 is characterized in that: The fermentation temperature control module comprises: A cross-domain joint encoding unit, used for performing cross-domain joint encoding on the fermentation growth state foreground significant semantic feature map and the fermentation temperature time series associated implicit feature vector to obtain a fermentation temperature-growth state cross-domain joint representation feature map; A control instruction generating unit is used to input the fermentation temperature-growth state cross-domain joint representation feature map into a classifier-based fermentation temperature control module to obtain a control instruction, wherein the control instruction is used to indicate whether the fermentation temperature value at the next time point should increase, decrease or remain unchanged.

7. The microbial fertilizer fermentation temperature intelligent control system based on the Internet of Things according to claim 6 is characterized in that: The cross-domain joint encoding unit is used to: The fermentation growth state foreground significant semantic feature map and the fermentation temperature time series associated implicit feature vector are input into a cross-domain joint encoder based on auxiliary modal information to obtain the fermentation temperature-growth state cross-domain joint representation feature map.

8. The microbial fertilizer fermentation temperature intelligent control system based on the Internet of Things according to claim 7 is characterized in that: The cross-domain joint encoding unit is used to: Performing linear transformation and characteristic shape reshaping on the fermentation temperature time series associated implicit feature vector to obtain a fermentation temperature auxiliary modal modulation matrix; Performing a first linear transformation and a second linear transformation on the fermentation growth state prospect significant semantic feature map to obtain a first linear transformation fermentation growth state prospect significant semantic feature map and a second linear transformation fermentation growth state prospect significant semantic feature map; Reshaping the first linear transformation fermentation growth state foreground significant semantic feature map and the second linear transformation fermentation growth state foreground significant semantic feature map to obtain a first linear transformation fermentation growth state foreground significant semantic feature matrix and a second linear transformation fermentation growth state foreground significant semantic feature matrix; Taking the first linear transformation fermentation growth state prospect significant semantic feature matrix and the second linear transformation fermentation growth state prospect significant semantic feature matrix as a key matrix and a value matrix and taking the fermentation temperature auxiliary modal modulation matrix as a query matrix, inputting the query matrix, the key matrix and the value matrix into a Transformer structure to obtain an auxiliary modal attention query result matrix; Reshape and perform point convolution encoding on the auxiliary modality attention query result matrix to obtain an auxiliary modality attention query result feature map; The auxiliary modal attention query result feature map and the fermentation growth state foreground significant semantic feature map are calculated and multiplied by the position points to obtain the fermentation temperature-growth state cross-domain joint representation feature map.

9. A method for intelligently controlling the fermentation temperature of microbial fertilizer based on the Internet of Things, characterized in that: include: Acquire a fermentation state image captured by a camera; Performing fermentation state feature extraction and feature foreground salient processing on the fermentation state image to obtain a fermentation growth state foreground salient semantic feature map; Acquire a time series of fermentation temperature values ​​collected by a temperature sensor for a predetermined time period; Performing time series encoding on the time series of the fermentation temperature values ​​to obtain an implicit feature vector associated with the fermentation temperature time series; Based on the multimodal cross-domain joint features of the fermentation growth state prospect salient semantic feature graph and the fermentation temperature time series associated implicit feature vector, the control instruction of the fermentation temperature is determined.

10. The method for intelligently controlling the fermentation temperature of microbial fertilizer based on the Internet of Things according to claim 9, characterized in that: The fermentation state image is subjected to fermentation state feature extraction and feature foreground salient processing to obtain a fermentation growth state foreground salient semantic feature map, including: Extracting image semantic features from the fermentation state image to obtain a fermentation growth state semantic feature map; The fermentation growth state semantic feature map is input into a feature mask salient device to obtain the fermentation growth state foreground salient semantic feature map.

Images