A garlic seed sprouting method based on multimodal learning network

Abstract

The present application discloses a garlic seed sprouting correction method based on a multimodal learning network, which relates to the field of garlic sowing technology, including: capturing dynamic video frames of garlic seeds before sprouting through an image acquisition device; characterizing and identifying the garlic seeds in the dynamic video frames and judging the orientation of the garlic seed bulbs through a multimodal learning fusion network that integrates knowledge graphs and computer vision; controlling a sprout correction device to correct the sprouts of the garlic seeds based on the judged orientation of the garlic seed bulbs, ensuring that the garlic seeds can stand upright in the soil with the bulbs upward when mechanically inserted. The captured dynamic video frames of the garlic seeds are judged through a multimodal learning fusion network to judge the orientation of the garlic seed bulbs, and finally, the orientation of the garlic seeds is adjusted using the sprout correction device based on the judgment result, ensuring that the garlic seeds can stand upright in the soil with the bulbs upward.

Description

Technical Field

[0001] The present application relates to the technical field of garlic sowing, and in particular to a garlic seed sprouting method based on a multimodal learning network. Background Art

[0002] Garlic is a widely cultivated crop worldwide. Its cultivation not only has high economic value but also plays a vital role in various industries, including food and medicine. With the continuous advancement of modern agricultural mechanization, traditional garlic cultivation methods are facing challenges such as low efficiency, high labor intensity, and uneven quality. Therefore, improving the mechanization of garlic planting, particularly planting accuracy and efficiency, has become a hot topic in agricultural science and technology research.

[0003] Currently, garlic sowing relies primarily on manual sowing. However, this method is not only labor-intensive but also suffers from uneven sowing density and planting depth, which in turn impacts garlic growth and yield. To improve the efficiency and quality of sowing operations, a number of mechanized sowing and bud alignment techniques have been implemented, including bud-bump alignment and airflow sowing. While these techniques address the shortcomings of manual sowing to some extent, they still have limitations, such as insufficient precision and poor adaptability.

[0004] Airflow bud correction techniques typically use the force of airflow to propel seeds into the soil. However, due to the instability of airflow, precise positioning of garlic seeds is difficult and can easily cause seeds to scatter and be lost. Bulb collision bud correction techniques, while able to address the issue of uneven sowing depth to some extent, have high requirements for bulb morphology, making it impossible to correct buds on unpeeled garlic. Furthermore, they still face challenges such as insufficient sowing accuracy and seed damage. Therefore, existing mechanical garlic sowing and bud correction techniques still have certain drawbacks and cannot meet the precise bud correction requirements during garlic planting. Summary of the Invention

[0005] In order to solve the above technical problems, this application proposes the following technical solutions:

[0006] In a first aspect, an embodiment of the present application provides a method for garlic seed sprouting based on a multimodal learning network, comprising:

[0007] The image acquisition device is used to capture dynamic video frames of the garlic seed before orthodontic treatment;

[0008] A multimodal learning fusion network that integrates knowledge graphs and computer vision is used to characterize and identify garlic seeds in dynamic video frames and determine the orientation of garlic bulbs.

[0009] According to the result of judging the direction of garlic bulbs, the bud-correcting device is controlled to correct the buds of the garlic seeds, so as to ensure that the garlic seeds can be planted upright in the soil with bulbs facing upwards during mechanical sowing.

[0010] In a possible implementation, the capturing of dynamic video frames of garlic seeds before orthodontic treatment by an image acquisition device includes:

[0011] Use the object detection algorithm to identify the object in each frame and generate its bounding box;

[0012] For each detected target area, the image segmentation model is used to segment the target at the pixel level;

[0013] Combined with target tracking technology, it ensures the consistency of the target across multiple video frames to maintain the consistency of the target in the video;

[0014] Finally, the segmentation results are processed to remove noise and smooth edges to further improve the segmentation accuracy.

[0015] In one possible implementation, the method of characterizing and identifying garlic seeds in dynamic video frames and determining the orientation of garlic bulbs by using a multimodal learning fusion network that integrates knowledge graphs and computer vision includes:

[0016] In graph learning, the multimodal learning fusion network cleans the data set by introducing external knowledge of garlic varieties and expert evaluation results, and constructs a knowledge graph based on prior knowledge.

[0017] Based on the semantic information of the knowledge graph, a multi-degree-of-freedom point-to-point semantic space was constructed. The quality of garlic seeds and their relationship with varieties were mapped to the graph point space. A graph attention network was used to capture multi-scale features between nodes in the knowledge graph. This enhanced the model network's expressive power in spatial dimensions, enabling the multimodal learning fusion network to more accurately capture and utilize complex relationships in graph data.

[0018] Finally, the point space features converted from the knowledge graph are combined with the visual features through point multiplication, realizing the function of detecting the characteristics of garlic bulbs and judging their orientation.

[0019] In one possible implementation, the use of a graph attention network to capture multi-scale features between nodes in a knowledge graph includes:

[0020] Learning a shared weight matrix W for all nodes and implementing the self-attention mechanism for each node can obtain the attention coefficient:

[0021]

[0022] Among them: vector and They represent the characteristic vectors of the nodes respectively, and a is a predefined function;

[0023] By applying masked attention to the graph structure, we ensure that attention is only allocated to the set N of neighbor nodes of node i. i Nodes in

[0024] The SoftMax function is used to normalize the attention coefficients of all neighboring nodes j of node i. The process is shown in the formula:

[0025]

[0026] The model calculates the attention coefficients of different nodes after regularization and uses them to estimate the output feature vector of each node:

[0027]

[0028] in: is the feature vector output by the i-th node, W is the weight matrix multiplied by the feature, α is the attention coefficient obtained in the above calculation, and σ is the activation function used to introduce nonlinearity;

[0029] A multi-head attention mechanism is used to improve the stability of the self-attention learning process, and the final expression of the graph attention network is obtained:

[0030]

[0031] Among them, the model integrates K different attention mechanisms, each of which consists of a k Indicates that for each attention mechanism k attention, its specific input feature is transformed through a linear transformation, and the weight matrix of the transformation is used by the machine W k To express.

[0032] In a possible implementation, the method further includes: using stochastic gradient descent to train models for different garlic varieties and postures: the model training uses stochastic gradient descent with a momentum of 0.9, sets the number of training rounds to 200, and the weight decay rate to 10. -5 The batch size is 12, the initial learning rate is set to 0.001, the learning rate decay steps are set to 40 steps, 80 steps, 120 steps and 160 steps, and Multi-Label Soft Margin Loss is used as the loss function. The output of each image of the model is a 14-dimensional vector b = (b1, b2, ..., b14), which represents the prediction of garlic seed characterization and its influencing factors.

[0033] In one possible implementation, to evaluate the model performance and compare it with other models and the pre-optimization model, the evaluation metrics used are:

[0034]

[0035] Among them, x represents the number of correctly predicted garlic seed images, y is the total number of all garlic seed images, TP represents the number of samples in which the model correctly detects the corresponding features of garlic seeds and proposes corresponding positive measures; FP represents the number of samples that the model incorrectly judges as non-corresponding features or fails to propose corresponding positive measures; FN is the number of samples that the model incorrectly predicts as non-corresponding features or proposes non-corresponding positive measures.

[0036] In one possible implementation, the multimodal learning fusion network increases the receptive field by fusing dilated convolutions in representation learning, and introduces the CBAM module to enhance the model's response to important features by integrating channel attention and spatial attention mechanisms, thereby improving the model's ability to extract subtle features of garlic seed images.

[0037] In one possible implementation, the bud-correcting device includes: a transparent opening and closing device, a photoelectric switch arranged on one side of the garlic seed inlet of the transparent opening and closing device, the photoelectric switch is used to detect whether garlic seeds have entered the transparent opening and closing device and activate it to a controller; solenoid valves are respectively provided on both sides of the transparent opening and closing device, the solenoid valves are tilted and fixed on the device frame, and the movable ends of the solenoid valves are provided with soft brushes, which are in contact when extended together and are located directly below the garlic seed outlet of the transparent opening and closing device; an image acquisition device is provided on one side of the garlic seed outlet of the transparent opening and closing device corresponding to the photoelectric switch, the image acquisition device and the solenoid valve are both electrically connected to the controller, and the garlic seed outlet of the transparent opening and closing device corresponds vertically to the garlic seed inlet of the duckbill inserter.

[0038] In a possible implementation, the method of controlling the bud-correcting device to correct the buds of the garlic seeds according to the determined result of the garlic bud orientation to ensure that the garlic seeds can stand upright in the soil with the buds upward during mechanical sowing includes:

[0039] When the photoelectric switch detects that the garlic seeds have entered the transparent opening and closing device, the controller sends a command to the image acquisition device;

[0040] After receiving the command, the image acquisition device begins recording images of the garlic seeds and transmits them to the controller. The controller preprocesses the real-time video based on the video frames. The multimodal learning network recognizes the processed image information and determines the orientation of the garlic bulbs. The judgment result is converted into the corresponding solenoid valve switching instruction.

[0041] When the transparent opening and closing device is opened, the garlic seeds begin to fall in a horizontal position, and at the same time transmit the switch command to the corresponding solenoid valve;

[0042] After receiving the command, the solenoid valve pushes out the soft brush to contact the head of the garlic seed, changing the falling posture of the garlic seed. Guided by the inclined direction of the soft brush, the garlic seed will fall into the duckbill inserter with the bulbs facing upward, almost vertically. The garlic seed will eventually be sown into the soil with the bulbs facing vertically upward.

[0043] In an embodiment of the present application, the captured dynamic video frames of garlic seeds are judged through a multimodal learning fusion network to determine the direction of the garlic seed bulbs. Finally, the direction of the garlic seeds is adjusted using a bud correction device based on the judgment result, ensuring that the garlic seeds can stand upright in the soil with the bulbs facing upwards. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 A schematic diagram of a process for a garlic seed sprouting method based on a multimodal learning network provided in an embodiment of the present application;

[0045] Figure 2 A schematic diagram of a process for segmenting a target object image in a video frame provided in an embodiment of the present application;

[0046] Figure 3 A schematic diagram of a multimodal learning network provided in an embodiment of the present application;

[0047] Figure 4 A schematic diagram of the knowledge graph provided in the embodiment of the present application;

[0048] Figure 5 A schematic diagram of the structure of the bud-forming device provided in an embodiment of the present application; DETAILED DESCRIPTION

[0049] The present invention will be described below with reference to the accompanying drawings and specific implementation methods.

[0050] See also Figure 1 The garlic seed sprouting method based on the multimodal learning network provided in this embodiment includes:

[0051] S101, capturing dynamic video frames of garlic seeds before orthodontic treatment through an image acquisition device.

[0052] The image acquisition device communicates with the industrial computer through an Ethernet connection via a switch; Figure 2 As shown in the figure, the industrial computer uses the target detection algorithm to identify targets in real-time video at a fixed frame rate and generate their bounding boxes. For each detected target area, the image segmentation model is used to segment the target at the pixel level. Finally, the segmentation results are subjected to noise removal and edge smoothing to further improve segmentation accuracy and facilitate subsequent garlic bulb feature recognition.

[0053] S102, using a multimodal learning fusion network that integrates knowledge graphs and computer vision to characterize and identify garlic seeds in dynamic video frames and determine the orientation of garlic bulbs.

[0054] After the industrial computer completes the image preprocessing of the real-time video, the processing results are imported into the multimodal learning fusion network to detect the characteristics of garlic bulbs and make orientation judgments. Figure 3 The multimodal learning fusion network shown in Figure 1 combines natural language processing (NLP) and computer vision (CV) technologies to interact with Euclidean and non-Euclidean data, effectively overcoming the feature extraction limitations of single-data modality analysis. It integrates multiple germplasm factors and multidimensional morphological representations, providing a comprehensive and accurate method for comprehensive analysis of variety factors and fine-grained identification of garlic varieties. This method is used to identify garlic varieties and trace their influencing factors, addressing the limitations of current garlic variety identification technology, which suffers from a single identification method and low network accuracy.

[0055] The network consists of two main components: graph learning and representation learning. These components are used to identify and verify garlic varieties by integrating factors influencing germplasm. In the graph learning phase, external knowledge about garlic varieties and expert evaluation results are introduced, and the dataset is cleaned. This prior knowledge is then transformed into a knowledge graph (KG). This knowledge graph is then constructed using a multi-degree-of-freedom semantic space to map the relationship between garlic variety quality and characterization quality to a graph point space. A graph attention network (GAT) is then used to extract key features from this point space. In the representation learning phase, an improved ResNet network is employed. By incorporating dilated convolutions to increase the receptive field, this overcomes the inability of traditional convolutional neural networks to adapt to changes in the external shooting environment. The CBAM module is also used to enhance the extraction of subtle features in garlic images. Finally, the point space features transformed from the knowledge graph are combined with visual features through point multiplication to effectively identify garlic morphological features.

[0056] like Figure 4 As shown in the figure, six common garlic varieties were selected and their shape, size, and color characteristics were characterized. In graph learning, a multi-degree-of-freedom point-to-point semantic space was constructed through the knowledge graph to map the representation quality of different garlic varieties and the relationship between varieties into a graph point space. Each point represents the appearance quality characteristics of the garlic variety or the variety name, and each edge represents the internal logic and dependency relationship between entities. The number on each edge is the association rate between the different garlic varieties and the corresponding representation. The higher the association rate, the higher the probability of the corresponding representation appearing in the current garlic variety. This representation method can intuitively analyze the key morphological characteristics of different garlic varieties.

[0057] By introducing an attention mechanism using a graph attention network (GAT), the network can capture high-dimensional features between knowledge graph nodes and improve the model's sensitivity to key features of garlic seeds. By prioritizing key nodes while ignoring less important ones, the GAT significantly improves the accuracy and efficiency of the model's extraction of key information about garlic seeds. Furthermore, the parameterized weights in the attention mechanism enable the network to flexibly adjust its structure during learning and expand the spatial dimension of processing. The GAT is enhanced by integrating the attention mechanism into the graph convolutional network (GCN) architecture. Under this architecture, each node in the graph can dynamically assign different levels of attention, or weights, based on the feature representations of neighboring nodes. The graph attention layer, as the core component of the GAT, is responsible for receiving a set of node feature vectors and, through weighted processing using the attention mechanism, generating an updated set of node feature vectors. To achieve effective conversion of input and output features in this process, the system needs to learn a shared weight matrix W for all nodes. Implementing a self-attention mechanism for each node yields the following attention coefficient:

[0058]

[0059] In this process, the vector and Represent the feature vectors of nodes respectively, and a is a predefined function. By applying masked attention to the graph structure, it ensures that attention is only assigned to the neighbor node set N of node i. i The SoftMax function is used to normalize the attention coefficients of all neighboring nodes j of node i. The process is shown in the formula:

[0060]

[0061] Based on these operations, the model calculates the regularized attention coefficients of different nodes, which are then used to estimate the output feature vector of each node:

[0062]

[0063] in, is the feature vector output by the i-th node, W is the weight matrix multiplied by the feature, α is the attention coefficient obtained in the previous calculation, and σ is the activation function used to introduce nonlinearity. To improve the stability of the self-attention learning process, the model adopts a multi-head attention mechanism. In this way, the final expression of the graph attention network is obtained:

[0064]

[0065] Among them, the model integrates K different attention mechanisms, each of which consists of a k For each attention mechanism k attention, its specific input feature is transformed through a linear transformation, and the weight matrix of the transformation is represented by machine W k To express.

[0066] In representation learning, the ResNet network was improved by integrating dilated convolutions into the third and fourth residual blocks of the ResNet. This increased the receptive field and overcame the problem that traditional convolutional neural networks cannot adapt to changes in the external shooting environment. The introduction of the CBAM module strengthened the model's response to important features by integrating channel attention and spatial attention mechanisms, improving the model's ability to extract subtle features of garlic seed images.

[0067] Finally, the features obtained from the graph learning part and the representation learning part are integrated through point multiplication to accurately determine the morphological representation and variety factors of different garlic varieties. The point multiplication method effectively simulates the complex interaction between visual and text data, captures more potential information, and significantly improves the overall performance of the model.

[0068] In this embodiment, the stochastic gradient descent method is used to train the network model for different garlic varieties and different postures. The model training adopts the stochastic gradient descent method (SGD) with a momentum of 0.9, the number of training rounds is set to 200, and the weight decay rate is 10. -5 The batch size was 12. The initial learning rate was set to 0.001, and the learning rate decay steps were set to 40, 80, 120, and 160 steps. Multi-Label Soft Margin Loss was used as the loss function. The model output for each image was a 14-dimensional vector b = (b1, b2, …, b14), representing the predicted garlic seed characteristics and the factors influencing them.

[0069] In order to evaluate the performance of the model and compare it with other models and the pre-optimization model, the evaluation indicators used are:

[0070]

[0071] Here, x represents the number of correctly predicted garlic seed images, y represents the total number of all garlic seed images, TP (True Positives) represents the number of samples in which the model correctly detects the corresponding features of garlic seeds and proposes corresponding positive measures; FP (False Positives) represents the number of samples that the model mistakenly judges as non-corresponding features or fails to propose corresponding positive measures; FN (False Negatives) represents the number of samples that the model mistakenly predicts as non-corresponding features or proposes non-corresponding positive measures.

[0072] S103, controlling the bud-aligning device to perform bud-aligning operation on the garlic seeds according to the determined result of the garlic bud orientation, so as to ensure that the garlic seeds can be planted upright in the soil with the buds upwards during mechanical sowing.

[0073] See also Figure 5 In this embodiment, the bud-forming device includes: a transparent opening and closing device 1, a photoelectric switch 2 arranged on the side of the garlic seed inlet of the transparent opening and closing device 1, the photoelectric switch 2 is used to detect whether garlic seeds have entered the transparent opening and closing device 1 and activate the controller; solenoid valves 3 are respectively provided on both sides of the transparent opening and closing device 1, the solenoid valves 3 are tilted and fixed on the device frame, and the movable end of the solenoid valve 3 is provided with a soft brush 4, the soft brushes 4 are in contact when they are extended together and are located directly below the garlic seed outlet of the transparent opening and closing device 1; an image acquisition device 5 is provided on the side of the garlic seed outlet of the transparent opening and closing device 1 corresponding to the photoelectric switch 2, the image acquisition device 5 and the solenoid valve 3 are both electrically connected to the controller, and the garlic seed outlet of the transparent opening and closing device 1 is vertically corresponding to the garlic seed inlet of the duckbill inserter 6.

[0074] When the photoelectric switch 2 detects the garlic seeds entering the transparent opening and closing device 1, the controller issues a command to the image acquisition device 5. Upon receiving the command, the image acquisition device 5 begins recording images of the garlic seeds and transmits them to the controller. The controller then preprocesses the real-time video based on the video frames. The multimodal learning network then identifies the processed image information and determines the orientation of the garlic bulbs. This judgment is then converted into a corresponding opening and closing command for the solenoid valve 3. When the transparent opening and closing device 1 opens, the garlic seeds begin to fall in a horizontal position, while simultaneously transmitting an opening and closing command to the corresponding solenoid valve 3. Upon receiving the command, the solenoid valve 3 pushes out the soft brush 4, which contacts the garlic seed head, changing the seed's falling position. Guided by the inclined direction of the soft brush 4, the garlic seeds fall into the duckbill inserter 6 with the bulbs pointing upward, almost vertically. The garlic seeds are ultimately sown into the soil with the bulbs pointing vertically upward.

[0075] In the embodiments of the present application, "at least one" refers to one or more, and "more" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. Among them, A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0076] The above description is merely a specific embodiment of the present application. Any person skilled in the art may easily conceive of variations or substitutions within the technical scope disclosed in this application, and such variations or substitutions shall be within the scope of protection of this application. The scope of protection of this application shall be subject to the scope of protection of the claims.

Claims

1. A garlic seed sprouting method based on a multimodal learning network, characterized in that: include: The dynamic video frames of the garlic seeds before sprouting are captured by an image acquisition device; A multimodal learning fusion network that integrates knowledge graphs and computer vision is used to characterize and identify garlic seeds in dynamic video frames and determine the orientation of garlic bulbs. According to the result of judging the direction of garlic bulbs, the bud-correcting device is controlled to correct the buds of garlic seeds, so as to ensure that the garlic seeds can be planted upright in the soil with bulbs facing upwards during mechanical sowing. The sprouting device comprises: a transparent opening and closing device, a photoelectric switch provided on one side of the garlic seed inlet of the transparent opening and closing device, the photoelectric switch being used to detect whether garlic seeds have entered the transparent opening and closing device and to send a signal to a controller; solenoid valves are provided on both sides of the transparent opening and closing device, the solenoid valves being fixed obliquely on the device frame, the movable ends of the solenoid valves being provided with soft brushes, which, when extended together, make contact and are located directly below the garlic seed outlet of the transparent opening and closing device; an image acquisition device is provided on one side of the garlic seed outlet of the transparent opening and closing device corresponding to the photoelectric switch, the image acquisition device and the solenoid valve being electrically connected to the controller, the garlic seed outlet of the transparent opening and closing device being perpendicular to the garlic seed inlet of the duckbill sower; The method controls the bud-correcting device to correct the buds of the garlic seeds according to the result of the determined garlic bud orientation, so as to ensure that the garlic seeds can be planted upright in the soil with the buds upwards during mechanical sowing, including: When the photoelectric switch detects that the garlic seeds have entered the transparent opening and closing device, the controller sends a command to the image acquisition device; After receiving the command, the image acquisition device begins recording images of the garlic seeds and transmits them to the controller. The controller preprocesses the real-time video based on the video frames. The multimodal learning network recognizes the processed image information and determines the orientation of the garlic bulbs. The judgment result is converted into the corresponding solenoid valve switching instruction. When the transparent opening and closing device is opened, the garlic seeds begin to fall in a horizontal position, and at the same time transmit the switch command to the corresponding solenoid valve; After receiving the command, the solenoid valve pushes out the soft brush to contact the head of the garlic seed, changing the falling posture of the garlic seed. Guided by the inclined direction of the soft brush, the garlic seed will fall into the duckbill inserter with the bulbs facing upward, almost vertically. The garlic seed will eventually be sown into the soil with the bulbs facing vertically upward.

2. The garlic seed positive sprouting method based on a multimodal learning network according to claim 1, wherein The method of capturing dynamic video frames of garlic seeds before sprouting by an image acquisition device includes: Use the object detection algorithm to identify the object in each frame and generate its bounding box; For each detected target area, the image segmentation model is used to segment the target at the pixel level; Combined with target tracking technology, it ensures the consistency of the target across multiple video frames to maintain the consistency of the target in the video; Finally, the segmentation results are processed to remove noise and smooth edges to further improve the segmentation accuracy.

3. The garlic seed positive sprouting method based on multimodal learning network according to claim 1, wherein The method of characterizing and identifying garlic seeds in dynamic video frames and judging the orientation of garlic seed bulbs by using a multimodal learning fusion network that integrates knowledge graphs and computer vision includes: In graph learning, the multimodal learning fusion network cleans the data set by introducing external knowledge of garlic varieties and expert evaluation results, and constructs a knowledge graph based on prior knowledge. Based on the semantic information of the knowledge graph, a multi-degree-of-freedom point-to-point semantic space was constructed. The quality representation of garlic seeds and their relationship with varieties were mapped to the graph point space. A graph attention network was used to capture multi-scale features between nodes in the knowledge graph, enhancing the model network's expressive power in spatial dimensions. This enabled the multimodal learning fusion network to more accurately capture and utilize complex relationships in graph data. Finally, the point space features converted from the knowledge graph are combined with the visual features through point multiplication, realizing the function of detecting the characteristics of garlic bulbs and judging their orientation.

4. The garlic seed positive sprouting method based on a multimodal learning network according to claim 3, wherein The use of graph attention networks to capture multi-scale features between nodes in the knowledge graph includes: Learning a shared weight matrix W for all nodes and implementing the self-attention mechanism for each node can obtain the attention coefficient: Among them: vector and Represent the feature vectors of the nodes, is a predefined function; By applying masked attention to the graph structure, we ensure that attention is only assigned to nodes. The set of neighbor nodes Nodes in Use SoftMax function to calculate the nodes All neighbor nodes of The attention coefficient is standardized, and the process is shown in the formula: The model calculates the attention coefficients of different nodes after regularization and uses them to estimate the output feature vector of each node: in: For the The feature vector output by each node, W is the weight matrix multiplied by the feature, α is the attention coefficient obtained in the above calculation, and σ is the activation function used to introduce nonlinearity; A multi-head attention mechanism is used to improve the stability of the self-attention learning process, and the final expression of the graph attention network is obtained: The model integrates Different attention mechanisms, each of which consists of Indicates that for each attention mechanism Attention, its specific input features are transformed through a linear transformation, the weight matrix of which is used To express.

5. The garlic seed positive sprouting method based on multimodal learning network according to claim 1, wherein Also includes: The stochastic gradient descent method was used to train the model for different garlic varieties and postures: the model training used the stochastic gradient descent method with a momentum of 0.9, set the number of training rounds to 200, and the weight decay rate to 10. -5 The batch size is 12, the initial learning rate is set to 0.001, the learning rate decay steps are set to 40 steps, 80 steps, 120 steps and 160 steps, and Multi-Label Soft MarginLoss is used as the loss function. The output of each picture of the model is a 14-dimensional vector , representing the prediction of garlic seed characterization and its influencing factors.

6. The garlic seed sprouting method based on a multimodal learning network according to claim 5, wherein In order to evaluate the performance of the model and compare it with other models and the pre-optimization model, the evaluation indicators used are: Among them, x represents the number of correctly predicted garlic seed images, y is the total number of all garlic seed images, TP represents the number of samples in which the model correctly detects the corresponding features of garlic seeds and proposes corresponding positive measures; FP represents the number of samples that the model incorrectly judges as non-corresponding features or fails to propose corresponding positive measures; FN is the number of samples that the model incorrectly predicts as non-corresponding features or proposes non-corresponding positive measures.

7. The garlic seed sprouting method based on a multimodal learning network according to claim 3, wherein In representation learning, the multimodal learning fusion network increases the receptive field by fusing dilated convolutions, and introduces the CBAM module to enhance the model's response to important features by integrating channel attention and spatial attention mechanisms, thereby improving the model's ability to extract subtle features of garlic seed images.

Images