Methods for training image recognition models, image recognition methods, devices, and agricultural machinery

Abstract

This application provides a method for training an image recognition model, an image recognition method, an apparatus, and agricultural machinery, relating to the field of artificial intelligence, specifically to the field of image recognition. The method includes: acquiring N images of a land plot, where N > 1 and is a positive integer; inputting the N images into a first neural network to obtain a first input sequence; inputting the first input sequence into a transformer model to obtain prediction data from the transformer model, wherein the transformer model includes M transformer layers, where M ≥ 1 and is a positive integer; and adjusting the model parameters of the image recognition model based on the prediction data to obtain the image recognition model. This application embodiment trains multiple images simultaneously and utilizes a self-attention mechanism to associate these multiple images, improving training efficiency and image recognition accuracy.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and more specifically, to methods for training image recognition models, image recognition methods, apparatus, and agricultural machinery. Background Technology

[0002] With the continuous development of artificial intelligence (AI) technology, it plays a vital role in various fields. In agriculture, to advance no-till planting, multiple images of the entire plot can be input into a neural network to obtain the recognition result of each image. For example, each image can identify whether the plot contains straw, the type of straw, and the amount of straw. The recognition results from these multiple images are then aggregated to obtain the total amount of straw for the entire plot, allowing for no-till planting based on this result. However, in the current process, the inclusion of invalid images may affect the recognition accuracy. Therefore, improving the accuracy of plot recognition has become a pressing technical problem. Summary of the Invention

[0003] This application provides a method for training an image recognition model, an image recognition method, an apparatus, and agricultural machinery. It can simultaneously input multiple plot images into the image recognition model and use the attention mechanism of the image model to associate the multiple images. The image recognition model is a serialized structure and can simultaneously output the straw type and / or straw coverage rate corresponding to the plot images.

[0004] In a first aspect, a method for training an image recognition model is provided, characterized in that the method includes: acquiring N images of a land parcel, where N > 1 and is a positive integer; inputting the N images into a first neural network to obtain a first input sequence; inputting the first input sequence into a transformer model to obtain prediction data of the transformer model, wherein the transformer model includes M transformer layers, where M ≥ 1 and is a positive integer; and adjusting the model parameters of the image recognition model according to the prediction data to obtain the image recognition model.

[0005] The image recognition model training method provided in this application acquires features from multiple land parcel images for direct learning and associates these multiple land parcel images through a self-attention mechanism to obtain the corresponding output results. This method has high training efficiency, high robustness, and effectively avoids the problems of underfitting and overfitting of the model.

[0006] In conjunction with the first aspect, in some implementations of the first aspect, inputting the N images into the first neural network to obtain the first input sequence includes: inputting the N images into the first neural network, expanding them to obtain N tensors; and stacking the N tensors to obtain the first sequence.

[0007] In conjunction with the first aspect, in some implementations of the first aspect, the size of the N tensors is W1×H1, where W1 represents the width and H1 represents the height.

[0008] In conjunction with the first aspect, in some implementations of the first aspect, H1 = 768 and W1 = 1.

[0009] In conjunction with the first aspect, in some implementations of the first aspect, the transformer model includes M transformer layers, including:

[0010] The transformer model includes a first module, a second module, a third module, and a fourth module. The first module includes M1 transformer layers, the second module includes M2 transformer layers, the third module includes M3 transformer layers, and the fourth module includes M4 transformer layers, where M1 + M2 + M3 + M4 = M, M1 ≥ 1 and is a positive integer, M2 ≥ 1 and is a positive integer, M3 ≥ 1 and is a positive integer, M4 ≥ 1 and is a positive integer, and M ≥ 4 and is a positive integer.

[0011] Each transformer layer in the first module includes K1 self-attention heads, each transformer layer in the second module includes K2 self-attention heads, each transformer layer in the third module includes K3 self-attention heads, and each transformer layer in the fourth module includes K4 self-attention heads, where K1≥1 and is a positive integer, K2≥1 and is a positive integer, K3≥1 and is a positive integer, K4≥1 and is a positive integer, and K1<K2<K3<K4;

[0012] The first module and the second module include a first transition layer, the second module and the third module include a second transition layer, and the third module and the fourth module include a third transition layer. The first transition layer is used to increase the dimension of the output sequence of the first module, the second transition layer is used to increase the dimension of the output sequence of the second module, and the third transition layer is used to increase the dimension of the output sequence of the third module.

[0013] The image recognition model of this application can include transformer layers of multiple scales, which can effectively reduce the amount of computation.

[0014] In conjunction with the first aspect, in some implementations of the first aspect, each of the M transformer layers has the same number of self-attention heads.

[0015] In conjunction with the first aspect, in certain implementations of the first aspect, the predictive data is characterized in that the predicted data includes: the type of straw in the plot and / or the amount of straw in the plot.

[0016] Secondly, an image recognition method is provided, characterized in that the method includes: acquiring L images of a land parcel; processing the L images using an image recognition model to obtain recognition results for the L images, wherein the image recognition model is obtained by training an image recognition model, the training method including: acquiring N images of the land parcel, N>1 and being a positive integer; inputting the N images into a first neural network to obtain a first input sequence; inputting the first input sequence into a transformer model to obtain prediction data of the transformer model, wherein the transformer model includes M transformer layers, M≥1 and being a positive integer; adjusting the model parameters of the image recognition model according to the prediction data to obtain the image recognition model.

[0017] In conjunction with the second aspect, in some implementations of the second aspect, inputting the N images into the first neural network to obtain the first input sequence includes: inputting the N images into the first neural network, expanding them to obtain N tensors; and stacking the N tensors to obtain the first sequence.

[0018] In conjunction with the second aspect, in some implementations of the second aspect, the size of the N tensors is W1×H1.

[0019] In conjunction with the second aspect, in some implementations of the second aspect, H1 = 768 and W1 = 1.

[0020] In conjunction with the second aspect, in some implementations of the second aspect, the transformer model includes M transformer layers, comprising: a first module, a second module, a third module, and a fourth module; the first module includes M1 transformer layers, the second module includes M2 transformer layers, the third module includes M3 transformer layers, and the fourth module includes M4 transformer layers, where M1 + M2 + M3 + M4 = M, M1 ≥ 1 and is a positive integer, M2 ≥ 1 and is a positive integer, M3 ≥ 1 and is a positive integer, M4 ≥ 1 and is a positive integer, and M ≥ 4 and is a positive integer; each transformer layer in the first module includes K1 self-attention heads, and the second module... Each transformer layer in the first module includes K2 self-attention heads, each transformer layer in the third module includes K3 self-attention heads, and each transformer layer in the fourth module includes K4 self-attention heads, where K1≥1 and is a positive integer, K2≥1 and is a positive integer, K3≥1 and is a positive integer, K4≥1 and is a positive integer, and K1<K2<K3<K4; a first transition layer is included between the first module and the second module, a second transition layer is included between the second module and the third module, and a third transition layer is included between the third module and the fourth module, wherein the first transition layer is used to increase the dimension of the output sequence of the first module, the second transition layer is used to increase the dimension of the output sequence of the second module, and the third transition layer is used to increase the dimension of the output sequence of the third module.

[0021] In conjunction with the second aspect, in some implementations of the second aspect, each of the M transformer layers has the same number of self-attention heads.

[0022] In conjunction with the second aspect, in some implementations of the second aspect, the prediction data includes: the type of straw in the plot and / or the amount of straw in the plot.

[0023] Thirdly, a training device is provided, characterized in that the training device is mounted on agricultural machinery, and the training device includes an acquisition unit and a processing unit, wherein the acquisition unit is used to acquire N images of a plot of land; the processing unit is used to input the N images into a first neural network to obtain a first input sequence; the processing unit is also used to input the first input sequence into a transformer model to obtain prediction data of the transformer model, wherein the transformer model includes M transformer layers, M≥1, and is a positive integer; the processing unit is also used to adjust the model parameters of the image recognition model according to the prediction data to obtain the image recognition model.

[0024] In conjunction with the third aspect, in some implementations of the third aspect, the processing unit is specifically used to: input the N images into the first neural network, expand them to obtain N tensors; and stack the N tensors to obtain a first sequence.

[0025] In conjunction with the third aspect, in some implementations of the third aspect, the size of the N tensors is W1×H1.

[0026] In conjunction with the third aspect, in some implementations of the third aspect, H1 = 768 and W1 = 1.

[0027] In conjunction with the third aspect, in some implementations of the third aspect, the transformer model includes M transformer layers, namely: the transformer model includes a first module, a second module, a third module, and a fourth module; the first module includes M1 transformer layers, the second module includes M2 transformer layers, the third module includes M3 transformer layers, and the fourth module includes M4 transformer layers, where M1 + M2 + M3 + M4 = M, M1 ≥ 1 and is a positive integer, M2 ≥ 1 and is a positive integer, M3 ≥ 1 and is a positive integer, M4 ≥ 1 and is a positive integer, and M ≥ 4 and is a positive integer; each transformer layer in the first module includes K1 self-attention heads, and the second module... Each transformer layer in the first module includes K2 self-attention heads, each transformer layer in the third module includes K3 self-attention heads, and each transformer layer in the fourth module includes K4 self-attention heads, where K1≥1 and is a positive integer, K2≥1 and is a positive integer, K3≥1 and is a positive integer, K4≥1 and is a positive integer, and K1<K2<K3<K4; a first transition layer is included between the first module and the second module, a second transition layer is included between the second module and the third module, and a third transition layer is included between the third module and the fourth module, wherein the first transition layer is used to increase the dimension of the output sequence of the first module, the second transition layer is used to increase the dimension of the output sequence of the second module, and the third transition layer is used to increase the dimension of the output sequence of the third module.

[0028] In conjunction with the third aspect, in some implementations of the third aspect, each of the M transformer layers has the same number of self-attention heads.

[0029] In conjunction with the third aspect, in certain implementations of the third aspect, the predictive data is characterized in that the predicted data includes: the type of straw in the plot and / or the amount of straw in the plot.

[0030] Fourthly, an image recognition device is provided, characterized in that the training device is mounted on agricultural machinery, and the training device includes an acquisition unit and a processing unit. The acquisition unit is used to acquire L images of a plot of land; the processing unit is used to process the L images using an image recognition model to obtain recognition results for the L images. The image recognition model is obtained by training an image recognition model, and the training method includes: acquiring N images of the plot of land, where N > 1 and is a positive integer; inputting the N images into a first neural network to obtain a first input sequence; inputting the first input sequence into a transformer model to obtain prediction data for the transformer model, where the transformer model includes M transformer layers, where M ≥ 1 and is a positive integer; and adjusting the model parameters of the image recognition model based on the prediction data to obtain the image recognition model.

[0031] Fifthly, an agricultural machine is provided, comprising: a training device as provided in the third aspect or any possible implementation thereof, or an image recognition device as provided in the fourth aspect. Attached Figure Description

[0032] Figure 1 This is a schematic diagram of an image recognition system provided in an embodiment of this application.

[0033] Figure 2 This is a schematic diagram of another image recognition system provided in an embodiment of this application.

[0034] Figure 3 This is a schematic flowchart of the method for training an image recognition model provided in the embodiments of this application.

[0035] Figure 4 This is an example of a schematic flowchart of a method for training an image recognition model according to an embodiment of this application.

[0036] Figure 5 This is a schematic diagram of the structure of the transformer layer provided in the embodiments of this application.

[0037] Figure 6 This is a schematic flowchart of the image recognition method provided in the embodiments of this application.

[0038] Figure 7 This is a schematic block diagram of a training device for an image recognition model according to an embodiment of this application.

[0039] Figure 8 A schematic block diagram of an image recognition device provided in the embodiments of this application.

[0040] Figure 9The hardware structure diagram of the agricultural machinery provided in this application embodiment. Detailed Implementation

[0041] The terminology used in the following embodiments is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to also include expressions such as “one or more,” unless the context clearly indicates otherwise. It should also be understood that in the following embodiments of this application, “at least one” and “one or more” refer to one, two, or more than two. The term “and / or” is used to describe the relationship between related objects, indicating that three relationships may exist; for example, A and / or B can indicate: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character “ / ” generally indicates that the preceding and following related objects are in an “or” relationship.

[0042] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0043] Figure 1 An image recognition system is illustrated, comprising a working device and a data processing device. The working device can be a no-till planter or other smart agricultural machinery including no-till planting functionality. The working device is the initiator of the image recognition process; typically, the operator initiates the request through the working device.

[0044] The aforementioned data processing equipment can be cloud servers, network servers, application servers, management servers, or other devices or servers with data processing capabilities. The data processing equipment receives image recognition requests from the operating equipment through an interactive interface, and then performs image recognition processing using methods such as machine learning, deep learning, search, reasoning, and decision-making through a storage device and a data processing processor. The storage device's memory can be a general term, including local storage and a database storing historical data. The database can reside on the data processing equipment or on other network servers.

[0045] For example, the working device can acquire an image to be identified, and then send a request to the data processing device so that the data processing device can identify the image to be identified, thereby obtaining a classification result for the image to be identified.

[0046] For example, in Figure 1 The data processing device can execute the image recognition method of the embodiments of this application.

[0047] Figure 2 Another image recognition system is shown, in Figure 2 In this process, the working equipment directly functions as a data processing device. This equipment can directly acquire the image to be recognized and process it directly through its own hardware. The specific process is similar to... Figure 1 Similar to the description above, it will not be repeated here.

[0048] exist Figure 2 In this application, the working equipment itself can execute the image recognition method of the embodiments of this application.

[0049] Figure 1 and Figure 2 The processor in the system can perform data training / machine learning / deep learning using neural network models or other models, and then use the trained or learned models to process the input image to be recognized, thereby obtaining the image recognition result.

[0050] Since the embodiments of this application involve a large number of neural network applications, for ease of understanding, the relevant terms and concepts of neural networks that may be involved in the embodiments of this application will be introduced below.

[0051] (1) Neural Network

[0052] Neural networks can be composed of neural units, which can refer to units represented by x. s The arithmetic unit that takes an intercept of 1 as input can output the following:

[0053]

[0054] Where s = 1, 2, ..., n, n is a natural number greater than 1, W s For x sThe weights are denoted by b, where b is the bias of the neural unit. f is the activation function of the neural unit, used to introduce nonlinear characteristics into the neural network to convert the input signal in the neural unit into an output signal. The output signal of this activation function can be used as the input of the next convolutional layer; the activation function can be the sigmoid function. A neural network is a network formed by connecting multiple of the above-mentioned individual neural units together; that is, the output of one neural unit can be the input of another neural unit. The input of each neural unit can be connected to the local receptive field of the previous layer to extract the features of the local receptive field, which can be a region composed of several neural units.

[0055] (2) Deep Neural Networks

[0056] A deep neural network (DNN), also known as a multilayer neural network, can be understood as a neural network with multiple hidden layers. Based on the position of the layers, the internal neural network of a DNN can be divided into three categories: input layer, hidden layer, and output layer. Generally, the first layer is the input layer, the last layer is the output layer, and the layers in between are hidden layers. The layers are fully connected, meaning that any neuron in the i-th layer is connected to any neuron in the (i+1)-th layer.

[0057] Although DNNs seem complex, the operation of each layer is actually not complicated. Simply put, it involves the following linear relationship expression: in, It is the input vector. It is the output vector. α is the offset vector, W is the weight matrix (also called coefficients), and α() is the activation function. Each layer is simply an adjustment of the input vector. The output vector is obtained through such a simple operation. Because DNNs have many layers, the coefficients W and the offset vector... The number of these parameters is also relatively large. The definitions of these parameters in DNNs are as follows: Taking the coefficient W as an example: Assuming a three-layer DNN, the linear coefficient from the 4th neuron in the second layer to the 2nd neuron in the third layer is defined as... The superscript 3 represents the layer number where coefficient W is located, while the subscript corresponds to the third layer index 2 of the output and the second layer index 4 of the input.

[0058] In summary, the coefficient from the k-th neuron in layer L-1 to the j-th neuron in layer L is defined as...

[0059] It's important to note that the input layer does not have a W parameter. In deep neural networks, more hidden layers allow the network to better represent complex real-world situations. Theoretically, the more parameters a model has, the higher its complexity and "capacity," meaning it can perform more complex learning tasks. Training a deep neural network is essentially the process of learning the weight matrix, with the ultimate goal of obtaining the weight matrix of all layers in the trained deep neural network (a weight matrix formed by the vectors W from many layers).

[0060] (3) Convolutional Neural Network

[0061] A convolutional neural network (CNN) is a deep neural network with a convolutional structure. A CNN contains a feature extractor consisting of convolutional layers and subsampling layers, which can be viewed as a filter. A convolutional layer is a layer of neurons in a CNN that performs convolutional processing on the input signal. In a convolutional layer of a CNN, a neuron may only be connected to some of its neighboring neurons. A convolutional layer typically contains several feature planes, each composed of rectangularly arranged neural units. Neural units on the same feature plane share weights, which are called the convolutional kernel. Shared weights can be understood as the way image information is extracted regardless of location. The convolutional kernel can be formalized as a matrix of random size, and during the training process of the CNN, the kernel can learn appropriate weights. Furthermore, the direct benefit of shared weights is reducing the connections between layers in the CNN, while also reducing the risk of overfitting.

[0062] (4) Loss Function

[0063] In training a deep neural network, to ensure the output closely approximates the desired predicted value, we compare the network's prediction with the target value. Based on the difference, we update the weight vector of each layer (usually pre-configuring parameters before the initial update). For example, if the prediction is too high, the weight vector is adjusted to predict a lower value. This adjustment continues until the deep neural network predicts the target value or a value very close to it. Therefore, we need to predefine "how to compare the difference between the predicted and target values," which is the loss function or objective function. These are important equations used to measure the difference between the predicted and target values. Taking the loss function as an example, a higher output value (loss) indicates a greater difference, and training the deep neural network becomes a process of minimizing this loss.

[0064] (5) Attention mechanism

[0065] Attention mechanisms mimic the internal processes of biological observation—aligning internal experience with external senses to increase the precision of observation in specific areas. They enable the rapid sifting of high-value information from a large volume of data using limited attentional resources. Attention mechanisms can quickly extract important features from sparse data. Self-attention mechanisms, an improvement on attention mechanisms, reduce reliance on external information and are better at capturing the internal correlations of data or features. The essential idea behind attention mechanisms can be rewritten as follows:

[0066] In this formula, Lx = ||Source|| represents the length of the Source. The meaning is that the elements in the Source are imagined as a series of data pairs. Given an element Query (Q) in the Target, the similarity or relevance between the Query and each Key (K) is calculated to obtain the weight coefficient of the Value (V) corresponding to each Key, thus obtaining the final Attention value. Therefore, the Attention mechanism essentially performs a weighted sum of the Values ​​of the elements in the Source, while the Query and Key are used to calculate the weight coefficients of the corresponding Values. Conceptually, Attention can be understood as selectively filtering a small amount of important information from a large amount of information and focusing on this important information, ignoring most of the unimportant information. The focusing process is reflected in the calculation of the weight coefficients; the larger the weight, the more focused it is on its corresponding Value. That is, the weight represents the importance of the information, and the Value is the corresponding information. Self-attention can be understood as intra attention. The attention mechanism occurs between the elements of the Target (Query) and all elements of the Source. Self-attention refers to the attention mechanism that occurs between elements within the Source or between elements within the Target. It can also be understood as the attention calculation mechanism in the special case where Target = Source. The specific calculation process is the same, only the calculation object changes.

[0067] Multi-head attention mechanisms utilize multiple attention vectors (Q) to compute information from the input in parallel, with each attention focusing on a different part of the input. Multi-head attention mechanisms include multiple self-attention mechanisms. When processing data based on a self-attention mechanism, the data is first projected onto a matrix to obtain Q, K, and V. Then, Q, K, and V are processed according to the self-attention mechanism to obtain weights based on Q and K, i.e., weight = Q × K. Finally, V is weighted according to these weights, and a linear transformation is performed before outputting the result. The computation process of a self-attention mechanism includes: first, converting the input data into vectors or tensors; then obtaining three target vectors or tensors (Q, K, and V); then calculating a score, i.e., a weight, for each target vector or tensor; weight normalization is used for gradient stability; a softmax activation function is applied to the weights; the softmax dot product of the value V yields a weighted score V for each input vector or tensor; finally, the scores V of each input vector or tensor are summed to obtain the output result.

[0068] (6) Transformer model

[0069] The transformer model, also known as a transformer module or transformer structure, is a multi-layer neural network based on self-attention modules. Currently, it is primarily used for natural language processing. The transformer model mainly consists of stacked multi-head self-attention modules and feed-forward neural networks (FFNs). The transformer model can be further divided into an encoder (also called an encoding module) and a decoder (also called a decoding module), whose structures are generally similar but also have some differences.

[0070] (6) Vision Transformer (ViT) Model

[0071] The ViT model combines computer vision and natural language processing techniques. It divides the input image into blocks, flattens them into a sequence, and feeds them into the encoder part of the transformer model. Then, it classifies the image by connecting a fully connected layer.

[0072] In agriculture, to promote no-till planting, multiple images of the entire plot can be input into a neural network to obtain the recognition result of each image. For example, each image can identify whether the plot contains straw, the type of straw, and the amount of straw. The recognition results from these multiple images are then aggregated to obtain the total amount of straw for the entire plot, allowing for no-till planting. However, in the current process, the inclusion of invalid images may affect the recognition accuracy. Therefore, improving the accuracy of plot recognition has become a pressing technical problem. Based on this, this application provides a method for training an image recognition model, an image recognition method, and an apparatus.

[0073] Figure 3 A schematic flowchart of a method 300 for training an image recognition model according to an embodiment of this application is shown, such as... Figure 3 As shown, method 300 includes:

[0074] S301, Obtain N images of the land parcel.

[0075] Specifically, when training the image recognition model, N images of the land parcel can be obtained, and these N images can constitute the image of the entire land parcel. In other words, each of the N images corresponds to a part of the entire land parcel.

[0076] It is understandable that these N images are already labeled.

[0077] For example, the images can be labeled according to the type of straw in them. For instance, if the N images include a first image and a second image, where the straw in the first image is corn straw and the straw in the second image is soybean straw, then the straw in the first image and the straw in the second image can be labeled as corn straw and soybean straw, respectively.

[0078] For example, the straw coverage rate in the images can be labeled. For instance, if the N images include a first image and a second image, where the straw coverage rate of the first image is 20% and the straw coverage rate of the second image is 50%, then the images can be labeled as having a straw coverage rate of 20% for the first image and 50% for the second image.

[0079] For example, images can be labeled based on the type and coverage rate of straw. For instance, if the N images include a first image and a second image, where the straw in the first image is corn straw with a coverage rate of 20%, and the straw in the second image is soybean straw with a coverage rate of 50%, then the images can be labeled as follows: the straw in the first image is corn straw with a coverage rate of 20%, and the straw in the second image is soybean straw with a coverage rate of 50%.

[0080] For example, images can be labeled based on the type of straw and the straw coverage level. For instance, if the N images include a first image and a second image, where the straw in the first image is corn straw with a coverage rate of 20% (level 1), and the straw in the second image is soybean straw with a coverage rate of 50%, then the first image can be labeled as corn straw with a level 1 coverage. Similarly, if the second image contains soybean straw with a coverage rate of 50% (level 3), then the second image can be labeled as soybean straw with a level 3 coverage.

[0081] Optionally, in some embodiments, the N images may be data-enhanced images.

[0082] In training image recognition models, data augmentation can be performed on the collected training set images to improve the model's robustness and generalization ability. Data augmentation can be understood as making limited data produce value equivalent to more data without substantially increasing the amount of data.

[0083] It should be understood that the data augmentation method is not limited in the embodiments of this application. For example, one or more data augmentation methods such as flipping, rotating, scaling, cropping, shifting, noise, interpolation, blurring, color transformation, erasing, and filling can be adopted in the embodiments of this application.

[0084] S302, input the N images into the first neural network to obtain the first input sequence.

[0085] Specifically, by inputting the N images into the first neural network, a first input sequence can be obtained.

[0086] Figure 4 A schematic flowchart illustrating an example of a method for training an image recognition model according to an embodiment of this application is shown, such as... Figure 4 As shown, the N images are input into the first neural network to obtain the first input sequence, which includes:

[0087] S3021, input the N images into the first neural network to obtain N tensors.

[0088] Specifically, by inputting the N images into the first neural network, N tensors are obtained.

[0089] For example, the first neural network can be a convolutional neural network. Each of the N images can be represented using a tensor. By inputting the N images into the convolutional neural network, the size of the tensor of each of the N images can be transformed.

[0090] For example, the N images include a first image, which can be represented using a tensor. The size of the first image can be understood as the size of the tensor, and thus, the size of the first image can be understood as a tensor of size (C, W, H), where C, H, and W represent the number of color channels, width, and height of the first image, respectively. The size of the tensor of the first image can also be represented as C × W × H. Inputting the first image into a convolutional neural network allows for feature extraction from the (C, W, H)-sized first image, resulting in a first tensor of size (W1, H1). This first tensor can be understood as an image feature of the first image.

[0091] It should be noted that, in this embodiment of the application, the size of the N images is not limited to be the same.

[0092] In some embodiments, each of the N images is the same size. For example, the N images include a first image and a second image, both of which have a size of C×W×H.

[0093] In other embodiments, the N images include at least two different image sizes. For example, the N images include a first image and a second image, where the first image has a size of C×W×H and the second image has a size of C×W²×H². It is understood that W≠W² and H≠H².

[0094] It should also be noted that inputting these N images into a convolutional neural network will yield tensors of the same size.

[0095] For example, if the first image and the second image are of size C×W×H, inputting the first image and the second image into a convolutional neural network can produce two tensors of size (W1,H1).

[0096] For example, if the size of the first image is C×W×H and the size of the second image is C×W2×H2, inputting the first image and the second image into a convolutional neural network can also produce two tensors of size (W1,H1).

[0097] Optionally, in some embodiments, W1 = 1 and H1 = 768.

[0098] S3022, stack the N tensors to obtain the first input sequence.

[0099] Specifically, in S3022, N tensors can be obtained, and then these N tensors are stacked to obtain the first input sequence.

[0100] For example, unfolding 256 images yields 256 tensors of size (W1, H1), namely Z1, Z2, Z3, Z4, Z5... Z. 256 Stacking the above 256 tensors yields the first input sequence, which can be denoted as Z = [Z1; Z2; Z3; Z4; Z5 ...; Z...]. 256 ].

[0101] Optionally, in some embodiments, the first input sequence may also include position encoding.

[0102] Understandably, in S301-S302, N images are serialized to be input into the transformer model.

[0103] S303, input the first input sequence into the transformer model to obtain the prediction data of the transformer model.

[0104] Specifically, after obtaining the first input sequence, a transformer model can be used as an encoder to extract features from N images and finally output the prediction data. The transformer model includes M transformer layers, where M > 1 and is a positive integer.

[0105] The transformer model in this embodiment includes M transformer layers, where the output of the previous transformer layer is the input of the next transformer layer. A transformer layer can also be called a transformer module, transformer block, etc. Figure 5 The diagram shown is a schematic of the transformer layer structure. Figure 5 As shown, the transformer layer includes a multi-head attention module and a multi-layer perceptron (MLP) module spaced apart, with a layer normalization module positioned between the multi-head attention module and the MLP module. The multi-head attention module includes multiple self-attention heads.

[0106] The transformer model in this application embodiment includes a first output head, which is used to output the straw coverage rate.

[0107] Optionally, in some embodiments, the transformer model also includes a second output header for outputting the straw type.

[0108] Optionally, in some embodiments, each transformer layer in the transformer model has the same structure, that is, the number of self-attention heads in the multi-head attention module of each transformer layer is the same.

[0109] For example, the transformer model in this application embodiment includes 12 transformer layers, and the multi-head attention module of each transformer layer includes 8 self-attention heads.

[0110] The first input sequence is fed into the first transformer layer of the transformer model. After processing by the layer normalization module, a normalized input sequence is obtained. This normalized input sequence is then fed into the multi-head attention module for feature extraction, yielding attention features. These attention features are then fed into the layer normalization module, and the normalized result is fed into the multilayer perceptron module to obtain the output features of the first transformer layer. The output features of the first transformer layer are used as the input sequence for the second transformer layer, resulting in the output features of the second transformer layer. This process is repeated to obtain the predicted data.

[0111] One possible implementation is that the predicted data can be the straw coverage rate corresponding to the N images.

[0112] For example, the entire plot consists of 3 images, i.e., N=3. The predicted data is that the straw coverage rate is 20% in the first image, 20% in the second image, and 40% in the third image, so the straw coverage rate of the plot is 30%.

[0113] One possible implementation is that the prediction data can be the straw coverage rate and straw type corresponding to the N images.

[0114] For example, the entire plot consists of 3 images, i.e., N=3. The predicted data is that the straw coverage rate of the first image is 20%, and the straw type is soybean; the straw coverage rate of the second image is 20%, and the straw type is soybean; the straw coverage rate of the third image is 40%, and the straw type is corn; the straw coverage rate of the plot is 30%, and the straw types are corn and soybean.

[0115] The following describes the computation process of inputting the first input sequence into the transformer model. Taking the first input sequence as Z and inputting it into the first transformer layer as an example, the first input sequence passes through the layer normalization module of the first transformer layer, which can be represented by Equation 1:

[0116] Z' 1-l =LN(Z) (1)

[0117] Z' 1-l As input to the multi-head attention module, this module consists of h self-attention heads. The input to each self-attention head is three matrices Q, P, and V. The calculation formulas for Q, P, and V are shown in Formula 2, and the calculation process for each self-attention head is shown in Formula 3.

[0118] Q = Z' 1-l W Q P = Z' 1-l W P V = Z' 1-l W V (2)

[0119] SA(Q,P,V)=softmax(f(Q,P,V)) (3)

[0120] The calculation process of the multi-head attention module is shown in Equation 4:

[0121] MSA(Z' 1-l )=concat(SA1,SA1,SA1...SA h W 0 (4)

[0122] Where concat represents merging the output matrices of h self-attention heads into a single matrix, W 0 This represents the weighting parameter.

[0123] After obtaining the output of the multi-head attention module, the output is input into the layer standard module, as shown in Equation 5:

[0124] Z”=LN(MSA(Z' 1-l (5)

[0125] Inputting Z” into the multilayer perceptron module can be represented using Equation 6:

[0126] Z1 = MLP(Z”) (6)

[0127] After obtaining the input Z1 of the first transformer layer, Z1 is fed into the second transformer layer to obtain the output of the second transformer layer. In this way, the predicted data is obtained.

[0128] Optionally, in other embodiments, the transformer model includes a first module, a second module, a third module, and a fourth module. The first module includes M1 transformer layers, the second module includes M2 transformer layers, the third module includes M3 transformer layers, and the fourth module includes M4 transformer layers, wherein M1 + M2 + M3 + M4 = M, M1 ≥ 1 and is a positive integer, M2 ≥ 1 and is a positive integer, M3 ≥ 1 and is a positive integer, M4 ≥ 1 and is a positive integer, M ≥ 4 and is a positive integer, and at least two of M1, M2, M3, and M4 are unequal. A first transition layer is included between the first module and the second module, a second transition layer is included between the second module and the third module, and a fourth transition layer is included between the third module and the fourth module. The first transition layer is used to increase the dimension of the output sequence of the first module, the second transition layer is used to increase the dimension of the output sequence of the second module, and the third transition layer is used to increase the dimension of the output sequence of the third module. Each transformer layer in the first module includes K self-attention heads, each transformer layer in the second module includes K self-attention heads, each transformer layer in the third module includes K self-attention heads, and each transformer layer in the fourth module includes K self-attention heads, where K > 1 and is a positive integer.

[0129] For example, the transformer model includes a first module, a second module, a third module, and a fourth module. The first module includes 8 transformer layers, the second module includes 8 transformer layers, the third module includes 12 transformer layers, and the fourth module includes 16 transformer layers. Each transformer layer in the first module includes 8 self-attention heads, each transformer layer in the second module includes 8 self-attention heads, each transformer layer in the third module includes 8 self-attention heads, and each transformer layer in the fourth module includes 8 self-attention heads.

[0130] The first input sequence is fed into the transformer model, which consists of four modules, with transition layers between the four modules to increase the dimensionality of the output sequence. The first input sequence is processed sequentially through the first module, the first transition layer, the second module, the second transition layer, the third module, the third transition layer, and the fourth module, resulting in a high-dimensional output with richer spatial details, which helps improve recognition accuracy.

[0131] It should be understood that the description of inputting the first input sequence into the transformer model can be found above, and will not be repeated here for the sake of brevity.

[0132] Optionally, in other embodiments, the transformer model includes a first module, a second module, a third module, and a fourth module. The first module includes M1 transformer layers, the second module includes M1 transformer layers, the third module includes M1 transformer layers, and the fourth module includes M1 transformer layers, where 4 × M1 = M, M1 ≥ 1 and is a positive integer, and M ≥ 4 and is a positive integer. A first transition layer is included between the first and second modules, a second transition layer is included between the second and third modules, and a fourth transition layer is included between the third and fourth modules. The first transition layer is used to increase the dimension of the output sequence of the first module, the second transition layer is used to increase the dimension of the output sequence of the second module, and the third transition layer is used to increase the dimension of the output sequence of the third module. The first module includes K1 self-attention heads, the second module includes K2 self-attention heads, the third module includes K3 self-attention heads, and the fourth module includes K4 self-attention heads. Among them, at least two of K1, K2, K3, and K4 are not equal, K1 ≥ 1 and is a positive integer, K2 ≥ 1 and is a positive integer, K3 ≥ 1 and is a positive integer, and K4 ≥ 1 and is a positive integer.

[0133] For example, the transformer model includes a first module, a second module, a third module, and a fourth module. The first module includes 8 transformer layers, the second module includes 8 transformer layers, the third module includes 8 transformer layers, and the fourth module includes 8 transformer layers. Each transformer layer in the first module includes 4 self-attention heads, each transformer layer in the second module includes 8 self-attention heads, each transformer layer in the third module includes 8 self-attention heads, and each transformer layer in the fourth module includes 16 self-attention heads.

[0134] Furthermore, in these embodiments, K1 < K2 < K3 < K4.

[0135] Increasing the number of self-attention heads in a transformer layer can improve the performance of an image recognition model. To match the increased self-attention heads, a transition layer can be used to increase the dimensionality of the output sequence, thus ensuring the dimensionality of each self-attention head. For example, if the first input sequence has a dimension of 768, and each transformer layer in the first module has 4 self-attention heads, then each self-attention head has a dimension of 192. Assuming each transformer layer in the second module has 8 self-attention heads, to ensure that the dimensionality of each self-attention head in the second module is the same as that in the first module, the first transition layer needs to increase the dimensionality of the output sequence of the first module from 768 to 1536. Similarly, as the number of self-attention heads in each transformer layer of the third and fourth modules increases, the second and third transition layers can respectively increase the dimensionality of the output sequence of the second output module, and the third transition layer can increase the dimensionality of the output sequence of the third module.

[0136] It should be understood that the description of inputting the first input sequence into the transformer model can be found above, and will not be repeated here for the sake of brevity.

[0137] Optionally, the transformer model includes a first module, a second module, a third module, and a fourth module. The first module includes M1 transformer layers, the second module includes M2 transformer layers, the third module includes M3 transformer layers, and the fourth module includes M4 transformer layers. M1 + M2 + M3 + M4 = M, M1 ≥ 1 and is a positive integer, M2 ≥ 1 and is a positive integer, M3 ≥ 1 and is a positive integer, M4 ≥ 1 and is a positive integer, M ≥ 4 and is a positive integer, and at least two of M1, M2, M3, and M4 are unequal. A first transition layer connects the first and second modules, a second transition layer connects the second and third modules, and a fourth transition layer connects the third and fourth modules. The first transition layer increases the dimensionality of the output sequence of the first module, the second transition layer increases the dimensionality of the output sequence of the second module, and the third transition layer increases the dimensionality of the output sequence of the third module. The first module includes K1 self-attention heads, the second module includes K2 self-attention heads, the third module includes K3 self-attention heads, and the fourth module includes K4 self-attention heads. Among them, at least two of K1, K2, K3, and K4 are not equal, K1 ≥ 1 and is a positive integer, K2 ≥ 1 and is a positive integer, K3 ≥ 1 and is a positive integer, and K4 ≥ 1 and is a positive integer.

[0138] For example, the transformer model includes a first module, a second module, a third module, and a fourth module. The first module includes 2 transformer layers, the second module includes 2 transformer layers, the third module includes 18 transformer layers, and the fourth module includes 2 transformer layers. Each transformer layer in the first module includes 4 self-attention heads, each transformer layer in the second module includes 8 self-attention heads, each transformer layer in the third module includes 16 self-attention heads, and each transformer layer in the fourth module includes 8 self-attention heads.

[0139] Furthermore, in these embodiments, K1 < K2 < K3 < K4.

[0140] It should be understood that the descriptions of the transition layer, self-attention head, and inputting the first input sequence into the transformer model can be found above, and will not be repeated here for the sake of brevity.

[0141] S304. Adjust the model parameters of the image recognition model based on the predicted data to obtain the image recognition model.

[0142] In this embodiment, multiple images can be input into a transformer model to obtain prediction data. Then, loss parameters are constructed based on the prediction data to balance the difference between the predicted value and the target value. The higher the output value of the loss function, the greater the difference. Therefore, the model parameters of the image recognition model can be adjusted to reduce the output value of the loss function.

[0143] In some embodiments, the loss function may be as shown in formula (7), comprising three parts.

[0144] L total =α1L cls +α2L reg +α3L exp (7)

[0145] Where L total This represents the loss function of the image recognition model in the embodiments of this application;

[0146] L cls Represents the classification loss function;

[0147] L reg Represents the quantization loss function;

[0148] L exp Represents the expected loss function;

[0149] α1, α2, and α3 are hyperparameters that balance the loss functions of each component.

[0150] Hyperparameters can be understood as defining the structure of a model, its optimization strategy, or controlling its operational state. Hyperparameters can optimize the loss function, thereby ensuring the model avoids underfitting and overfitting. Common hyperparameters include the number of layers in a neural network and the kernel function. Hyperparameters can be selected in combination; however, this application does not limit the selection of hyperparameters, which can be either of the two types mentioned above or other hyperparameters.

[0151] The loss function of each part of the image text recognition model will be introduced below.

[0152] In some embodiments, the classification loss function can be constructed according to formula (8), and the cross-entropy loss function is used when constructing the classification loss function.

[0153]

[0154] Where N represents the number of images input to the image recognition model;

[0155] w i Indicates hyperparameters;

[0156] c i Indicates the classification target value;

[0157] This represents the predicted value for classification.

[0158] In some embodiments, the quantization loss function can be constructed according to formula (9), and the average error function is used when constructing the classification loss function.

[0159]

[0160] Where N represents the number of images input to the image recognition model;

[0161] s i Indicates the quantified target value;

[0162] This represents the predicted value for classification.

[0163] In some embodiments, the expected loss function can be constructed according to formula (10).

[0164]

[0165] By continuously training, the output value of the loss function is reduced, and the final training result is output, which includes the trained image recognition model. The training result can also include the image recognition model's processing results on a portion of the training data, as well as highlighted markers for the parts of each training data set that have the greatest impact on the processing result. For example, pixels in the training data image that have the greatest impact on the processing result can be highlighted to emphasize them.

[0166] Based on the highlighted markers of the parts of each training data point that have the greatest impact on the processing results, the reasons affecting the accuracy of the trained neural network model can be manually determined. These reasons may include, for example, poor training data, and / or the need for further optimization of the hyperparameters used in training.

[0167] The image recognition model training method provided in this application acquires features from multiple land parcel images for direct learning and associates these multiple land parcel images through a self-attention mechanism to obtain the corresponding output results. This method has high training efficiency, high robustness, and effectively avoids the problems of underfitting and overfitting of the model.

[0168] Furthermore, the image recognition model of this application may include transformer layers of multiple scales, which can effectively reduce the amount of computation.

[0169] Figure 6 A schematic flowchart of the image recognition method 600 provided in an embodiment of this application is shown. Figure 6 The method shown can be executed by image recognition equipment, server equipment, or agricultural machinery. The process includes:

[0170] Method 600 includes steps S601 to S602, which are described in detail below.

[0171] S601, acquire L images of the land parcel.

[0172] Optionally, the agricultural machinery may include a camera, which can acquire L images of the plot; or, the L images may be acquired from a data storage system, such as images stored in the data storage system, or images may be acquired from the cloud. In this embodiment, the method of acquiring the L images is not limited.

[0173] S602, use an image recognition model to process L images to obtain the recognition results of L images.

[0174] The image text recognition model can be obtained through method 300, which will not be elaborated here.

[0175] The following is combined Figures 7 to 8 The apparatus of the embodiments of this application will be described below. It should be understood that the apparatus described below is capable of performing the methods of the foregoing embodiments of this application. To avoid unnecessary repetition, repeated descriptions will be appropriately omitted when describing the apparatus of the embodiments of this application below.

[0176] Figure 7 This is a schematic block diagram of a training device for an image recognition model according to an embodiment of this application. Figure 7 The training device 700 shown includes an acquisition unit 710 and a processing unit 720.

[0177] The acquisition unit 710 and the processing unit 720 can be used to execute the method of training an image recognition model according to the embodiments of this application, specifically, they can be used to execute method 300.

[0178] Acquisition unit 710 is used to acquire N images of the land parcel.

[0179] The processing unit 720 is used to input N images into the first neural network to obtain the first input sequence.

[0180] The processing unit 720 is also used to input the first input sequence into the transformer model to obtain the prediction data of the transformer model.

[0181] The processing unit 720 is also used to adjust the model parameters of the image recognition model based on the prediction data to obtain the image recognition model.

[0182] It should be noted that all relevant content of each step involved in the above method embodiments can be referenced from the functional description of the corresponding functional module, and will not be repeated here. The image recognition model training device provided in this application embodiment is used to execute the above method for training the image recognition model, and therefore can achieve the same effect as described above.

[0183] Figure 8 This is a schematic block diagram of the image recognition device provided in the embodiments of this application. Figure 8 The apparatus 800 shown includes an acquisition unit 810 and a processing unit 820.

[0184] The acquisition unit 810 and the processing unit 820 can be used to execute the image recognition method of the embodiments of this application, for example, they can be used to execute method 600.

[0185] The acquisition unit 810 is used to acquire L images of the land parcel.

[0186] The processing unit 820 is used to process L images using an image recognition model to obtain recognition results.

[0187] The image recognition model can be obtained through method 300, which will not be elaborated here.

[0188] It should be noted that the training devices 700 and 800 described above are embodied in the form of functional units. The term "unit" here can be implemented in software and / or hardware, and there is no specific limitation on this.

[0189] For example, a "unit" can be a software program, a hardware circuit, or a combination of both that implements the above functions. The hardware circuit may include an application-specific integrated circuit (ASIC), electronic circuitry, a processor (e.g., a shared processor, a proprietary processor, or a group processor) and memory for executing one or more software or firmware programs, integrated logic circuitry, and / or other suitable components that support the described functions.

[0190] Therefore, the units of the various examples described in the embodiments of this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0191] Figure 9 This is a schematic diagram of the hardware structure of the agricultural machinery provided in the embodiments of this application. Figure 9The agricultural machinery 900 shown includes a memory 901, a processor 902, a camera 903, a communication interface 904, and a bus 905. The memory 901, processor 902, camera 903, and communication interface 904 are interconnected via the bus 905.

[0192] The memory 901 can be a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 901 can store programs. When the program stored in the memory 901 is executed by the processor 902, the processor 902 executes various steps of the method for training an image recognition model or various steps of the image recognition method according to the embodiments of this application. Specifically, the processor 902 can execute the steps described above... Figure 3 Method 300 or shown Figure 6 Method 600 is shown.

[0193] The processor 902 may be a general-purpose central processing unit (CPU), microprocessor, application-specific integrated circuit (ASIC), graphics processing unit (GPU), or one or more integrated circuits, used to execute relevant programs to implement the method of training an image recognition model according to the method embodiments of this application.

[0194] The processor 902 described above can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory 901, and processor 902 reads the information in memory 901 and combines it with its hardware to complete... Figure 7 The training device shown includes the functions that the units are required to perform.

[0195] The communication interface 904 uses transceiver devices, such as, but not limited to, transceivers, to enable communication between the agricultural machinery 900 and other devices or communication networks.

[0196] Bus 905 may include a pathway for transmitting information between various components of device 900 (e.g., memory 901, processor 902, camera 903, communication interface 904).

[0197] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0198] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0199] In this application, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0200] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0201] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0202] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0203] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0204] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0205] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0206] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0207] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for training an image recognition model, the method comprising: The method comprises: Obtaining N images of a first land plot, N>1 and being a positive integer, the N images being used to construct an overall image of the first land plot, each of the N images corresponding to a part of the first land plot; Inputting the N images into a first neural network to obtain a first input sequence; Inputting the first input sequence into a converter model to obtain predicted data of the converter model, wherein the converter model comprises M converter layers, M≥1 and being a positive integer; Adjusting model parameters of the image recognition model according to the predicted data to obtain the image recognition model; The converter model comprises a first module, a second module, a third module and a fourth module, the first module comprises M1 converter layers, the second module comprises M2 converter layers, the third module comprises M3 converter layers, and the fourth module comprises M4 converter layers, wherein M1+M2+M3+M4=M, M1≥1 and being a positive integer, M2≥1 and being a positive integer, M3≥1 and being a positive integer, M4≥1 and being a positive integer, and M≥4 and being a positive integer; Each converter layer in the first module comprises K1 self-attention heads, each converter layer in the second module comprises K2 self-attention heads, each converter layer in the third module comprises K3 self-attention heads, and each converter layer in the fourth module comprises K4 self-attention heads, wherein K1≥1 and being a positive integer, K2≥1 and being a positive integer, K3≥1 and being a positive integer, K4≥1 and being a positive integer, and K1 The first transition layer is used to increase the dimension of the output sequence of the first module, the second transition layer is used to increase the dimension of the output sequence of the second module, and the third transition layer is used to increase the dimension of the output sequence of the third module.

2. The method of claim 1, wherein, The method comprises: Inputting the N images into the first neural network to obtain N tensors; Stacking the N tensors to obtain the first input sequence.

3. The method of claim 2, wherein, The size of the N tensors is W1×H1, wherein W1 represents the width and H1 represents the height.

4. The method of claim 3, wherein, H1=768, W1=1.

5. The method of claim 4, wherein, The number of self-attention heads of each converter layer of the M converter layers is the same.

6. The method according to any one of claims 1 to 5, characterized in that, The predicted data comprises: The straw type of the land plot and / or the straw amount of the land plot.

7. An image recognition method characterized by, The method comprises: Obtaining L images of a second land plot; The L images are processed by using an image recognition model to obtain recognition results of the L images, wherein the image recognition model is obtained by a method for training an image recognition model, and the method for training the image recognition model comprises: acquiring N images of a first land plot, N > 1 and N is a positive integer, the N images are used to construct a whole image of the first land plot, and each of the N images corresponds to a part of the first land plot; inputting the N images into a first neural network to obtain a first input sequence; inputting the first input sequence into a converter model to obtain prediction data of the converter model, wherein the converter model comprises M converter layers, M ≥ 1 and M is a positive integer; and adjusting model parameters of the image recognition model according to the prediction data to obtain the image recognition model. The converter model comprises a first module, a second module, a third module and a fourth module, each of the first module, the second module, the third module and the fourth module comprises a plurality of converter layers, and the converter layers in the first module, the second module, the third module and the fourth module are different in number. Each of the converter layers in the first module comprises K1 self-attention heads, each of the converter layers in the second module comprises K2 self-attention heads, each of the converter layers in the third module comprises K3 self-attention heads, and each of the converter layers in the fourth module comprises K4 self-attention heads, wherein K1 ≥ 1 and K1 is a positive integer, K2 ≥ 1 and K2 is a positive integer, K3 ≥ 1 and K3 is a positive integer, K4 ≥ 1 and K4 is a positive integer, and K1 < K2 < K3 < K4. The first transition layer is arranged between the first module and the second module, the second transition layer is arranged between the second module and the third module, and the third transition layer is arranged between the third module and the fourth module.

8. The method of claim 7, wherein, The N images are input into the first neural network to obtain N tensors. The N tensors are stacked to obtain the first input sequence. The size of the N tensors is W1 × H1.

9. The method of claim 8, wherein, H1 = 768 and W1 = 1.

10. The method of claim 8, wherein, The number of self-attention heads of each of the M converter layers is the same.

11. The method of claim 10, wherein, The prediction data comprises:

12. The method according to any one of claims 7 to 11, characterized in that, The straw type of the land plot and / or the straw amount of the land plot. The training device is arranged on an agricultural machine, and the training device is used to train the image recognition model.

13. A training device, characterized by The training device comprises an acquisition unit and a processing unit. The acquisition unit is configured to acquire N images of a first land plot, N>1 and being a positive integer, and the N images are used to construct an overall image of the first land plot, and each of the N images corresponds to a part of the first land plot. The processing unit is configured to input the N images into a first neural network to obtain a first input sequence. The processing unit is further configured to input the first input sequence into a converter model to obtain predicted data of the converter model, wherein the converter model comprises M converter layers, M≥1 and being a positive integer. The processing unit is further configured to adjust model parameters of the image recognition model according to the predicted data to obtain the image recognition model. The converter model comprises a first module, a second module, a third module and a fourth module, the first module comprises M1 converter layers, the second module comprises M2 converter layers, the third module comprises M3 converter layers, and the fourth module comprises M4 converter layers, wherein M1+M2+M3+M4=M, M1≥1 and being a positive integer, M2≥1 and being a positive integer, M3≥1 and being a positive integer, M4≥1 and being a positive integer, and M≥4 and being a positive integer. Each converter layer in the first module comprises K1 self-attention heads, each converter layer in the second module comprises K2 self-attention heads, each converter layer in the third module comprises K3 self-attention heads, and each converter layer in the fourth module comprises K4 self-attention heads, wherein K1≥1 and being a positive integer, K2≥1 and being a positive integer, K3≥1 and being a positive integer, K4≥1 and being a positive integer, and K1 The first transition layer is used to increase the dimension of the output sequence of the first module, the second transition layer is used to increase the dimension of the output sequence of the second module, and the third transition layer is used to increase the dimension of the output sequence of the third module.

14. The exercise device of claim 13, wherein, The processing unit is specifically configured to: input the N images into the first neural network to obtain N tensors; stack the N tensors to obtain the first input sequence.

15. The exercise device of claim 14, wherein, The size of the N tensors is W1×H1.

16. The exercise device of claim 15, wherein, H1=768 and W1=1.

17. The exercise device of claim 16, wherein, The number of self-attention heads of each converter layer of the M converter layers is the same.

18. The training device of any one of claims 13 to 17, wherein, The predicted data comprises: the straw type of the land plot and / or the straw amount of the land plot.

19. An image recognition apparatus characterized by comprising: The image recognition device is on a farm machine, and the image recognition device comprises an acquisition unit and a processing unit, wherein the acquisition unit is configured to acquire L images of a second land plot. The processing unit is configured to process the L images by using an image recognition model to obtain recognition results of the L images, wherein the image recognition model is obtained by a method for training an image recognition model, and the method for training an image recognition model comprises the following steps: acquiring N images of a first land plot, wherein N>1 and N is a positive integer, and the N images are used to construct an overall image of the first land plot, and each of the N images corresponds to a part of the first land plot; inputting the N images into a first neural network to obtain a first input sequence; inputting the first input sequence into a converter model to obtain prediction data of the converter model, wherein the converter model comprises M converter layers, M≥1 and M is a positive integer; and adjusting model parameters of the image recognition model according to the prediction data to obtain the image recognition model. The converter model comprises a first module, a second module, a third module and a fourth module, each of the first module, the second module, the third module and the fourth module comprises one or more converter layers, and the converter layers in the first module, the second module, the third module and the fourth module are different in number. Each of the converter layers in the first module comprises K1 self-attention heads, each of the converter layers in the second module comprises K2 self-attention heads, each of the converter layers in the third module comprises K3 self-attention heads, and each of the converter layers in the fourth module comprises K4 self-attention heads, wherein K1, K2, K3 and K4 are positive integers, K1 The first module and the second module comprise a first transition layer, the second module and the third module comprise a second transition layer, and the third module and the fourth module comprise a third transition layer.

20. An agricultural machine, comprising: The training device according to any one of claims 13 to 18 or the image recognition device according to claim 19. The agricultural machine comprises one or more processors, one or more memories, and one or more computer programs stored in the one or more memories, wherein the one or more computer programs comprise instructions which, when executed by the one or more processors, cause the method according to any one of claims 1 to 6 or the method according to any one of claims 7 to 12 to be performed.

21. An agricultural machine, comprising: ​

Images