Plant identification method and device

Abstract

The invention provides a plant identification method and device. The method comprises the following steps: constructing a local information branch, a global information branch and a prediction network, and constructing a plant identification model by using the local information branch, the global information branch and the prediction network; acquiring a training image, and inputting the training image into the plant identification model; extracting target local semantic information of the training image through the local information branches; extracting target long-distance semantic information of the training image through the global information branch; fusing the target local semantic information and the target long-distance semantic information through a prediction network, and predicting the probability that the plants in the training image are of various types based on a fusion result; and calculating a loss value by using a loss function based on the probability that the plants in the training image belong to various types, and optimizing model parameters of the plant identification model according to the loss value so as to complete training of the plant identification model.

Description

Technical Field

[0001] The present application relates to the technical field of plant identification, and in particular to a plant identification method and device. Background Art

[0002] Commonly used plant recognition methods can be divided into traditional pattern recognition methods and deep learning methods. Traditional pattern recognition methods mainly rely on manual algorithm design of image features and feature extraction for plant recognition and classification, and are heavily dependent on predefined features and classifiers. Manual work cannot handle large-scale data sets and it is difficult to select the best features. Deep learning methods are different from the differences between general objects. Plants in the same category have great similarities, resulting in a decrease in the accuracy of plant recognition. In addition, traditional deep learning models may be lacking in capturing global information, and plant images need to consider global information in recognition due to complex backgrounds, postures, and shape differences. Summary of the invention

[0003] In view of this, embodiments of the present application provide a plant identification method, device, electronic device and computer-readable storage medium to solve the problem of low plant identification accuracy due to similarity between plant image classes and lack of global information in the prior art.

[0004] In a first aspect of an embodiment of the present application, a plant identification method is provided, including: constructing a local information branch, a global information branch and a prediction network, and constructing a plant identification model using the local information branch, the global information branch and the prediction network; acquiring a training image, and inputting the training image into the plant identification model: extracting the target local semantic information of the training image through the local information branch; extracting the target long-distance semantic information of the training image through the global information branch; fusing the target local semantic information and the target long-distance semantic information through the prediction network, and predicting the probability of plants in the training image being of each species based on the fusion result; based on the probability of plants in the training image being of each species, calculating the loss value using a loss function, and optimizing the model parameters of the plant identification model according to the loss value to complete the training of the plant identification model.

[0005] According to a second aspect of an embodiment of the present application, a plant identification device is provided, comprising: a construction module, configured to construct a local information branch, a global information branch and a prediction network, and to construct a plant identification model using the local information branch, the global information branch and the prediction network; a first extraction module, configured to obtain a training image, and to input the training image into the plant identification model: to extract the target local semantic information of the training image through the local information branch; a second extraction module, configured to extract the target long-distance semantic information of the training image through the global information branch; a prediction module, configured to fuse the target local semantic information and the target long-distance semantic information through the prediction network, and to predict the probability of plants in the training image being of each species based on the fusion result; an optimization module, configured to calculate a loss value using a loss function based on the probability of plants in the training image being of each species, and to optimize the model parameters of the plant identification model according to the loss value to complete the training of the plant identification model.

[0006] According to a third aspect of an embodiment of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0007] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, which stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0008] Compared with the prior art, the embodiments of the present application have the following beneficial effects: constructing local information branches, global information branches and prediction networks, and constructing a plant recognition model using local information branches, global information branches and prediction networks; obtaining training images, and inputting the training images into the plant recognition model: extracting the target local semantic information of the training image through the local information branch; extracting the target long-distance semantic information of the training image through the global information branch; fusing the target local semantic information and the target long-distance semantic information through the prediction network, and predicting the probability of plants in the training image being of each species based on the fusion result; based on the probability of plants in the training image being of each species, calculating the loss value using the loss function, and optimizing the model parameters of the plant recognition model based on the loss value to complete the training of the plant recognition model. The above technical means can solve the problem of low plant recognition accuracy in the prior art due to the similarity between plant image classes and the lack of global information, thereby improving the accuracy of plant recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0010] Figure 1 It is a flow chart of a plant identification method provided in an embodiment of the present application;

[0011] Figure 2 is a flow chart of another plant identification method provided in an embodiment of the present application;

[0012] Figure 3 is a structural schematic diagram of a plant identification device provided in an embodiment of the present application;

[0013] Figure 4 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0014] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0015] A plant identification method and device according to an embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0016] Figure 1 It is a flow chart of a plant identification method provided in an embodiment of the present application. Figure 1 The plant identification method can be executed by a computer or a server, or software on the computer or the server. Figure 1 As shown, the plant identification method comprises:

[0017] S101, constructing a local information branch, a global information branch and a prediction network, and constructing a plant recognition model using the local information branch, the global information branch and the prediction network;

[0018] S102, obtaining a training image, and inputting the training image into a plant recognition model:

[0019] S103, extracting target local semantic information of the training image through the local information branch;

[0020] S104, extracting target long-range semantic information of the training image through the global information branch;

[0021] S105, fusing the target local semantic information and the target long-range semantic information through a prediction network, and predicting the probability of the plants in the training image being of various species based on the fusion result;

[0022] S106, based on the probability that the plants in the training image are of various types, a loss value is calculated using a loss function, and model parameters of the plant recognition model are optimized according to the loss value to complete the training of the plant recognition model.

[0023] The plants identified in the embodiment of the present application may be crops to provide support in agricultural production, and the types of crops include wheat, millet, and rice.

[0024] It is understandable that there are multiple training images, and the multiple training images are about different types of plants. The plant recognition model is trained using the training images. Within the plant recognition model: the local information branch extracts the target local semantic information of the training image, and the global information branch extracts the target long-distance semantic information of the training image (the target long-distance semantic information is the global semantic information). The prediction network fuses the target local semantic information and the target long-distance semantic information, and then predicts the probability of the plants in the training image being of each type based on the fusion result. For example, if the plant is a crop, then the prediction result is the probability that the crop is wheat, the probability that it is millet, and the probability that it is rice, etc. Based on the probability that the plant is of each type in the training image, the loss value is calculated using the loss function. Finally, the model parameters of the plant recognition model are optimized based on the loss value to complete the training of the plant recognition model.

[0025] According to the technical solution provided in the embodiment of the present application, a local information branch, a global information branch and a prediction network are constructed, and a plant recognition model is constructed using the local information branch, the global information branch and the prediction network; a training image is obtained, and the training image is input into the plant recognition model: the target local semantic information of the training image is extracted through the local information branch; the target long-distance semantic information of the training image is extracted through the global information branch; the target local semantic information and the target long-distance semantic information are fused through the prediction network, and the probability of the plant in the training image being each species is predicted based on the fusion result; based on the probability of the plant in the training image being each species, the loss value is calculated using the loss function, and the model parameters of the plant recognition model are optimized according to the loss value to complete the training of the plant recognition model. The above technical means can solve the problem of low plant recognition accuracy due to the similarity between plant image classes and the lack of global information in the prior art, thereby improving the accuracy of plant recognition.

[0026] Furthermore, the local information branch is composed of a first local network, a second local network, a third local network, a fourth local network, a fifth local network and a convolutional attention network in sequence; the first local network is composed of a convolutional layer, a normalization layer, an activation layer and a maximum pooling layer in sequence; the second local network, the third local network, the fourth local network and the fifth local network are respectively composed of a first preset number of residual blocks, a second preset number of residual blocks, a third preset number of residual blocks and a first preset number of residual blocks; wherein the first preset number is less than the second preset number, and the second preset number is less than the third preset number.

[0027] The residual block can be any commonly used residual block structure, such as ResNet34.

[0028] In one embodiment, the second local network consists of 3 residual blocks, the third local network consists of 4 residual blocks, the fourth local network consists of 6 residual blocks, and the fifth local network consists of 3 residual blocks.

[0029] The convolutional attention network can be Dy-CBAM. Dy-CBAM (Dynamic Convolutional Block Attention Module) is an attention mechanism that combines channel attention and spatial attention to enhance the feature representation capability of convolutional neural networks (CNNs). Dy-CBAM is improved based on CBAM (Convolutional Block Attention Module) and introduces the ability of dynamic adjustment, so that the model can adaptively adjust the attention weight.

[0030] The embodiment of the present application introduces Dy-CBAM to focus on local key identification information in the training image.

[0031] Dy-CBAM can also be improved: replace the ReLU activation function in the channel attention with the Dynamic ReLU activation function. The Dy-ReLU activation function consists of an auxiliary function θ(x) and an activation function Fθ(x). θ(x) is mainly used to calculate the parameters of the activation function, and Fθ(x) is mainly used to calculate the output activation function. x is the input of the activation function. Unlike the ReLU activation function, the Fθ(x) in the Dy-ReLU activation function has learnable parameters and changes with the input, which can better avoid the gradient vanishing problem and improve the recognition ability of the network.

[0032] Furthermore, the global information branch is composed of a first global network, a second global network, a third global network and a fourth global network in sequence; the first global network is composed of a slicing layer, a linear embedding layer and a Swim Transformer Block in sequence; the second global network, the third global network and the fourth global network are all composed of a downsampling layer and a Swim Transformer Block.

[0033] The slicing layer is the Patch Partition layer, the linear embedding layer is the Linear Embedding layer, and the downsampling layer is the Patch Merging layer. After each Patch Merging, the height and width of the output feature map will be halved, and the depth will be doubled.

[0034] The advantage of Swim Transformer Block is that it only performs self-attention calculations in a small window with 7×7 patches, which reduces the algorithm complexity and computational complexity. Unlike the traditional multi-head self-attention (MSA) module, SwimTransformer Block is constructed based on a shift window, that is, the traditional multi-head self-attention MSA is replaced by the window multi-head self-attention W-MSA. The information exchange of windows that originally did not communicate with each other is achieved by moving the window. The feature map is input into the SwimTransformer Block, and passes through the normalization layer, window multi-head self-attention, and multi-layer perceptron in turn, and then passes through the normalization layer, window multi-head self-attention, and multi-layer perceptron again.

[0035] Plant recognition has the characteristics of similarity between classes, complex image background and unbalanced data samples, which leads to low plant recognition accuracy. In order to improve recognition accuracy, accurately locate local important features and comprehensively consider complex global information. The embodiment of the present application combines the advantages of convolutional networks being good at extracting local features and Swim Transformer Block being good at capturing global representations, extracts the target local semantic information of the training image through the local information branch, and extracts the target long-distance semantic information of the training image through the global information branch. A convolutional attention network is embedded in the local information branch to fully extract local key features with discriminability, and a designed loss function is used to solve the problem of data sample imbalance. This fusion method can fully extract image features in plant recognition compared to a single-branch network, and its precision, recall rate and F1 score are better than existing methods.

[0036] Furthermore, extracting target local semantic information of a training image through a local information branch includes: inputting the training image into the local information branch, and within the local information branch: extracting first local semantic information of the training image through a first local network; processing the first local semantic information through a second local network to obtain second local semantic information; processing the second local semantic information through a third local network to obtain third local semantic information; processing the third local semantic information through a fourth local network to obtain fourth local semantic information; and processing the fourth local semantic information through a fifth local network to obtain target local semantic information.

[0037] The training image is input into the plant recognition model, and the training image will be processed by the local information branch. Specifically: the first local network extracts the first local semantic information of the training image, the second local network processes the first local semantic information to obtain the second local semantic information, the third local network processes the second local semantic information to obtain the third local semantic information, the fourth local network processes the third local semantic information to obtain the fourth local semantic information, and the fifth local network processes the fourth local semantic information to obtain the target local semantic information.

[0038] Furthermore, the long-distance semantic information of the training image is extracted through the global information branch, including: the training image is input into the global information branch, and inside the global information branch: the first long-distance semantic information of the training image is extracted through the first global network; the first long-distance semantic information is processed through the second global network to obtain the second long-distance semantic information; the second long-distance semantic information is processed through the third global network to obtain the third long-distance semantic information; and the third long-distance semantic information is processed through the fourth global network to obtain the target long-distance semantic information.

[0039] The training image is input into the plant recognition model, and the training image will be processed by the global information branch. Specifically: the first global network extracts the first long-distance semantic information of the training image, the second global network processes the first long-distance semantic information to obtain the second long-distance semantic information, the third global network processes the second long-distance semantic information to obtain the third long-distance semantic information, and the fourth global network processes the third long-distance semantic information to obtain the target long-distance semantic information.

[0040] Furthermore, local semantic information and long-distance semantic information are fused through a prediction network, and the probability of plants in the training image being of various species is predicted based on the fusion result, including: the prediction network is composed of a splicing layer and a classification layer in sequence, and the classification layer is composed of a fully connected layer and an activation layer; local semantic information and long-distance semantic information are input into the prediction network: the local semantic information and the long-distance semantic information are fused through the splicing layer to obtain a fusion result; the fusion result is processed through the classification layer to obtain the probability of plants in the training image being of various species.

[0041] Inside the plant recognition model: after the local information branch extracts the target local semantic information of the training image and the global information branch extracts the target long-distance semantic information of the training image, the splicing layer fuses the local semantic information and the long-distance semantic information to obtain the fusion result, and the classification layer processes the fusion result to obtain the probability of the plants in the training image being of each species.

[0042] Furthermore, based on the loss function, the loss value FL(P t ),include:

[0043] FL(p t )=-α t (1-p t ) γ log(p t )

[0044] Among them, P t is the probability that the plant is of the tth species, γ is a hyperparameter, which is used to regulate the proportion of training images with recognition difficulty lower than the threshold and training images with recognition difficulty higher than the threshold, γ∈[0,5], α t is the hyperparameter corresponding to the t-th type, α t ∈[0,1].

[0045] Figure 2 FIG. 1 is a flow chart of another plant identification method provided in an embodiment of the present application. Figure 2 As shown, the method includes:

[0046] S201, obtaining a plant image to be identified, and inputting the plant image to be identified into a plant identification model:

[0047] S202, extracting target local semantic information of the plant image to be identified through the local information branch;

[0048] S203, extracting target long-range semantic information of the plant image to be identified through the global information branch;

[0049] S204, fusing the target local semantic information and the target long-range semantic information of the plant image to be identified through a prediction network, and predicting the type of the plant in the plant image to be identified based on the fusion result.

[0050] In fact, based on the fusion result, the prediction results are the probabilities of the plants in the plant image to be identified belonging to various species. The species with the highest probability is regarded as the identified species of the plants in the plant image to be identified.

[0051] The present embodiment is the inference stage of the plant recognition model, and the previous embodiments are the training stage of the plant recognition model. The inference stage and training stage of the model are similar, and will not be repeated here.

[0052] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present application, which will not be described one by one here.

[0053] The following is an embodiment of the device of the present application, which can be used to implement the embodiment of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the present application.

[0054] Figure 3 Schematic diagram of a plant identification device provided in an embodiment of the present application. Figure 3 As shown, the plant identification device comprises:

[0055] A construction module 301 is configured to construct a local information branch, a global information branch and a prediction network, and to construct a plant recognition model using the local information branch, the global information branch and the prediction network;

[0056] The first extraction module 302 is configured to obtain a training image, input the training image into the plant recognition model: extract the target local semantic information of the training image through the local information branch;

[0057] A second extraction module 303 is configured to extract target long-range semantic information of the training image through a global information branch;

[0058] The prediction module 304 is configured to fuse the target local semantic information and the target long-range semantic information through a prediction network, and predict the probability of the plant in the training image being of each species based on the fusion result;

[0059] The optimization module 305 is configured to calculate the loss value using the loss function based on the probability of the plants in the training image being of each type, and optimize the model parameters of the plant recognition model according to the loss value to complete the training of the plant recognition model.

[0060] According to the technical solution provided in the embodiment of the present application, a local information branch, a global information branch and a prediction network are constructed, and a plant recognition model is constructed using the local information branch, the global information branch and the prediction network; a training image is obtained, and the training image is input into the plant recognition model: the target local semantic information of the training image is extracted through the local information branch; the target long-distance semantic information of the training image is extracted through the global information branch; the target local semantic information and the target long-distance semantic information are fused through the prediction network, and the probability of the plant in the training image being each species is predicted based on the fusion result; based on the probability of the plant in the training image being each species, the loss value is calculated using the loss function, and the model parameters of the plant recognition model are optimized according to the loss value to complete the training of the plant recognition model. The above technical means can solve the problem of low plant recognition accuracy due to the similarity between plant image classes and the lack of global information in the prior art, thereby improving the accuracy of plant recognition.

[0061] In some embodiments, the construction module 301 is also configured that the local information branch is composed of a first local network, a second local network, a third local network, a fourth local network, a fifth local network and a convolutional attention network in sequence; the first local network is composed of a convolutional layer, a normalization layer, an activation layer and a maximum pooling layer in sequence; the second local network, the third local network, the fourth local network and the fifth local network are respectively composed of a first preset number of residual blocks, a second preset number of residual blocks, a third preset number of residual blocks and a first preset number of residual blocks; wherein the first preset number is less than the second preset number, and the second preset number is less than the third preset number.

[0062] In some embodiments, the construction module 301 is also configured that the global information branch is composed of a first global network, a second global network, a third global network and a fourth global network in sequence; the first global network is composed of a slicing layer, a linear embedding layer and a Swim Transformer Block in sequence; the second global network, the third global network and the fourth global network are all composed of a downsampling layer and a Swim Transformer Block.

[0063] In some embodiments, the first extraction module 302 is also configured to input a local information branch for the training image, and within the local information branch: the first local semantic information of the training image is extracted through the first local network; the first local semantic information is processed through the second local network to obtain the second local semantic information; the second local semantic information is processed through the third local network to obtain the third local semantic information; the third local semantic information is processed through the fourth local network to obtain the fourth local semantic information; the fourth local semantic information is processed through the fifth local network to obtain the target local semantic information.

[0064] In some embodiments, the second extraction module 303 is also configured to input a global information branch for the training image, and within the global information branch: the first long-distance semantic information of the training image is extracted through the first global network; the first long-distance semantic information is processed through the second global network to obtain the second long-distance semantic information; the second long-distance semantic information is processed through the third global network to obtain the third long-distance semantic information; the third long-distance semantic information is processed through the fourth global network to obtain the target long-distance semantic information.

[0065] In some embodiments, the prediction module 304 is further configured as a prediction network consisting of a splicing layer and a classification layer in sequence, the classification layer consisting of a fully connected layer and an activation layer; local semantic information and long-distance semantic information are input into the prediction network: the local semantic information and the long-distance semantic information are fused through the splicing layer to obtain a fusion result; the fusion result is processed through the classification layer to obtain the probability of plants in the training image being of various species.

[0066] In some embodiments, the optimization module 305 is further configured to calculate the loss value FL(P t ),include:

[0067] FL(p t )=-α t (1-p t ) γ log(p t )

[0068] Among them, P t is the probability that the plant is of the tth species, γ is a hyperparameter, which is used to regulate the proportion of training images with recognition difficulty lower than the threshold and training images with recognition difficulty higher than the threshold, γ∈[0,5], α t is the hyperparameter corresponding to the t-th type, α t ∈[0,1].

[0069] In some embodiments, the optimization module 305 is also configured to obtain a plant image to be identified, input the plant image to be identified into a plant recognition model: extract the target local semantic information of the plant image to be identified through a local information branch; extract the target long-distance semantic information of the plant image to be identified through a global information branch; fuse the target local semantic information and the target long-distance semantic information of the plant image to be identified through a prediction network, and predict the type of plant in the plant image to be identified based on the fusion result.

[0070] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0071] Figure 4 Schematic diagram of an electronic device 4 provided in an embodiment of the present application. Figure 4 As shown, the electronic device 4 of this embodiment includes: a processor 401, a memory 402, and a computer program 403 stored in the memory 402 and executable on the processor 401. When the processor 401 executes the computer program 403, the steps in the above-mentioned method embodiments are implemented. Alternatively, when the processor 401 executes the computer program 403, the functions of the modules / units in the above-mentioned device embodiments are implemented.

[0072] The electronic device 4 may be a desktop computer, a notebook, a PDA, a cloud server, or other electronic device. The electronic device 4 may include, but is not limited to, a processor 401 and a memory 402. Those skilled in the art will appreciate that Figure 4 The electronic device 4 is merely an example and does not limit the electronic device 4 , and may include more or less components than those shown in the figure, or different components.

[0073] The processor 401 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0074] The memory 402 may be an internal storage unit of the electronic device 4, for example, a hard disk or memory of the electronic device 4. The memory 402 may also be an external storage device of the electronic device 4, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 4. The memory 402 may also include both an internal storage unit and an external storage device of the electronic device 4. The memory 402 is used to store computer programs and other programs and data required by the electronic device.

[0075] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units.

[0076] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. The computer program may include computer program code, and the computer program code may be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electric carrier signals and telecommunication signals.

[0077] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A plant identification method, characterized in that: include: Constructing a local information branch, a global information branch and a prediction network, and constructing a plant recognition model using the local information branch, the global information branch and the prediction network; Obtain a training image and input the training image into the plant recognition model: Extracting target local semantic information of the training image through the local information branch; Extracting target long-range semantic information of the training image through the global information branch; fusing the target local semantic information and the target long-range semantic information through the prediction network, and predicting the probability of the plant in the training image being of each species based on the fusion result; Based on the probability that the plants in the training image are of various types, a loss value is calculated using a loss function, and the model parameters of the plant recognition model are optimized according to the loss value to complete the training of the plant recognition model.

2. The method according to claim 1, characterized in that The local information branch is composed of a first local network, a second local network, a third local network, a fourth local network, a fifth local network and a convolutional attention network in sequence; The first local network is composed of a convolution layer, a normalization layer, an activation layer and a maximum pooling layer in sequence; The second partial network, the third partial network, the fourth partial network and the fifth partial network are respectively composed of a first preset number of residual blocks, a second preset number of residual blocks, a third preset number of residual blocks and a first preset number of residual blocks; The first preset number is smaller than the second preset number, and the second preset number is smaller than the third preset number.

3. The method according to claim 1, characterized in that The global information branch is composed of a first global network, a second global network, a third global network and a fourth global network in sequence; The first global network is composed of a slicing layer, a linear embedding layer, and a Swim Transformer Block in sequence; The second global network, the third global network and the fourth global network are all composed of a downsampling layer and a SwimTransformer Block.

4. The method according to claim 2, characterized in that: Extracting target local semantic information of the training image through the local information branch includes: The training image is input into the local information branch, and inside the local information branch: Extracting first local semantic information of the training image through the first local network; Processing the first local semantic information through the second local network to obtain second local semantic information; Processing the second local semantic information through the third local network to obtain third local semantic information; Processing the third local semantic information through the fourth local network to obtain fourth local semantic information; The fourth local semantic information is processed by the fifth local network to obtain the target local semantic information.

5. The method according to claim 3, characterized in that: Extracting long-range semantic information of the training image through the global information branch includes: The training image is input into the global information branch, and inside the global information branch: Extracting first long-distance semantic information of the training image through the first global network; Processing the first long-distance semantic information through the second global network to obtain second long-distance semantic information; Processing the second long-distance semantic information through the third global network to obtain third long-distance semantic information; The third long-distance semantic information is processed by the fourth global network to obtain the target long-distance semantic information.

6. The method according to claim 1, characterized in that The local semantic information and the long-distance semantic information are fused through the prediction network, and the probability of the plants in the training image being of various types is predicted based on the fusion result, including: The prediction network is composed of a concatenation layer and a classification layer in sequence, and the classification layer is composed of a fully connected layer and an activation layer; The local semantic information and the long-distance semantic information are input into the prediction network: fusing the local semantic information and the long-distance semantic information through the splicing layer to obtain the fusion result; The fusion result is processed by the classification layer to obtain the probability that the plants in the training image are of various types.

7. The method according to claim 1, characterized in that Based on the loss value FL(P t ),include: FL(p t )=-a t (1-p t ) γ log(p t ) Among them, P t is the probability that the plant is of the tth species, γ is a hyperparameter, which is used to regulate the proportion of training images with recognition difficulty lower than the threshold and training images with recognition difficulty higher than the threshold, γ∈[0,5], α t is the hyperparameter corresponding to the t-th type, α t ∈[0,1].

8. A plant identification device, characterized in that: include: A construction module is configured to construct a local information branch, a global information branch and a prediction network, and to construct a plant recognition model using the local information branch, the global information branch and the prediction network; A first extraction module is configured to obtain a training image, input the training image into the plant recognition model, and extract target local semantic information of the training image through the local information branch; A second extraction module is configured to extract target long-range semantic information of the training image through the global information branch; A prediction module, configured to fuse the target local semantic information and the target long-range semantic information through the prediction network, and predict the probability of the plant in the training image being of each species based on the fusion result; The optimization module is configured to calculate the loss value using the loss function based on the probability that the plants in the training image are of various types, and optimize the model parameters of the plant recognition model according to the loss value to complete the training of the plant recognition model.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images