Crop growth vigor identification method and device

Abstract

The invention provides a crop growth vigor identification method and device. The method comprises the following steps: constructing a feature cross fusion network, constructing a feature deep fusion network by using the feature cross fusion network, and sequentially connecting a splicing layer, a convolutional layer, a multi-layer sensing layer and an activation layer to obtain a classification network; adding a feature deep fusion network between corresponding stage networks in the two residual networks to obtain a residual fusion network, and connecting the residual fusion network with a classification network to obtain a crop growth identification model; obtaining a vegetation index image and a visible light image of a crop, and inputting the vegetation index image and the visible light image into a crop growth identification model: processing the vegetation index image and the visible light image through a residual fusion network to obtain a vegetation index fusion feature and a visible light fusion feature; and processing the vegetation index fusion features and the visible light fusion features through a classification network to obtain an identification result about whether the growth vigor of the crops is uniform or not.

Description

Technical Field

[0001] The present application relates to the technical field of crop yield prediction, and in particular to a method and device for identifying crop growth potential. Background Art

[0002] Judging whether crops grow evenly and then judging the quality of crop land is of great significance to improving crop yields. The existing crop growth uniformity recognition algorithm only uses single information such as normalized vegetation index images or visible light images. This method has problems such as insufficient accuracy and weak interference resistance. Visible light images lack detailed information on the size of crop leaves, and normalized vegetation index images are easily interfered by weeds. If the image features of visible light images and normalized vegetation index images are directly fused, the difference between the two features is ignored, and the fusion effect is not good. It does not improve the problems of insufficient accuracy and weak interference resistance. Summary of the invention

[0003] In view of this, the embodiments of the present application provide a method, device, electronic device and computer-readable storage medium for identifying the growth potential of crops to solve the problems of poor accuracy and weak interference resistance in the prior art in identifying the growth potential of crops.

[0004] In a first aspect of an embodiment of the present application, a method for identifying the growth of crops is provided, including: constructing a feature cross-fusion network, using the feature cross-fusion network to construct a feature deep fusion network, sequentially connecting a splicing layer, a convolution layer, a multi-layer perception layer and an activation layer to obtain a classification network; adding a feature deep fusion network between corresponding stage networks in two residual networks to obtain a residual fusion network, connecting the residual fusion network to a classification network to obtain a crop growth identification model; obtaining a vegetation index image and a visible light image of the crops, and inputting the vegetation index image and the visible light image into the crop growth identification model: processing the vegetation index image and the visible light image through the residual fusion network to obtain vegetation index fusion features and visible light fusion features; processing the vegetation index fusion features and the visible light fusion features through the classification network to obtain an identification result on whether the growth of crops is uniform.

[0005] According to a second aspect of an embodiment of the present application, a device for identifying the growth of crops is provided, comprising: a first construction module, configured to construct a feature cross-fusion network, use the feature cross-fusion network to construct a feature deep fusion network, and sequentially connect a splicing layer, a convolution layer, a multi-layer perception layer and an activation layer to obtain a classification network; a second construction module, configured to add a feature deep fusion network between corresponding stage networks in two residual networks to obtain a residual fusion network, and connect the residual fusion network to a classification network to obtain a crop growth identification model; an acquisition module, configured to acquire a vegetation index image and a visible light image of crops, and input the vegetation index image and the visible light image into the crop growth identification model; a processing module, configured to process the vegetation index image and the visible light image through the residual fusion network to obtain vegetation index fusion features and visible light fusion features; and an identification module, configured to process the vegetation index fusion features and the visible light fusion features through the classification network to obtain an identification result on whether the growth of crops is uniform.

[0006] According to a third aspect of an embodiment of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0007] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, which stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0008] Compared with the prior art, the embodiments of the present application have the following beneficial effects: constructing a feature cross-fusion network, using the feature cross-fusion network to construct a feature deep fusion network, sequentially connecting the splicing layer, the convolution layer, the multi-layer perception layer and the activation layer to obtain a classification network; adding a feature deep fusion network between the corresponding stage networks in the two residual networks to obtain a residual fusion network, and connecting the residual fusion network to the classification network to obtain a crop growth recognition model; obtaining the vegetation index image and visible light image of the crop, and inputting the vegetation index image and the visible light image into the crop growth recognition model: processing the vegetation index image and the visible light image through the residual fusion network to obtain the vegetation index fusion feature and the visible light fusion feature; processing the vegetation index fusion feature and the visible light fusion feature through the classification network to obtain the recognition result of whether the crop growth is uniform. The above-mentioned technical means can solve the problems of poor crop growth recognition accuracy and weak interference resistance in the prior art, thereby improving the crop growth recognition accuracy, improving the interference resistance, providing strong support for agricultural production, and increasing crop yields. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0010] Figure 1 It is a flow chart of a method for identifying the growth potential of crops provided in an embodiment of the present application;

[0011] Figure 2 It is a flowchart of a method for training a crop growth potential recognition model provided in an embodiment of the present application;

[0012] Figure 3 It is a structural schematic diagram of a crop growth identification device provided in an embodiment of the present application;

[0013] Figure 4 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0014] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0015] A method and device for identifying the growth potential of crops according to an embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0016] Figure 1 It is a flow chart of a method for identifying the growth potential of crops provided in an embodiment of the present application. Figure 1 The crop growth recognition method can be executed by a computer or a server, or software on the computer or the server. Figure 1 As shown, the crop growth recognition method includes:

[0017] S101, construct a feature cross fusion network, use the feature cross fusion network to construct a feature deep fusion network, and sequentially connect the splicing layer, convolution layer, multi-layer perception layer and activation layer to obtain a classification network;

[0018] S102, adding a feature deep fusion network between the corresponding stage networks in the two residual networks to obtain a residual fusion network, and connecting the residual fusion network to a classification network to obtain a crop growth recognition model;

[0019] S103, obtaining a vegetation index image and a visible light image of crops, and inputting the vegetation index image and the visible light image into a crop growth recognition model:

[0020] S104, processing the vegetation index image and the visible light image through a residual fusion network to obtain a vegetation index fusion feature and a visible light fusion feature;

[0021] S105, processing the vegetation index fusion feature and the visible light fusion feature through a classification network to obtain an identification result on whether the crop growth is uniform.

[0022] Crops can be corn, wheat, millet, rice, etc. The vegetation index image is an NDVI (Normalized Difference Vegetation Index) image, and the visible light image is an RGB (Red-Green-Blue) visible light image. The embodiment of the present application constructs a feature deep fusion network. On the basis of two residual networks, a feature deep fusion network is added between the stage networks corresponding to the two residual networks to obtain a residual fusion network. The feature deep fusion network is used to better fuse the image features of the vegetation index image and the visible light image, ultimately improving the recognition accuracy and interference resistance.

[0023] According to the technical solution provided by the embodiment of the present application, a feature cross-fusion network is constructed, and a feature deep fusion network is constructed using the feature cross-fusion network. The splicing layer, convolution layer, multi-layer perception layer and activation layer are sequentially connected to obtain a classification network; a feature deep fusion network is added between the corresponding stage networks in the two residual networks to obtain a residual fusion network, and the residual fusion network is connected to the classification network to obtain a crop growth recognition model; the vegetation index image and visible light image of the crop are obtained, and the vegetation index image and the visible light image are input into the crop growth recognition model: the vegetation index image and the visible light image are processed by the residual fusion network to obtain the vegetation index fusion feature and the visible light fusion feature; the vegetation index fusion feature and the visible light fusion feature are processed by the classification network to obtain the recognition result of whether the crop growth is uniform. The above-mentioned technical means can solve the problems of poor crop growth recognition accuracy and weak interference resistance in the prior art, thereby improving the crop growth recognition accuracy and interference resistance, providing strong support for agricultural production, and increasing crop yields.

[0024] Furthermore, a feature cross-fusion network is used to construct a feature deep fusion network, including: sequentially connecting a convolutional layer, a linear layer, a feature cross-fusion network, a dimensionality transformation layer, a convolutional layer, and an addition layer to obtain a visible light feature branch and a vegetation index feature branch; and connecting the visible light feature branch and the vegetation index feature branch in parallel to the addition layer to obtain a feature deep fusion network.

[0025] The feature cross-fusion networks in the visible light feature branch and the vegetation index feature branch are used to process the output of the linear layer in the visible light feature branch and the vegetation index feature branch. The addition layer in the visible light feature branch adds the input of the first convolutional layer and the output of the second convolutional layer in the visible light feature branch. The addition layer in the vegetation index feature branch is similar. The addition layer connected after the visible light feature branch and the vegetation index feature branch is used to add the output of the visible light feature branch and the vegetation index feature branch, and the result of the addition is used as the output of the feature deep fusion network.

[0026] Furthermore, the vegetation index image and the visible light image are processed by a residual fusion network to obtain vegetation index fusion features and visible light fusion features, including: taking the vegetation index image as the input of a residual network, and using each stage network in the residual network to process the vegetation index image in sequence, taking the visible light image as the input of another residual network, and using each stage network in the residual network to process the visible light image in sequence, and using a feature deep fusion network between the corresponding stage networks in the two residual networks to fuse the features output by the two stage networks, and optimizing the features output by the two stage networks based on the fused features.

[0027] Furthermore, the vegetation index image and the visible light image are processed by the residual fusion network to obtain the vegetation index fusion feature and the visible light fusion feature, including: the two residual networks both contain four-stage networks, the four-stage networks inside one residual network are recorded as the first-stage network, the second-stage network, the third-stage network and the fourth-stage network from front to back, and the four-stage networks inside the other residual network are recorded as the fifth-stage network, the sixth-stage network, the seventh-stage network and the eighth-stage network from front to back; the vegetation index image is processed by the first-stage network to obtain the first vegetation index feature, the visible light image is processed by the fifth-stage network to obtain the first visible light feature, the first vegetation index feature and the first visible light feature are fused by the feature deep fusion network between the first-stage network and the fifth-stage network to obtain the first fusion feature, and the first vegetation index feature and the first visible light feature are optimized by the first fusion feature; the optimized first vegetation index feature is processed by the second-stage network to obtain the second vegetation index feature, the optimized first visible light feature is processed by the sixth-stage network to obtain the second visible light feature, and the feature deep fusion network between the second-stage network and the sixth-stage network is used to obtain the first fusion feature. The second vegetation index feature and the second visible light feature are fused by the network to obtain the second fusion feature, and the second fusion feature is used to optimize the second vegetation index feature and the second visible light feature; the second vegetation index feature after the optimization is processed by the third-stage network to obtain the third vegetation index feature, the second visible light feature after the optimization is processed by the seventh-stage network to obtain the third visible light feature, the third vegetation index feature and the third visible light feature are fused by the feature deep fusion network between the third-stage network and the seventh-stage network to obtain the third fusion feature, and the third vegetation index feature and the third visible light feature are optimized by the third fusion feature; the third vegetation index feature after the optimization is processed by the fourth-stage network to obtain the fourth vegetation index feature, the third visible light feature after the optimization is processed by the eighth-stage network to obtain the fourth visible light feature, the fourth vegetation index feature and the fourth visible light feature are fused by the feature deep fusion network between the fourth-stage network and the eighth-stage network to obtain the fourth fusion feature, and the fourth vegetation index feature and the fourth visible light feature are optimized by the fourth fusion feature; the optimized fourth vegetation index feature is used as the vegetation index fusion feature, and the optimized fourth visible light feature is used as the visible light fusion feature.

[0028] For example, both residual networks use the structure of ResNet50, which includes Stage1, Stage2, Stage3 and Stage4. Stage1, Stage2, Stage3 and Stage4 in one residual network are recorded as the first stage network, the second stage network, the third stage network and the fourth stage network respectively, and Stage1, Stage2, Stage3 and Stage4 in another residual network are recorded as the fifth stage network, the sixth stage network, the seventh stage network and the eighth stage network respectively. The first stage network, the second stage network, the third stage network and the fourth stage network correspond to the fifth stage network, the sixth stage network, the seventh stage network and the eighth stage network respectively. In other words, when constructing a residual fusion network, a feature deep fusion network is added between the first stage network and the fifth stage network, a feature deep fusion network is added between the second stage network and the sixth stage network, a feature deep fusion network is added between the third stage network and the seventh stage network, and a feature deep fusion network is added between the fourth stage network and the eighth stage network.

[0029] The first fusion feature is used to optimize the first vegetation index feature and the first visible light feature, and the first fusion feature and the first vegetation index feature are added as the optimized first vegetation index feature, and the first fusion feature and the first visible light feature are added as the optimized first visible light feature. The optimization of other vegetation index features and visible light features using other fusion features is similar.

[0030] Furthermore, the first vegetation index feature and the first visible light feature are fused through a feature deep fusion network between the first stage network and the fifth stage network to obtain a first fused feature, including: in the feature deep fusion network between the first stage network and the fifth stage network: in the visible light feature branch: the first visible light feature is processed through a convolution layer and a linear layer in sequence to obtain a first visible light linear feature; the first visible light linear feature and the first vegetation index linear feature are processed through a feature cross fusion network to obtain a visible light linear fusion feature; the visible light linear fusion feature is processed through a dimensionality transformation layer and a convolution layer in sequence to obtain a first visible light convolution feature, and the first visible light convolution feature and the first visible light feature are combined through an addition layer. Add to obtain the first visible light fusion feature; in the vegetation index feature branch: process the first vegetation index feature through the convolution layer and the linear layer in turn to obtain the first vegetation index linear feature; process the first vegetation index linear feature and the first visible light linear feature through the feature cross fusion network to obtain the vegetation index linear fusion feature; process the vegetation index linear fusion feature through the dimension transformation layer and the convolution layer in turn to obtain the first vegetation index convolution feature, add the first vegetation index convolution feature and the first vegetation index feature through the addition layer to obtain the first vegetation index fusion feature; add the first visible light fusion feature and the first vegetation index fusion feature through the visible light feature branch and the addition layer after the vegetation index feature to obtain the first fusion feature.

[0031] The vegetation index features and visible light features corresponding to the other feature deep fusion networks are similar to those in the embodiment of the present application and will not be described in detail here.

[0032] Furthermore, the feature cross-fusion network processes the first visible light linear feature T0 according to the following formula: R and the first vegetation index linear characteristic T0 N , and obtain the visible light linear fusion feature T2 N ,include:

[0033] V R =T0 R *W v ,K R =T0 R *W k ,Q N =T0 N *W Q ;

[0034]

[0035] T2 N =γ*T1 N +δ*FFN(T1 N );

[0036] Among them, Wv , W k , W Q , D K It is a trainable matrix in the feature cross-fusion network, α, β, γ, δ are trainable modulation parameters in the feature cross-fusion network, T is the transposed symbol, softmax() is the activation layer, Linear() is the linear layer, and FFN() is the feedforward network.

[0037] Furthermore, the feature cross-fusion network processes the first vegetation index linear feature T0 according to the following formula: N and the first visible linear feature T0 R , and obtain the vegetation index linear fusion feature T2 R ,include:

[0038] V N =T0 N *W v ,K N =T0 N *W k ,Q R =T0 R *W Q ;

[0039]

[0040] T2 R =γ*T1 R +δ*FFN(T1 R );

[0041] Among them, W v , W k , W Q , D K It is a trainable matrix in the feature cross-fusion network, α, β, γ, δ are trainable modulation parameters in the feature cross-fusion network, T is the transposed symbol, softmax() is the activation layer, Linear() is the linear layer, and FFN() is the feedforward network.

[0042] Figure 2 1 is a flow chart of a method for training a crop growth recognition model provided in an embodiment of the present application. Figure 2 As shown, the method includes:

[0043] S201, obtaining a training vegetation index image and a training visible light image, and inputting the training vegetation index image and the training visible light image into a crop growth recognition model:

[0044] S202, processing the training vegetation index image and the training visible light image through a residual fusion network to obtain a training vegetation index fusion feature and a training visible light fusion feature;

[0045] S203, processing the training vegetation index fusion feature and the training visible light fusion feature through a classification network to obtain a training recognition result;

[0046] S204, using the mean square error loss function to calculate the loss value between the training recognition result and the training label, and optimizing the model parameters of the crop growth recognition model according to the loss value to complete the training of the crop growth recognition model.

[0047] The embodiment of the present application is the training stage of the crop growth identification model. Before the embodiment of the present application, it is the reasoning stage of the crop growth identification model. The training stage and the reasoning stage of the model are similar and will not be repeated here.

[0048] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present application, which will not be described one by one here.

[0049] The following is an embodiment of the device of the present application, which can be used to implement the embodiment of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the present application.

[0050] Figure 3 Schematic diagram of a crop growth identification device provided in an embodiment of the present application. Figure 3 As shown, the crop growth recognition device comprises:

[0051] The first construction module 301 is configured to construct a feature cross fusion network, use the feature cross fusion network to construct a feature deep fusion network, and sequentially connect the splicing layer, the convolution layer, the multi-layer perception layer and the activation layer to obtain a classification network;

[0052] The second construction module 302 is configured to add a feature deep fusion network between the corresponding stage networks in the two residual networks to obtain a residual fusion network, and connect the residual fusion network to the classification network to obtain a crop growth recognition model;

[0053] The acquisition module 303 is configured to acquire the vegetation index image and the visible light image of the crop, and input the vegetation index image and the visible light image into the crop growth recognition model:

[0054] The processing module 304 is configured to process the vegetation index image and the visible light image through a residual fusion network to obtain a vegetation index fusion feature and a visible light fusion feature;

[0055] The identification module 305 is configured to process the vegetation index fusion feature and the visible light fusion feature through a classification network to obtain an identification result on whether the growth of crops is uniform.

[0056] According to the technical solution provided by the embodiment of the present application, a feature cross-fusion network is constructed, and a feature deep fusion network is constructed using the feature cross-fusion network. The splicing layer, convolution layer, multi-layer perception layer and activation layer are sequentially connected to obtain a classification network; a feature deep fusion network is added between the corresponding stage networks in the two residual networks to obtain a residual fusion network, and the residual fusion network is connected to the classification network to obtain a crop growth recognition model; the vegetation index image and visible light image of the crop are obtained, and the vegetation index image and the visible light image are input into the crop growth recognition model: the vegetation index image and the visible light image are processed by the residual fusion network to obtain the vegetation index fusion feature and the visible light fusion feature; the vegetation index fusion feature and the visible light fusion feature are processed by the classification network to obtain the recognition result of whether the crop growth is uniform. The above-mentioned technical means can solve the problems of poor crop growth recognition accuracy and weak interference resistance in the prior art, thereby improving the crop growth recognition accuracy and interference resistance, providing strong support for agricultural production, and increasing crop yields.

[0057] In some embodiments, the first construction module 301 is also configured to sequentially connect a convolutional layer, a linear layer, a feature cross fusion network, a dimensionality transformation layer, a convolutional layer, and an addition layer to obtain a visible light feature branch and a vegetation index feature branch; and connect the visible light feature branch and the vegetation index feature branch in parallel to the addition layer to obtain a feature deep fusion network.

[0058] In some embodiments, the processing module 304 is also configured to use the vegetation index image as the input of a residual network, and use the various stage networks within the residual network to process the vegetation index image in sequence, use the visible light image as the input of another residual network, and use the various stage networks within the residual network to process the visible light image in sequence, and use the feature deep fusion network between the corresponding stage networks in the two residual networks to fuse the features output by the two stage networks, and optimize the features output by the two stage networks based on the fused features.

[0059] In some embodiments, the processing module 304 is further configured that the two residual networks each include four-stage networks, the four-stage networks within one residual network are sequentially recorded as the first-stage network, the second-stage network, the third-stage network and the fourth-stage network from front to back, and the four-stage networks within the other residual network are sequentially recorded as the fifth-stage network, the sixth-stage network, the seventh-stage network and the eighth-stage network from front to back; the vegetation index image is processed by the first-stage network to obtain the first vegetation index feature, the visible light image is processed by the fifth-stage network to obtain the first visible light feature, the first vegetation index feature and the first visible light feature are fused by the feature deep fusion network between the first-stage network and the fifth-stage network to obtain the first fused feature, and the first vegetation index feature and the first visible light feature are optimized using the first fused feature; the optimized first vegetation index feature is processed by the second-stage network to obtain the second vegetation index feature, the optimized first visible light feature is processed by the sixth-stage network to obtain the second visible light feature, and the second vegetation index feature and the second visible light feature are fused by the feature deep fusion network between the second-stage network and the sixth-stage network. The second fusion feature is obtained by processing the optimized second vegetation index feature through the third-stage network to obtain the third vegetation index feature, the second visible light feature is processed through the seventh-stage network to obtain the third visible light feature, the third vegetation index feature and the third visible light feature are fused through the feature deep fusion network between the third-stage network and the seventh-stage network to obtain the third fusion feature, and the third vegetation index feature and the third visible light feature are optimized by the third fusion feature; the third vegetation index feature after the optimization is processed through the fourth-stage network to obtain the fourth vegetation index feature, the third visible light feature after the optimization is processed through the eighth-stage network to obtain the fourth visible light feature, the fourth vegetation index feature and the fourth visible light feature are fused through the feature deep fusion network between the fourth-stage network and the eighth-stage network to obtain the fourth fusion feature, and the fourth vegetation index feature and the fourth visible light feature are optimized by the fourth fusion feature; the optimized fourth vegetation index feature is used as the vegetation index fusion feature, and the optimized fourth visible light feature is used as the visible light fusion feature.

[0060] In some embodiments, the processing module 304 is further configured to: in the feature deep fusion network between the first stage network and the fifth stage network: in the visible light feature branch: sequentially process the first visible light feature through the convolution layer and the linear layer to obtain the first visible light linear feature; process the first visible light linear feature and the first vegetation index linear feature through the feature cross fusion network to obtain the visible light linear fusion feature; sequentially process the visible light linear fusion feature through the dimension transformation layer and the convolution layer to obtain the first visible light convolution feature, add the first visible light convolution feature and the first visible light feature through the addition layer to obtain the first visible light fusion feature; in the vegetation index feature branch, the first visible light linear feature and the first vegetation index linear feature are processed through the feature cross fusion network to obtain the visible light linear fusion feature; In the branch: the first vegetation index feature is processed by the convolution layer and the linear layer in turn to obtain the first vegetation index linear feature; the first vegetation index linear feature and the first visible light linear feature are processed by the feature cross fusion network to obtain the vegetation index linear fusion feature; the vegetation index linear fusion feature is processed by the dimension transformation layer and the convolution layer in turn to obtain the first vegetation index convolution feature, and the first vegetation index convolution feature and the first vegetation index feature are added by the addition layer to obtain the first vegetation index fusion feature; the first visible light fusion feature and the first vegetation index fusion feature are added by the addition layer after the visible light feature branch and the vegetation index feature to obtain the first fusion feature.

[0061] The feature cross-fusion network processes the first visible light linear feature T0 according to the following formula R and the first vegetation index linear characteristic T0 N , and obtain the visible light linear fusion feature T2 N ,include:

[0062] V R =T0 R *W v ,K R =T0 R *W k ,Q N =T0 N *W Q ;

[0063]

[0064] T2 N =γ*T1 N +δ*FFN(T1 N );

[0065] Among them, W v , W k , W Q , D KIt is a trainable matrix in the feature cross-fusion network, α, β, γ, δ are trainable modulation parameters in the feature cross-fusion network, T is the transposed symbol, softmax() is the activation layer, Linear() is the linear layer, and FFN() is the feedforward network.

[0066] The feature cross-fusion network processes the first vegetation index linear feature T0 according to the following formula N and the first visible linear feature T0 R , and obtain the vegetation index linear fusion feature T2 R ,include:

[0067] V N =T0 N *W v ,K N =T0 N *W k ,Q R =T0 R *W Q ;

[0068]

[0069] T2 R =γ*T1 R +δ*FFN(T1 R );

[0070] Among them, W v , W k , W Q , D K It is a trainable matrix in the feature cross-fusion network, α, β, γ, δ are trainable modulation parameters in the feature cross-fusion network, T is the transposed symbol, softmax() is the activation layer, Linear() is the linear layer, and FFN() is the feedforward network.

[0071] In some embodiments, the acquisition module 303 is also configured to acquire a training vegetation index image and a training visible light image, and input the training vegetation index image and the training visible light image into a crop growth recognition model: process the training vegetation index image and the training visible light image through a residual fusion network to obtain training vegetation index fusion features and training visible light fusion features; process the training vegetation index fusion features and the training visible light fusion features through a classification network to obtain a training recognition result; use a mean square error loss function to calculate the loss value between the training recognition result and the training label, and optimize the model parameters of the crop growth recognition model based on the loss value to complete the training of the crop growth recognition model.

[0072] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0073] Figure 4 Schematic diagram of an electronic device 4 provided in an embodiment of the present application. Figure 4 As shown, the electronic device 4 of this embodiment includes: a processor 401, a memory 402, and a computer program 403 stored in the memory 402 and executable on the processor 401. When the processor 401 executes the computer program 403, the steps in the above-mentioned method embodiments are implemented. Alternatively, when the processor 401 executes the computer program 403, the functions of the modules / units in the above-mentioned device embodiments are implemented.

[0074] The electronic device 4 may be a desktop computer, a notebook, a PDA, a cloud server, or other electronic device. The electronic device 4 may include, but is not limited to, a processor 401 and a memory 402. Those skilled in the art will appreciate that Figure 4 The electronic device 4 is merely an example and does not limit the electronic device 4 , and may include more or less components than those shown in the figure, or different components.

[0075] The processor 401 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0076] The memory 402 may be an internal storage unit of the electronic device 4, for example, a hard disk or memory of the electronic device 4. The memory 402 may also be an external storage device of the electronic device 4, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 4. The memory 402 may also include both an internal storage unit and an external storage device of the electronic device 4. The memory 402 is used to store computer programs and other programs and data required by the electronic device.

[0077] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units.

[0078] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. The computer program may include computer program code, and the computer program code may be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electric carrier signals and telecommunication signals.

[0079] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for identifying the growth potential of crops, characterized in that: include: Constructing a feature cross fusion network, using the feature cross fusion network to construct a feature deep fusion network, and sequentially connecting the splicing layer, the convolution layer, the multi-layer perception layer and the activation layer to obtain a classification network; A feature deep fusion network is added between the corresponding stage networks in the two residual networks to obtain a residual fusion network, and the residual fusion network is connected to the classification network to obtain a crop growth recognition model; Obtaining a vegetation index image and a visible light image of crops, and inputting the vegetation index image and the visible light image into the crop growth recognition model: Processing the vegetation index image and the visible light image through the residual fusion network to obtain vegetation index fusion features and visible light fusion features; The classification network processes the vegetation index fusion features and the visible light fusion features to obtain an identification result on whether the growth of the crops is uniform.

2. The method according to claim 1, characterized in that The feature cross fusion network is used to construct a feature deep fusion network, including: The convolution layer, the linear layer, the feature cross fusion network, the dimension transformation layer, the convolution layer and the addition layer are sequentially connected to obtain a visible light feature branch and a vegetation index feature branch; The visible light feature branch and the vegetation index feature branch are connected in parallel to the addition layer to obtain the feature deep fusion network.

3. The method according to claim 1, characterized in that Processing the vegetation index image and the visible light image through the residual fusion network to obtain vegetation index fusion features and visible light fusion features, including: The vegetation index image is used as the input of a residual network, and the various stage networks in the residual network are used to process the vegetation index image in sequence. The visible light image is used as the input of another residual network, and the various stage networks in the residual network are used to process the visible light image in sequence. The feature deep fusion network between the corresponding stage networks in the two residual networks is used to fuse the features output by the two stage networks, and the features output by the two stage networks are optimized based on the fused features.

4. The method according to claim 2, characterized in that: Processing the vegetation index image and the visible light image through the residual fusion network to obtain vegetation index fusion features and visible light fusion features, including: Both residual networks contain four-stage networks. The four-stage networks in one residual network are recorded as the first-stage network, the second-stage network, the third-stage network and the fourth-stage network from front to back, and the four-stage networks in the other residual network are recorded as the fifth-stage network, the sixth-stage network, the seventh-stage network and the eighth-stage network from front to back. Processing the vegetation index image through the first-stage network to obtain a first vegetation index feature, processing the visible light image through the fifth-stage network to obtain a first visible light feature, fusing the first vegetation index feature and the first visible light feature through a feature deep fusion network between the first-stage network and the fifth-stage network to obtain a first fused feature, and optimizing the first vegetation index feature and the first visible light feature using the first fused feature; Processing the optimized first vegetation index feature through the second-stage network to obtain a second vegetation index feature, processing the optimized first visible light feature through the sixth-stage network to obtain a second visible light feature, fusing the second vegetation index feature and the second visible light feature through a feature deep fusion network between the second-stage network and the sixth-stage network to obtain a second fused feature, and optimizing the second vegetation index feature and the second visible light feature using the second fused feature; Processing the optimized second vegetation index feature through the third-stage network to obtain a third vegetation index feature, processing the optimized second visible light feature through the seventh-stage network to obtain a third visible light feature, fusing the third vegetation index feature and the third visible light feature through a feature deep fusion network between the third-stage network and the seventh-stage network to obtain a third fused feature, and optimizing the third vegetation index feature and the third visible light feature using the third fused feature; Processing the third vegetation index feature optimized by the fourth-stage network to obtain a fourth vegetation index feature, processing the third visible light feature optimized by the eighth-stage network to obtain a fourth visible light feature, fusing the fourth vegetation index feature and the fourth visible light feature through a feature deep fusion network between the fourth-stage network and the eighth-stage network to obtain a fourth fused feature, and optimizing the fourth vegetation index feature and the fourth visible light feature using the fourth fused feature; The optimized fourth vegetation index feature is used as the vegetation index fusion feature, and the optimized fourth visible light feature is used as the visible light fusion feature.

5. The method according to claim 4, characterized in that The first vegetation index feature and the first visible light feature are fused through a feature deep fusion network between the first stage network and the fifth stage network to obtain a first fused feature, including: In the feature deep fusion network between the first stage network and the fifth stage network: In the visible light feature branch: the first visible light feature is processed by a convolution layer and a linear layer in sequence to obtain a first visible light linear feature; the first visible light linear feature and the first vegetation index linear feature are processed by a feature cross fusion network to obtain a visible light linear fusion feature; the visible light linear fusion feature is processed by a dimensionality transformation layer and a convolution layer in sequence to obtain a first visible light convolution feature, and the first visible light convolution feature and the first visible light feature are added by an addition layer to obtain a first visible light fusion feature; In the vegetation index feature branch: the first vegetation index feature is processed by a convolution layer and a linear layer in sequence to obtain a first vegetation index linear feature; the first vegetation index linear feature and the first visible light linear feature are processed by a feature cross fusion network to obtain a vegetation index linear fusion feature; the vegetation index linear fusion feature is processed by a dimensionality transformation layer and the convolution layer in sequence to obtain a first vegetation index convolution feature, and the first vegetation index convolution feature and the first vegetation index feature are added by an addition layer to obtain a first vegetation index fusion feature; The first visible light fusion feature and the first vegetation index fusion feature are added through the addition layer after the visible light feature branch and the vegetation index feature to obtain the first fusion feature.

6. The method according to claim 5, characterized in that The feature cross-fusion network processes the first visible light linear feature T0 according to the following formula R and the first vegetation index linear characteristic T0 N , and obtain the visible light linear fusion feature T2 N ,include: V R =T0 R *W v ,K R =TO R *W k ,Q N =T0 N *W Q ; T2 N =γ*T1 N +δ*FFN(T1 N ); Among them, W v , W k , W Q , D K is a trainable matrix in the feature cross-fusion network, α, β, γ, δ are trainable modulation parameters in the feature cross-fusion network, T is the transposed symbol, softmax() is the activation layer, Linear() is the linear layer, and FFN() is the feedforward network.

7. The method according to claim 1, characterized in that Before obtaining the vegetation index image and the visible light image of the crops, the method further includes: Obtain a training vegetation index image and a training visible light image, and input the training vegetation index image and the training visible light image into the crop growth recognition model: Processing the training vegetation index image and the training visible light image through the residual fusion network to obtain a training vegetation index fusion feature and a training visible light fusion feature; The classification network processes the training vegetation index fusion feature and the training visible light fusion feature to obtain a training recognition result; The mean square error loss function is used to calculate the loss value between the training recognition result and the training label, and the model parameters of the crop growth recognition model are optimized according to the loss value to complete the training of the crop growth recognition model.

8. A device for identifying the growth of crops, characterized in that: include: The first construction module is configured to construct a feature cross fusion network, use the feature cross fusion network to construct a feature deep fusion network, and sequentially connect the splicing layer, the convolution layer, the multi-layer perception layer and the activation layer to obtain a classification network; The second construction module is configured to add a feature deep fusion network between the corresponding stage networks in the two residual networks to obtain a residual fusion network, and connect the residual fusion network to the classification network to obtain a crop growth recognition model; The acquisition module is configured to acquire a vegetation index image and a visible light image of crops, and input the vegetation index image and the visible light image into the crop growth recognition model: A processing module is configured to process the vegetation index image and the visible light image through the residual fusion network to obtain vegetation index fusion features and visible light fusion features; The recognition module is configured to process the vegetation index fusion feature and the visible light fusion feature through the classification network to obtain a recognition result on whether the growth of the crops is uniform.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images