A highly integrated multi-source heterogeneous agricultural condition online monitoring method and device

Through image recognition and processing of AI/ML models and combined with deep learning algorithms, the time-consuming and labor-intensive and artificial error problems of fruit maturity monitoring in traditional agriculture are solved, and the automatic and accurate judgment of fruit maturity is realized, and the picking efficiency is improved. and monitoring accuracy.

CN118521903BActive Publication Date: 2024-12-03GUANGZHOU HAIRUI INFORMATION TECH CO LTD
View PDF 11 Cites 0 Cited by

Patent Information

Application Number
CN202410985871.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-23
Publication Date
2024-12-03
Estimated Expiration
2044-07-23

AI Technical Summary

Technical Problem

The existing fruit maturity monitoring relies on manual observation, which is time-consuming and labor-intensive and is prone to fruit loss due to human error. It is difficult to achieve efficient and accurate fruit maturity judgment in traditional agriculture.

Method used

The AI/ML model is used to automatically judge the maturity of the fruit through image recognition and processing, combined with deep learning algorithms, and build a prediction model through historical data and environmental factors to achieve real-time monitoring and self-optimization.

Benefits of technology

It realizes automated and accurate judgment of fruit maturity, improves picking efficiency, reduces labor costs, and improves monitoring accuracy and real-time through the deep integration of AI/ML models.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118521903B_ABST
    Figure CN118521903B_ABST
Patent Text Reader

Abstract

The present application provides a highly integrated multi-source heterogeneous agricultural online monitoring method and device, which belongs to the field of image processing technology and is used to realize the full-process automated tracking and monitoring of the growth cycle of agricultural products through AI / ML models. The method includes: the electronic device obtains M crop images collected at M times for the same crop, where M is an integer greater than or equal to 3; the electronic device processes the M crop images through the AI / ML model to obtain the growth information of the crop in each of the M crop images, and obtains a total of M growth information of the crop; the electronic device determines the growth status of the crop based on the M growth information of the crop, wherein the growth status of the crop includes at least one of the following: the disease status of the crop, the male and female flower status of the crop, the growth stage status of the crop, or the fruit status of the crop.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a highly integrated multi-source heterogeneous agricultural condition online monitoring method and device. Background Art

[0002] With the rapid development of science and technology, artificial intelligence (AI) and machine learning (ML) have penetrated into various industries, and agriculture is no exception. The introduction of these advanced technologies is changing the production model of traditional agriculture, especially in the fruit maturity monitoring and picking process, bringing significant efficiency improvement and cost reduction.

[0003] In traditional agriculture, the judgment of fruit maturity usually relies on manual observation, which is not only time-consuming and labor-intensive, but may also lead to fruit loss due to human error. The application of AI and ML technology has effectively solved this problem. Through deep learning models, computers can learn and understand the characteristics of fruit maturity, such as color changes, shape changes, etc., thereby achieving automated and accurate fruit maturity judgment.

[0004] Specifically, image recognition and processing: The AI ​​system captures fruit images through a camera and then uses deep learning algorithms to analyze them. These algorithms are able to identify the ripening characteristics of the fruit, such as color, texture, size, etc., after learning from a large amount of training data, to determine whether the fruit is ripe. Data model construction: The machine learning model constructs a model to predict the maturity of the fruit by analyzing historical data, such as past records of fruit maturity, as well as environmental factors such as climate and soil. This model can predict the maturity state of the fruit in the future and provide a basis for decision-making for picking. Real-time feedback and adjustment: The AI ​​system can not only monitor the maturity of the fruit in real time, but also adjust and optimize itself according to the actual situation. For example, if the maturity judgment of a certain fruit does not meet expectations, the system will learn and adjust the model to improve the accuracy of future predictions.

[0005] In practical applications, intelligent fruit monitoring and picking systems have been used in many farms. For example, some agricultural technology companies in China have developed similar systems that use drones or ground equipment to achieve real-time monitoring of the maturity of fruits in large tracts of farmland, greatly improving picking efficiency and reducing labor costs. However, in the future, how to further integrate it with AI and ML technologies is a hot topic in current research. Summary of the invention

[0006] The embodiments of the present application provide a highly integrated multi-source heterogeneous agricultural condition online monitoring method and device, which are used to realize full-process automated tracking and monitoring of the growth cycle of agricultural products through AI / ML models.

[0007] In order to achieve the above objectives, this application adopts the following technical solutions:

[0008] In a first aspect, a highly integrated multi-source heterogeneous agricultural condition online monitoring method is provided, which is applied to electronic devices, and the method includes: the electronic device obtains M crop images collected at M times for the same crop, where M is an integer greater than or equal to 3; the electronic device processes the M crop images through an AI / ML model to obtain growth information of the crop in each of the M crop images, and obtains M growth information of the crop in total; the electronic device determines the growth condition of the crop based on the M growth information of the crop, wherein the growth condition of the crop includes at least one of the following: the disease condition of the crop, the condition of male and female flowers of the crop, the growth stage condition of the crop, or the fruit condition of the crop.

[0009] Optionally, the AI / ML model includes a feature extraction layer and four feature processing layers, the four feature processing layers are respectively open to the corresponding interfaces of the AI / ML models, some neuron nodes of the four feature processing layers are the same, and the four feature processing layers include feature processing layer #1, feature processing layer #2, feature processing layer #3 and feature processing layer #4; if the M crop images contain the branches and leaves of the crops, the feature extraction layer extracts M feature information #1 from the M crop images one by one, and the feature extraction layer calls the interface of the feature processing layer #1 to input the M feature information #1 into the feature processing layer #1 for processing in sequence, so as to obtain the diseased area of ​​the crops in each of the M crop images output by the feature processing layer #1; if the M crop images contain the flower and branch parts of the crops, the feature extraction layer extracts M feature information #2 from the M crop images one by one, and the feature extraction layer calls the interface of the feature processing layer #2 to The M feature information #2 are sequentially input into the feature processing layer #2 for processing, and the number of male and female flowers of the crops in each of the M crop images output by the feature processing layer #2 is obtained; if the M crop images contain branches and leaves of the crops, the feature extraction layer extracts M feature information #3 one by one from the M crop images, and the feature extraction layer inputs the M feature information #3 sequentially into the feature processing layer #3 for processing by calling the interface of the feature processing layer #3, and the growth stage of the crops in each of the M crop images output by the feature processing layer #3 is obtained; if the M crop images contain fruit parts of the crops, the feature extraction layer extracts M feature information #4 one by one from the M crop images, and the feature extraction layer inputs the M feature information #4 sequentially into the feature processing layer #3 for processing by calling the interface of the feature processing layer #4, and the fruit size of the crops in each of the M crop images output by the feature processing layer #4 is obtained.

[0010] Optionally, feature processing layer #1, feature processing layer #2, feature processing layer #3 and feature processing layer #4 each include K common part neurons, K is an integer greater than 2, and the structure of each common part neuron in the K common part neurons is , M is an integer greater than 1, The matrix includes weight parameter #1 and weight parameter #2. Matrix: Weight parameter #1 is located at On the diagonal of the matrix, weight parameter #1 is located Before AI / ML model training, The initial value of the weight parameter #1 in the matrix is ​​the reciprocal of the weight parameter #2; the training of the AI / ML model includes the first stage and the second stage; in the first stage, the feature extraction layer of the AI / ML model is only configured with K common part neurons, and the electronic device uses a training image set containing the branches and leaves of crops, a training image set containing the flower branches of crops, and a training image set containing the fruit parts of crops to train the AI / ML model to adjust the values ​​of the weight parameters #1 and the weight parameters #2 in each of the K common part neurons; after the first stage, the second stage is entered; in the second stage, the electronic device copies the K common part neurons in the feature extraction layer to 4 copies, and constructs 4 feature processing layers one by one on the basis of the 4 copies, and the electronic device uses the training and test image sets containing the branches and leaves of crops to train and test the feature processing Layer #1, and during the training process of feature processing layer #1, the K common part neurons contained in feature processing layer #1 are not regressed, the electronic device uses the training and test image sets containing the flower and branch parts of crops to train and test feature processing layer #2, and during the training process of feature processing layer #2, the K common part neurons contained in feature processing layer #2 are not regressed, the electronic device uses the training and test image sets containing the branches and leaves of crops to train and test feature processing layer #3, and during the training process of feature processing layer #3, the K common part neurons contained in feature processing layer #3 are not regressed, the electronic device uses the training and test image sets containing the fruit part of crops to train and test feature processing layer #4, and during the training process of feature processing layer #4, the K common part neurons contained in feature processing layer #4 are not regressed.

[0011] Optionally, any common part neuron among the K common part neurons is connected to all other common part neurons among the common part neurons except the common part neuron; the construction of feature processing layer #1 includes: configuring L1 non-common part neurons connected only to some of the common part neurons in the periphery among the K common part neurons, L1 is an integer greater than 2, and the structure of each non-common part neuron among the L1 non-common part neurons is A matrix, N is an integer greater than 1 and less than M, The matrix includes weight parameter #1 and weight parameter #2. Matrix: Weight parameter #1 is located at On the diagonal of the matrix, weight parameter #1 is located The matrix is ​​located at other positions except the diagonal; the construction of feature processing layer #2 includes: configuring L2 non-public part neurons that are only connected to some public part neurons in the periphery of the K public part neurons, L2 is an integer greater than 2, and the structure of each non-public part neuron in the L1 non-public part neurons is The matrix of feature processing layer #2 before training, The initial value of the weight parameter #1 in the matrix is ​​the reciprocal of the initial value of the weight parameter #2; the construction of the feature processing layer #3 includes: configuring L3 non-public part neurons that are only connected to some public part neurons in the periphery of the K public part neurons, L3 is an integer greater than 2, and the structure of each non-public part neuron in the L3 non-public part neurons is The construction of feature processing layer #2 includes: configuring L4 non-public part neurons connected only to some public part neurons in the periphery of the K public part neurons, L4 is an integer greater than 2, and the structure of each non-public part neuron in the L4 non-public part neurons is ; before training the 4 feature processing layers, In the matrix, the initial value of weight parameter #1 and the initial value of weight parameter #2 are reciprocals of each other.

[0012] Optionally, with respect to the growth condition of crops being the disease condition of crops, the electronic device processes M crop images through an AI / ML model to obtain growth information of crops in each of the M crop images, including: the electronic device processes the M crop images through an AI / ML model to obtain diseased areas of crops in each of the M crop images; the electronic device determines the growth condition of crops based on the M growth information of crops, including: the electronic device determines the severity of crop diseases by analyzing size changes of the same diseased area in at least some of the M crop images, and the number of diseased areas in the M crop images.

[0013] Optionally, with respect to the growth condition of crops being the number of male and female flowers of crops, the electronic device processes M crop images through an AI / ML model to obtain growth information of crops in each of the M crop images, including: the electronic device processes M crop images through an AI / ML model to obtain the number of male and female flowers of crops in each of the M crop images; the electronic device determines the growth condition of crops based on the M growth information of crops, including: the electronic device determines a change in the number of male and female flowers of crops based on the number of male and female flowers of crops in each of the M crop images.

[0014] Optionally, with respect to the growth condition of crops being the growth stage of crops, the electronic device processes M crop images through an AI / ML model to obtain growth information of crops in each of the M crop images, including: the electronic device processes M crop images through an AI / ML model to obtain the growth stage of crops in each of the M crop images.

[0015] Optionally, with respect to the growth condition of crops being the fruit condition of crops, the electronic device processes M crop images through an AI / ML model to obtain growth information of crops in each of the M crop images, including: the electronic device processes the M crop images through an AI / ML model to obtain the fruit size of crops in each of the M crop images; the electronic device determines the growth condition of crops based on the M growth information of crops, including: the electronic device determines the fruit growth rate of crops based on the fruit size of crops in each of the M crop images.

[0016] In a second aspect, a highly integrated multi-source heterogeneous agricultural condition online monitoring device is provided, characterized in that, when applied to an electronic device, the device is configured as follows: the electronic device obtains M crop images collected at M times for the same crop, where M is an integer greater than or equal to 3; the electronic device processes the M crop images through an AI / ML model to obtain growth information of the crop in each of the M crop images, and obtains a total of M growth information of the crop; the electronic device determines the growth condition of the crop based on the M growth information of the crop, wherein the growth condition of the crop includes at least one of the following: the disease condition of the crop, the condition of male and female flowers of the crop, the growth stage condition of the crop, or the fruit condition of the crop.

[0017] Optionally, the AI / ML model includes a feature extraction layer and four feature processing layers, the four feature processing layers are respectively open to the corresponding interfaces of the AI / ML models, some neuron nodes of the four feature processing layers are the same, and the four feature processing layers include feature processing layer #1, feature processing layer #2, feature processing layer #3 and feature processing layer #4; if the M crop images contain the branches and leaves of the crops, the feature extraction layer extracts M feature information #1 from the M crop images one by one, and the feature extraction layer calls the interface of the feature processing layer #1 to input the M feature information #1 into the feature processing layer #1 for processing in sequence, so as to obtain the diseased area of ​​the crops in each of the M crop images output by the feature processing layer #1; if the M crop images contain the flower and branch parts of the crops, the feature extraction layer extracts M feature information #2 from the M crop images one by one, and the feature extraction layer calls the interface of the feature processing layer #2 to The M feature information #2 are sequentially input into the feature processing layer #2 for processing, and the number of male and female flowers of the crops in each of the M crop images output by the feature processing layer #2 is obtained; if the M crop images contain branches and leaves of the crops, the feature extraction layer extracts M feature information #3 one by one from the M crop images, and the feature extraction layer inputs the M feature information #3 sequentially into the feature processing layer #3 for processing by calling the interface of the feature processing layer #3, and the growth stage of the crops in each of the M crop images output by the feature processing layer #3 is obtained; if the M crop images contain fruit parts of the crops, the feature extraction layer extracts M feature information #4 one by one from the M crop images, and the feature extraction layer inputs the M feature information #4 sequentially into the feature processing layer #3 for processing by calling the interface of the feature processing layer #4, and the fruit size of the crops in each of the M crop images output by the feature processing layer #4 is obtained.

[0018] Optionally, feature processing layer #1, feature processing layer #2, feature processing layer #3 and feature processing layer #4 each include K common part neurons, K is an integer greater than 2, and the structure of each common part neuron in the K common part neurons is , M is an integer greater than 1, The matrix includes weight parameter #1 and weight parameter #2. Matrix: Weight parameter #1 is located at On the diagonal of the matrix, weight parameter #1 is located At other positions of the matrix except the diagonal, before the AI / ML model is trained, the initial value of the weight parameter #1 in the matrix is ​​the reciprocal of the weight parameter #2; the training of the AI / ML model includes the first stage and the second stage; in the first stage, the feature extraction layer of the AI / ML model is only configured with K common part neurons, and the electronic device uses a training image set containing the branches and leaves of crops, a training image set containing the flower branches of crops, and a training image set containing the fruit parts of crops to train the AI / ML model to adjust the values ​​of the weight parameters #1 and the weight parameters #2 in each of the K common part neurons; after the first stage, the second stage is entered; in the second stage, the electronic device copies the K common part neurons in the feature extraction layer to 4 copies, and constructs 4 feature processing layers one by one on the basis of the 4 copies, and the electronic device uses the training and The test image set is used to train and test feature processing layer #1, and during the training of feature processing layer #1, the K common part neurons contained in feature processing layer #1 are not regressed. The electronic device uses the training and test image sets containing the flower and branch parts of crops to train and test feature processing layer #2, and during the training of feature processing layer #2, the K common part neurons contained in feature processing layer #2 are not regressed. The electronic device uses the training and test image sets containing the branches and leaves of crops to train and test feature processing layer #3, and during the training of feature processing layer #3, the K common part neurons contained in feature processing layer #3 are not regressed. The electronic device uses the training and test image sets containing the fruit parts of crops to train and test feature processing layer #4, and during the training of feature processing layer #4, the K common part neurons contained in feature processing layer #4 are not regressed.

[0019] Optionally, any common part neuron among the K common part neurons is connected to all other common part neurons among the common part neurons except the common part neuron; the construction of feature processing layer #1 includes: configuring L1 non-common part neurons connected only to some of the common part neurons in the periphery among the K common part neurons, L1 is an integer greater than 2, and the structure of each non-common part neuron among the L1 non-common part neurons is A matrix, N is an integer greater than 1 and less than M, The matrix includes weight parameter #1 and weight parameter #2. Matrix: Weight parameter #1 is located at On the diagonal of the matrix, weight parameter #1 is located The matrix is ​​located at other positions except the diagonal; the construction of feature processing layer #2 includes: configuring L2 non-public part neurons that are only connected to some public part neurons in the periphery of the K public part neurons, L2 is an integer greater than 2, and the structure of each non-public part neuron in the L1 non-public part neurons is The matrix of feature processing layer #2 before training, The initial value of the weight parameter #1 in the matrix is ​​the reciprocal of the initial value of the weight parameter #2; the construction of the feature processing layer #3 includes: configuring L3 non-public part neurons that are only connected to some public part neurons in the periphery of the K public part neurons, L3 is an integer greater than 2, and the structure of each non-public part neuron in the L3 non-public part neurons is The construction of feature processing layer #2 includes: configuring L4 non-public part neurons connected only to some public part neurons in the periphery of the K public part neurons, L4 is an integer greater than 2, and the structure of each non-public part neuron in the L4 non-public part neurons is ; before training the 4 feature processing layers, In the matrix, the initial value of weight parameter #1 and the initial value of weight parameter #2 are reciprocals of each other.

[0020] Optionally, with respect to the growth condition of crops being the disease condition of crops, the electronic device processes M crop images through an AI / ML model to obtain growth information of crops in each of the M crop images, including: the electronic device processes the M crop images through an AI / ML model to obtain diseased areas of crops in each of the M crop images; the electronic device determines the growth condition of crops based on the M growth information of crops, including: the electronic device determines the severity of crop diseases by analyzing size changes of the same diseased area in at least some of the M crop images, and the number of diseased areas in the M crop images.

[0021] Optionally, with respect to the growth condition of crops being the number of male and female flowers of crops, the electronic device processes M crop images through an AI / ML model to obtain growth information of crops in each of the M crop images, including: the electronic device processes M crop images through an AI / ML model to obtain the number of male and female flowers of crops in each of the M crop images; the electronic device determines the growth condition of crops based on the M growth information of crops, including: the electronic device determines a change in the number of male and female flowers of crops based on the number of male and female flowers of crops in each of the M crop images.

[0022] Optionally, with respect to the growth condition of crops being the growth stage of crops, the electronic device processes M crop images through an AI / ML model to obtain growth information of crops in each of the M crop images, including: the electronic device processes M crop images through an AI / ML model to obtain the growth stage of crops in each of the M crop images.

[0023] Optionally, with respect to the growth condition of crops being the fruit condition of crops, the electronic device processes M crop images through an AI / ML model to obtain growth information of crops in each of the M crop images, including: the electronic device processes the M crop images through an AI / ML model to obtain the fruit size of crops in each of the M crop images; the electronic device determines the growth condition of crops based on the M growth information of crops, including: the electronic device determines the fruit growth rate of crops based on the fruit size of crops in each of the M crop images.

[0024] In a third aspect, a computer-readable storage medium is provided, comprising: a computer program or instructions; when the computer program or instructions are executed on a computer, the computer is caused to execute the method described in the first aspect.

[0025] In summary, the above method and system have the following technical effects:

[0026] The electronic device can obtain M crop images collected at M times for the same crop, and then process the M crop images through the AI / ML model, that is, continuously process, to obtain the growth information of the crop in each of the M crop images, and obtain a total of M growth information of the crop. In this way, the electronic device can determine the growth status of the crop, such as the disease status of the crop, the male and female flowers of the crop, the growth stage of the crop, or the fruit status of the crop, based on the M growth information of the crop, that is, the staged and continuous information, thereby realizing the full automated tracking and monitoring of the growth cycle of agricultural products through the AI / ML model. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A schematic diagram of the process of a highly integrated multi-source heterogeneous agricultural condition online monitoring method provided in an embodiment of the present application;

[0028] Figure 2 A schematic diagram of the structure of the model in the highly integrated multi-source heterogeneous agricultural condition online monitoring method provided in an embodiment of the present application;

[0029] Figure 3A , Figure 3B , Figure 3C A schematic diagram of an application scenario of a highly integrated multi-source heterogeneous agricultural condition online monitoring method provided in an embodiment of the present application;

[0030] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] The technical solution in this application will be described below in conjunction with the accompanying drawings.

[0032] The present application will present various aspects, embodiments or features around a system that may include multiple devices, components, modules, etc. It should be understood and appreciated that each system may include additional devices, components, modules, etc., and / or may not include all of the devices, components, modules, etc. discussed in conjunction with the figures. In addition, combinations of these schemes may also be used.

[0033] In addition, in the embodiments of the present application, words such as "exemplary" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present concepts in a concrete way.

[0034] In the embodiments of the present application, "information", "signal", "message", "channel" and "signaling" can sometimes be used interchangeably. It should be noted that when the distinction between them is not emphasized, the meanings they intend to express are matched. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the distinction between them is not emphasized, the meanings they intend to express are matched. In addition, the " / " mentioned in the present application can be used to express an "or" relationship.

[0035] The network architecture and business scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. A person of ordinary skill in the art can appreciate that with the evolution of the network architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0036] For example, Figure 1 The present invention provides a flow chart of a highly integrated multi-source heterogeneous agricultural condition online monitoring method. The method can be applied to electronic devices.

[0037] like Figure 1 As shown, the process of the highly integrated multi-source heterogeneous agricultural condition online monitoring method is as follows:

[0038] S101, the electronic device obtains M crop images collected at M moments for the same crop.

[0039] M is an integer greater than or equal to 3. The M crop images include branches and leaves of crops, or the M crop images include branches and leaves of crops, or the M crop images include fruit parts of crops.

[0040] S102, the electronic device processes M crop images through the AI / ML model to obtain growth information of crops in each of the M crop images, and obtains M growth information of crops in total.

[0041] The AI / ML model can be a convolutional neural network, such as the yolov5 network with improved network structure in this application.

[0042] The AI / ML model may include a feature extraction layer and four feature processing layers.

[0043] The feature extraction layer is a combination of at least one convolutional layer + a downsampling layer.

[0044] The four feature processing layers are open to the corresponding interfaces of the AI / ML models respectively, and some neuron nodes of the four feature processing layers are the same. For example, the four feature processing layers include feature processing layer #1, feature processing layer #2, feature processing layer #3, and feature processing layer #4.

[0045] Specifically, feature processing layer #1, feature processing layer #2, feature processing layer #3, and feature processing layer #4 all include K common part neurons, K is an integer greater than 2, and the structure of each common part neuron in the K common part neurons is , M is an integer greater than 1, The matrix includes weight parameter #1 and weight parameter #2. Matrix: Weight parameter #1 is located at On the diagonal of the matrix, weight parameter #1 is located Before AI / ML model training, The initial value of the weight parameter #1 in the matrix is ​​the reciprocal of the weight parameter #2, and the value of M is relatively large, such as 15 or 20. The size and structure of the matrix are suitable for coupling multi-modal capabilities through training, that is, the training effect of the first stage described below is better.

[0046] The training of AI / ML models consists of the first and second phases;

[0047] In the first stage, the feature extraction layer of the AI / ML model is only configured with K common part neurons, and the electronic device uses a training image set containing the branches and leaves of crops, a training image set containing the flower branches of crops, and a training image set containing the fruit parts of crops to train the AI / ML model to adjust the values ​​of weight parameter #1 and weight parameter #2 in each of the K common part neurons. The advantage of this is that the capabilities of different modalities of the AI / ML model, such as the ability to identify diseases, male and female flowers, growth stages, and fruit size, are coupled, and the accuracy of the capabilities of each modality is subsequently improved through the second stage of training and testing.

[0048] After the first stage, the second stage begins;

[0049] In the second stage, the electronic device replicates the K common part neurons in the feature extraction layer by 4 copies, and constructs 4 feature processing layers based on the 4 copies. The electronic device uses the training and test image sets containing the branches and leaves of crops to train and test the feature processing layer #1. During the training of the feature processing layer #1, the K common part neurons contained in the feature processing layer #1 are not regressed (i.e. The values ​​of weight parameter #1 and weight parameter #2 in the matrix of are not adjusted), the electronic device uses the training and test image sets containing the flower branch part of the crop to train and test the feature processing layer #2, and during the training of the feature processing layer #2, the K common part neurons contained in the feature processing layer #2 are not regressed (i.e. The values ​​of weight parameter #1 and weight parameter #2 in the matrix of are not adjusted), the electronic device uses the training and test image sets containing the branches and leaves of crops to train and test feature processing layer #3, and during the training of feature processing layer #3, the K common part neurons contained in feature processing layer #3 are not regressed (i.e. The values ​​of weight parameter #1 and weight parameter #2 in the matrix of are not adjusted), the electronic device uses the training and test image sets containing the fruit part of the crop to train and test the feature processing layer #4, and during the training of the feature processing layer #4, the K common part neurons contained in the feature processing layer #4 are not regressed (i.e. The values ​​of weight parameter #1 and weight parameter #2 in the matrix are not adjusted).

[0050] Among them, any common part neuron among the K common part neurons is connected to all other common part neurons among the common part neurons except this common part neuron, that is, a fully connected structure.

[0051] The construction of feature processing layer #1 includes: configuring L1 non-common part neurons that are only connected to some of the public part neurons in the periphery of the K public part neurons, that is, a non-fully connected structure, which can be specifically as follows: Figure 2 As shown, the advantage of this structure is that in the second stage of training, only the neural network elements with non-fully connected structures can be selected for regression, that is, the K common part neurons contained in the feature processing layer #1 are not regressed. L1 is an integer greater than 2, and the structure of each non-common part neuron in L1 non-common part neurons is A matrix, N is an integer greater than 1 and less than M, The matrix includes weight parameter #1 and weight parameter #2. Matrix: Weight parameter #1 is located at On the diagonal of the matrix, weight parameter #1 is located The matrix is ​​in other positions except the diagonal.

[0052] The construction of feature processing layer #2 includes: configuring L2 non-public part neurons that are only connected to some public part neurons in the periphery of the K public part neurons. For details, please refer to Figure 2 Understand that L2 is an integer greater than 2, and the structure of each non-public part neuron in L1 non-public part neurons is The matrix of feature processing layer #2 before training, In the matrix, the initial value of weight parameter #1 and the initial value of weight parameter #2 are reciprocals of each other.

[0053] The construction of feature processing layer #3 includes: configuring L3 non-public part neurons that are only connected to some public part neurons in the periphery of the K public part neurons. For details, please refer to Figure 2 Understand that L3 is an integer greater than 2, and the structure of each non-public part neuron in the L3 non-public part neurons is Matrix of

[0054] The construction of feature processing layer #2 includes: configuring L4 non-public part neurons that are only connected to some public part neurons in the periphery of the K public part neurons. For details, please refer to Figure 2 Understand that L4 is an integer greater than 2, and the structure of each non-public part neuron in the L4 non-public part neurons is The matrix of .

[0055] The values ​​of L1, L2, L3, and L4 can be different, and their respective values ​​can be set according to actual conditions.

[0056] Before training the 4 feature processing layers, In the matrix, the initial value of weight parameter #1 and the initial value of weight parameter #2 are reciprocals of each other.

[0057] It is understandable that the value of N is smaller than M because the second stage of training is aimed at improving the single-modal capability / accuracy, so the matrix structure can be reduced to reduce overhead.

[0058] On this basis, if M crop images contain the branches and leaves of crops, the electronic device processes the M crop images through the AI / ML model to obtain the diseased areas of crops in each of the M crop images. For example, the feature extraction layer extracts M feature information #1 from the M crop images one by one, and the feature extraction layer calls the interface of the feature processing layer #1 to input the M feature information #1 into the feature processing layer #1 for processing, and obtains the diseased areas of crops in each of the M crop images output by the feature processing layer #1. Specifically, it can be as follows Figure 3A shown.

[0059] If M crop images contain flower branches of crops, the electronic device processes the M crop images through the AI / ML model to obtain the number of male and female flowers of the crops in each of the M crop images. For example, the feature extraction layer extracts M feature information #2 from the M crop images one by one, and the feature extraction layer calls the interface of the feature processing layer #2 to input the M feature information #2 into the feature processing layer #2 for processing in sequence, and obtains the number of male and female flowers of the crops in each of the M crop images output by the feature processing layer #2.

[0060] If M crop images contain the branches and leaves of crops, the electronic device processes the M crop images through the AI / ML model to obtain the growth stage of the crops in each of the M crop images. For example, the feature extraction layer extracts M feature information #3 from the M crop images one by one, and the feature extraction layer calls the interface of the feature processing layer #3 to input the M feature information #3 into the feature processing layer #3 for processing in sequence, and obtains the growth stage of the crops in each of the M crop images output by the feature processing layer #3. Specifically, it can be as follows Figure 3B shown.

[0061] If M crop images contain the fruit part of the crop, the electronic device processes the M crop images through the AI / ML model to obtain the fruit size of the crop in each of the M crop images. For example, the feature extraction layer extracts M feature information #4 from the M crop images one by one, and the feature extraction layer calls the interface of the feature processing layer #4 to input the M feature information #4 into the feature processing layer #3 for processing in turn, and obtains the fruit size of the crop in each of the M crop images output by the feature processing layer #4. Specifically, it can be as follows Figure 3C shown.

[0062] S103, the electronic device determines the growth status of the crops according to the M growth information of the crops.

[0063] The growth condition of crops includes at least one of the following: the disease condition of crops, the condition of male and female flowers of crops, the growth stage condition of crops, or the condition of fruits of crops.

[0064] With respect to the growth condition of crops and the disease condition of crops, the electronic device determines the severity of the disease of the crops by analyzing the size change of the same diseased area in at least part of the M crop images and the number of diseased areas in the M crop images. The greater the size change and the greater the number, the higher the severity of the disease, and vice versa.

[0065] With respect to the growth condition of crops, which is the condition of male and female flowers of the crops, the electronic device determines the change in the number of male and female flowers of the crops according to the number of male and female flowers of the crops in each of the M crop images.

[0066] Regarding the growth condition of crops, which is the condition of the fruits of crops, the electronic device determines the fruit growth rate of crops according to the fruit size of crops in each of the M crop images, if the average growth rate of the fruits is as shown in the following formula 1. (1)

[0067] In Formula 1, v represents the growth rate, St+1 represents the area of ​​the fruit on the image at time t+1, and St represents the area of ​​the fruit on the image at time t+1. Indicates the time interval between two images.

[0068] In summary: the electronic device can obtain images taken of the weld from different dimensions / angles, such as M weld images; thus, the electronic device processes the M weld images through a neural network model to obtain a set of image feature information of each of the M weld images, and the scales of the image feature information sets of any two of the M weld images are different. In this way, the electronic device processes the set of image feature information of each of the M weld images through a neural network model, thereby evaluating the weld quality from different dimensions, and obtaining a more accurate and robust self-assessment result of the weld quality.

[0069] Combination of the above Figure 1 The highly integrated multi-source heterogeneous agricultural condition online monitoring method provided by the embodiment of the present application is described in detail. The following is a highly integrated multi-source heterogeneous agricultural condition online monitoring device for executing the highly integrated multi-source heterogeneous agricultural condition online monitoring method provided by the embodiment of the present application.

[0070] The device is applied to an electronic device, and the device is configured as follows: the electronic device obtains M crop images collected at M times for the same crop, where M is an integer greater than or equal to 3; the electronic device processes the M crop images through an AI / ML model to obtain growth information of the crop in each of the M crop images, and obtains a total of M growth information of the crop; the electronic device determines the growth condition of the crop based on the M growth information of the crop, wherein the growth condition of the crop includes at least one of the following: the disease condition of the crop, the condition of male and female flowers of the crop, the growth stage condition of the crop, or the fruit condition of the crop.

[0071] Optionally, the AI / ML model includes a feature extraction layer and four feature processing layers, the four feature processing layers are respectively open to the corresponding interfaces of the AI / ML models, some neuron nodes of the four feature processing layers are the same, and the four feature processing layers include feature processing layer #1, feature processing layer #2, feature processing layer #3 and feature processing layer #4; if the M crop images contain the branches and leaves of the crops, the feature extraction layer extracts M feature information #1 from the M crop images one by one, and the feature extraction layer calls the interface of the feature processing layer #1 to input the M feature information #1 into the feature processing layer #1 for processing in sequence, so as to obtain the diseased area of ​​the crops in each of the M crop images output by the feature processing layer #1; if the M crop images contain the flower and branch parts of the crops, the feature extraction layer extracts M feature information #2 from the M crop images one by one, and the feature extraction layer calls the interface of the feature processing layer #2 to The M feature information #2 are sequentially input into the feature processing layer #2 for processing, and the number of male and female flowers of the crops in each of the M crop images output by the feature processing layer #2 is obtained; if the M crop images contain branches and leaves of the crops, the feature extraction layer extracts M feature information #3 one by one from the M crop images, and the feature extraction layer inputs the M feature information #3 sequentially into the feature processing layer #3 for processing by calling the interface of the feature processing layer #3, and the growth stage of the crops in each of the M crop images output by the feature processing layer #3 is obtained; if the M crop images contain fruit parts of the crops, the feature extraction layer extracts M feature information #4 one by one from the M crop images, and the feature extraction layer inputs the M feature information #4 sequentially into the feature processing layer #3 for processing by calling the interface of the feature processing layer #4, and the fruit size of the crops in each of the M crop images output by the feature processing layer #4 is obtained.

[0072] Optionally, feature processing layer #1, feature processing layer #2, feature processing layer #3 and feature processing layer #4 each include K common part neurons, K is an integer greater than 2, and the structure of each common part neuron in the K common part neurons is , M is an integer greater than 1, The matrix includes weight parameter #1 and weight parameter #2. Matrix: Weight parameter #1 is located at On the diagonal of the matrix, weight parameter #1 is located Before AI / ML model training, The initial value of the weight parameter #1 in the matrix is ​​the reciprocal of the weight parameter #2; the training of the AI / ML model includes the first stage and the second stage; in the first stage, the feature extraction layer of the AI / ML model is only configured with K common part neurons, and the electronic device uses a training image set containing the branches and leaves of crops, a training image set containing the flower branches of crops, and a training image set containing the fruit parts of crops to train the AI / ML model to adjust the values ​​of the weight parameters #1 and the weight parameters #2 in each of the K common part neurons; after the first stage, the second stage is entered; in the second stage, the electronic device copies the K common part neurons in the feature extraction layer to 4 copies, and constructs 4 feature processing layers one by one on the basis of the 4 copies, and the electronic device uses the training and test image sets containing the branches and leaves of crops to train and test the feature processing Layer #1, and during the training process of feature processing layer #1, the K common part neurons contained in feature processing layer #1 are not regressed, the electronic device uses the training and test image sets containing the flower and branch parts of crops to train and test feature processing layer #2, and during the training process of feature processing layer #2, the K common part neurons contained in feature processing layer #2 are not regressed, the electronic device uses the training and test image sets containing the branches and leaves of crops to train and test feature processing layer #3, and during the training process of feature processing layer #3, the K common part neurons contained in feature processing layer #3 are not regressed, the electronic device uses the training and test image sets containing the fruit part of crops to train and test feature processing layer #4, and during the training process of feature processing layer #4, the K common part neurons contained in feature processing layer #4 are not regressed.

[0073] Optionally, any common part neuron among the K common part neurons is connected to all other common part neurons among the common part neurons except the common part neuron; the construction of feature processing layer #1 includes: configuring L1 non-common part neurons connected only to some of the common part neurons in the periphery among the K common part neurons, L1 is an integer greater than 2, and the structure of each non-common part neuron among the L1 non-common part neurons is A matrix, N is an integer greater than 1 and less than M, The matrix includes weight parameter #1 and weight parameter #2. Matrix: Weight parameter #1 is located at On the diagonal of the matrix, weight parameter #1 is located The matrix is ​​located at other positions except the diagonal; the construction of feature processing layer #2 includes: configuring L2 non-public part neurons that are only connected to some public part neurons in the periphery of the K public part neurons, L2 is an integer greater than 2, and the structure of each non-public part neuron in the L1 non-public part neurons is The matrix of feature processing layer #2 before training, The initial value of the weight parameter #1 in the matrix is ​​the reciprocal of the initial value of the weight parameter #2; the construction of the feature processing layer #3 includes: configuring L3 non-public part neurons that are only connected to some public part neurons in the periphery of the K public part neurons, L3 is an integer greater than 2, and the structure of each non-public part neuron in the L3 non-public part neurons is The construction of feature processing layer #2 includes: configuring L4 non-public part neurons connected only to some public part neurons in the periphery of the K public part neurons, L4 is an integer greater than 2, and the structure of each non-public part neuron in the L4 non-public part neurons is ; before training the 4 feature processing layers, In the matrix, the initial value of weight parameter #1 and the initial value of weight parameter #2 are reciprocals of each other.

[0074] Optionally, with respect to the growth condition of crops being the disease condition of crops, the electronic device processes M crop images through an AI / ML model to obtain growth information of crops in each of the M crop images, including: the electronic device processes the M crop images through an AI / ML model to obtain diseased areas of crops in each of the M crop images; the electronic device determines the growth condition of crops based on the M growth information of crops, including: the electronic device determines the severity of crop diseases by analyzing size changes of the same diseased area in at least some of the M crop images, and the number of diseased areas in the M crop images.

[0075] Optionally, with respect to the growth condition of crops being the number of male and female flowers of crops, the electronic device processes M crop images through an AI / ML model to obtain growth information of crops in each of the M crop images, including: the electronic device processes M crop images through an AI / ML model to obtain the number of male and female flowers of crops in each of the M crop images; the electronic device determines the growth condition of crops based on the M growth information of crops, including: the electronic device determines a change in the number of male and female flowers of crops based on the number of male and female flowers of crops in each of the M crop images.

[0076] Optionally, with respect to the growth condition of crops being the growth stage of crops, the electronic device processes M crop images through an AI / ML model to obtain growth information of crops in each of the M crop images, including: the electronic device processes M crop images through an AI / ML model to obtain the growth stage of crops in each of the M crop images.

[0077] Optionally, with respect to the growth condition of crops being the fruit condition of crops, the electronic device processes M crop images through an AI / ML model to obtain growth information of crops in each of the M crop images, including: the electronic device processes the M crop images through an AI / ML model to obtain the fruit size of crops in each of the M crop images; the electronic device determines the growth condition of crops based on the M growth information of crops, including: the electronic device determines the fruit growth rate of crops based on the fruit size of crops in each of the M crop images.

[0078] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. For example, the electronic device may be a terminal device, or a chip (system) or other component or assembly that can be set in the terminal device. Figure 4 As shown, the electronic device 400 may include a processor 401. Optionally, the electronic device 400 may also include a memory 402 and / or a transceiver 403. The processor 401 is coupled to the memory 402 and the transceiver 403, such as being connected via a communication bus. In addition, the electronic device 400 may also be a chip, such as including the processor 401, in which case the transceiver may be an output and input interface of the chip.

[0079] Combine the following Figure 4 The components of the electronic device 400 are specifically introduced as follows:

[0080] The processor 401 is the control center of the electronic device 400, and may be a processor or a general term for multiple processing elements. For example, the processor 401 is one or more central processing units (CPUs), or may be application specific integrated circuits (ASICs), or may be one or more integrated circuits configured to implement the embodiments of the present application, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (FPGAs).

[0081] Optionally, the processor 401 can execute various functions of the electronic device 400 by running or executing the software program stored in the memory 402 and calling the scientific data stored in the memory 402, such as executing the above Figure 1 The highly integrated multi-source heterogeneous agricultural condition online monitoring method shown.

[0082] In a specific implementation, as an embodiment, the processor 401 may include one or more CPUs, such as Figure 4 CPU0 and CPU1 are shown in FIG.

[0083] In a specific implementation, as an embodiment, the electronic device 400 may also include multiple processors. Each of these processors may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here may refer to one or more devices, circuits, and / or processing cores for processing scientific data (such as computer programs or instructions).

[0084] The memory 402 is used to store the software program for executing the solution of the present application, and the execution is controlled by the processor 401. The specific implementation method can refer to the above method embodiment, which will not be repeated here.

[0085] Optionally, the memory 402 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or scientific data structures and can be accessed by a computer, but is not limited thereto. The memory 402 may be integrated with the processor 401, or may exist independently and accessed through the interface circuit ( Figure 4 (not shown) is coupled to the processor 401, which is not specifically limited in this embodiment of the present application.

[0086] The transceiver 403 is used for communication with other electronic devices. For example, if the electronic device 400 is a terminal device, the transceiver 403 can be used to communicate with a network device, or with another terminal device. For another example, if the electronic device 400 is a network device, the transceiver 403 can be used to communicate with a terminal device, or with another network device.

[0087] Optionally, the transceiver 403 may include a receiver and a transmitter ( Figure 4 The receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

[0088] Optionally, the transceiver 403 may be integrated with the processor 401, or may exist independently and communicate with the electronic device 400 through an interface circuit ( Figure 4 (not shown) is coupled to the processor 401, which is not specifically limited in this embodiment of the present application.

[0089] Understandably, Figure 4 The structure of the electronic device 400 shown in the figure does not constitute a limitation on the electronic device, and the actual electronic device may include more or fewer components than those shown in the figure, or combine certain components, or arrange the components differently.

[0090] In addition, the technical effects of the electronic device 400 can refer to the technical effects of the methods described in the above method embodiments, which will not be repeated here.

[0091] It should be understood that the processor in the embodiments of the present application may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0092] It should also be understood that the memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0093] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer program or instruction can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer program or instruction can be transmitted from one website, computer, server or scientific data center to another website, computer, server or scientific data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a scientific data storage device such as a server, scientific data center, etc. that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0094] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0095] In this application, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0096] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0097] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0098] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0099] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0100] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0101] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0102] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0103] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A highly integrated multi-source heterogeneous agricultural condition online monitoring method, characterized in that: Applied to electronic equipment, the method comprises: The electronic device acquires M crop images collected at M moments for the same crop, where M is an integer greater than or equal to 3; The electronic device processes the M crop images through the AI / ML model to obtain growth information of the crop in each of the M crop images, and obtains M growth information of the crop in total; The electronic device determines the growth condition of the crop according to the M growth information of the crop, wherein the growth condition of the crop includes at least one of the following: the disease condition of the crop, the condition of male and female flowers of the crop, the growth stage condition of the crop, or the condition of the fruit of the crop; The AI / ML model includes a feature extraction layer and four feature processing layers, the four feature processing layers are respectively open to interfaces corresponding to the AI / ML models, some neuron nodes of the four feature processing layers are the same, and the four feature processing layers include feature processing layer #1, feature processing layer #2, feature processing layer #3 and feature processing layer #4; If the M crop images contain the branches and leaves of the crop, the feature extraction layer extracts M feature information #1 from the M crop images one by one, and the feature extraction layer calls the interface of the feature processing layer #1 to sequentially input the M feature information #1 to the feature processing layer #1 for processing, so as to obtain the diseased area of ​​the crop in each of the M crop images output by the feature processing layer #1; If the M crop images contain flower branches of the crop, the feature extraction layer extracts M feature information #2 from the M crop images one by one, and the feature extraction layer calls the interface of the feature processing layer #2 to sequentially input the M feature information #2 to the feature processing layer #2 for processing, so as to obtain the number of male and female flowers of the crop in each of the M crop images output by the feature processing layer #2; If the M crop images contain the branches and leaves of the crop, the feature extraction layer extracts M feature information #3 from the M crop images one by one, and the feature extraction layer calls the interface of the feature processing layer #3 to sequentially input the M feature information #3 to the feature processing layer #3 for processing, so as to obtain the growth stage of the crop in each of the M crop images output by the feature processing layer #3; If the M crop images contain the fruit part of the crop, the feature extraction layer extracts M feature information #4 from the M crop images one by one, and the feature extraction layer calls the interface of the feature processing layer #4 to sequentially input the M feature information #4 to the feature processing layer #4 for processing, so as to obtain the fruit size of the crop in each of the M crop images output by the feature processing layer #4; The feature processing layer #1, feature processing layer #2, feature processing layer #3 and feature processing layer #4 all include K common part neurons, K is an integer greater than 2, and the structure of each common part neuron in the K common part neurons is A matrix, M is an integer greater than 1, The matrix includes weight parameter #1 and weight parameter #2. Matrix: The weight parameter #1 is located in the On the diagonal of the matrix, the weight parameter #2 is located at Before the AI / ML model is trained, The initial value of the weight parameter #1 in the matrix is ​​the reciprocal of the weight parameter #2; The training of the AI / ML model includes a first stage and a second stage; In the first stage, the feature extraction layer of the AI / ML model is configured with only the K common part neurons, and the electronic device trains the AI / ML model using a training image set containing the branches and leaves of the crop, a training image set containing the flower branches of the crop, and a training image set containing the fruit parts of the crop to adjust the values ​​of the weight parameter #1 and the weight parameter #2 in each of the K common part neurons; entering the second stage after the first stage; In the second stage, the electronic device copies the K common part neurons in the feature extraction layer into four copies, and constructs the four feature processing layers one by one based on the four copies, the electronic device uses the training and test image sets containing the branches and leaves of the crop to train and test the feature processing layer #1, and during the training of the feature processing layer #1, the K common part neurons contained in the feature processing layer #1 are not regressed, the electronic device uses the training and test image sets containing the flower and branch parts of the crop to train and test the feature processing layer #2, and during the training of the feature processing layer #2 During the process, the K common part neurons included in the feature processing layer #2 are not regressed, the electronic device uses the training and test image sets containing the branches and leaves of the crop to train and test the feature processing layer #3, and during the training of the feature processing layer #3, the K common part neurons included in the feature processing layer #3 are not regressed, the electronic device uses the training and test image sets containing the fruit part of the crop to train and test the feature processing layer #4, and during the training of the feature processing layer #4, the K common part neurons included in the feature processing layer #4 are not regressed; Any common part neuron among the K common part neurons is connected to all other common part neurons among the common part neurons except the common part neuron; The construction of the feature processing layer #1 includes: configuring L1 non-common part neurons connected only to some of the public part neurons in the periphery of the K public part neurons, L1 is an integer greater than 2, and the structure of each non-common part neuron in the L1 non-common part neurons is A matrix, N is an integer greater than 1 and less than M, the The matrix includes weight parameter #1 and weight parameter #2. Matrix: The weight parameter #1 is located in the On the diagonal of the matrix, the weight parameter #2 is located at The matrix is ​​at other positions except the diagonal; The construction of the feature processing layer #2 includes: configuring L2 non-public part neurons connected only to the peripheral public part neurons among the K public part neurons, L2 is an integer greater than 2, and the structure of each non-public part neuron among the L1 non-public part neurons is as follows: The matrix of the feature processing layer #2 before training, In the matrix, the initial value of the weight parameter #1 and the initial value of the weight parameter #2 are reciprocals of each other; The construction of the feature processing layer #3 includes: configuring L3 non-public part neurons connected only to some public part neurons in the periphery of the K public part neurons, L3 is an integer greater than 2, and the structure of each non-public part neuron in the L3 non-public part neurons is as follows: Matrix of The construction of the feature processing layer #2 includes: configuring L4 non-public part neurons connected only to some public part neurons in the periphery of the K public part neurons, L4 is an integer greater than 2, and the structure of each non-public part neuron in the L4 non-public part neurons is as follows: Matrix of Before the four feature processing layers are trained, In the matrix, the initial value of the weight parameter #1 and the initial value of the weight parameter #2 are reciprocals of each other.

2. The method according to claim 1, characterized in that With respect to the growth condition of the crop being the disease condition of the crop, the electronic device processes the M crop images through an AI / ML model to obtain growth information of the crop in each of the M crop images, including: The electronic device processes the M crop images by using the AI / ML model to obtain the diseased area of ​​the crop in each of the M crop images; The electronic device determines the growth condition of the crop according to the M growth information of the crop, including: The electronic device determines the severity of the crop disease by analyzing the size changes of the same diseased area in at least some of the M crop images and the number of diseased areas in the M crop images.

3. The method according to claim 1, characterized in that The growth condition of the crop is the condition of male and female flowers of the crop, the electronic device processes the M crop images through an AI / ML model to obtain growth information of the crop in each of the M crop images, including: The electronic device processes the M crop images by using the AI / ML model to obtain the number of male and female flowers of the crop in each of the M crop images; The electronic device determines the growth condition of the crop according to the M growth information of the crop, including: The electronic device determines a change in the number of male and female flowers of the crop according to the number of male and female flowers of the crop in each of the M crop images.

4. The method according to claim 1, characterized in that: The growth condition of the crop is the growth stage of the crop, and the electronic device processes the M crop images through an AI / ML model to obtain growth information of the crop in each of the M crop images, including: The electronic device processes the M crop images through the AI / ML model to obtain the growth stage of the crops in each of the M crop images.

5. The method according to claim 1, characterized in that The growth condition of the crop is the fruit condition of the crop, the electronic device processes the M crop images through an AI / ML model to obtain growth information of the crop in each of the M crop images, including: The electronic device processes the M crop images by using the AI / ML model to obtain the fruit size of the crop in each of the M crop images; The electronic device determines the growth condition of the crop according to the M growth information of the crop, including: The electronic device determines the fruit growth rate of the crop according to the fruit size of the crop in each of the M crop images.

6. A highly integrated multi-source heterogeneous agricultural condition online monitoring device, characterized in that: Applied to electronic equipment, the device is configured as follows: The electronic device acquires M crop images collected at M moments for the same crop, where M is an integer greater than or equal to 3; The electronic device processes the M crop images through the AI / ML model to obtain growth information of the crop in each of the M crop images, and obtains M growth information of the crop in total; The electronic device determines the growth condition of the crop according to the M growth information of the crop, wherein the growth condition of the crop includes at least one of the following: the disease condition of the crop, the condition of male and female flowers of the crop, the growth stage condition of the crop, or the condition of the fruit of the crop; The AI / ML model includes a feature extraction layer and four feature processing layers, the four feature processing layers are respectively open to interfaces corresponding to the AI / ML models, some neuron nodes of the four feature processing layers are the same, and the four feature processing layers include feature processing layer #1, feature processing layer #2, feature processing layer #3 and feature processing layer #4; If the M crop images contain the branches and leaves of the crop, the feature extraction layer extracts M feature information #1 from the M crop images one by one, and the feature extraction layer calls the interface of the feature processing layer #1 to sequentially input the M feature information #1 to the feature processing layer #1 for processing, so as to obtain the diseased area of ​​the crop in each of the M crop images output by the feature processing layer #1; If the M crop images contain flower branches of the crop, the feature extraction layer extracts M feature information #2 from the M crop images one by one, and the feature extraction layer calls the interface of the feature processing layer #2 to sequentially input the M feature information #2 to the feature processing layer #2 for processing, so as to obtain the number of male and female flowers of the crop in each of the M crop images output by the feature processing layer #2; If the M crop images contain the branches and leaves of the crop, the feature extraction layer extracts M feature information #3 from the M crop images one by one, and the feature extraction layer calls the interface of the feature processing layer #3 to sequentially input the M feature information #3 to the feature processing layer #3 for processing, so as to obtain the growth stage of the crop in each of the M crop images output by the feature processing layer #3; If the M crop images contain the fruit part of the crop, the feature extraction layer extracts M feature information #4 from the M crop images one by one, and the feature extraction layer calls the interface of the feature processing layer #4 to sequentially input the M feature information #4 to the feature processing layer #4 for processing, so as to obtain the fruit size of the crop in each of the M crop images output by the feature processing layer #4; The feature processing layer #1, feature processing layer #2, feature processing layer #3 and feature processing layer #4 all include K common part neurons, K is an integer greater than 2, and the structure of each common part neuron in the K common part neurons is A matrix, M is an integer greater than 1, The matrix includes weight parameter #1 and weight parameter #2. Matrix: The weight parameter #1 is located in the On the diagonal of the matrix, the weight parameter #2 is located at Before the AI / ML model is trained, The initial value of the weight parameter #1 in the matrix is ​​the reciprocal of the weight parameter #2; The training of the AI / ML model includes a first stage and a second stage; In the first stage, the feature extraction layer of the AI / ML model is configured with only the K common part neurons, and the electronic device trains the AI / ML model using a training image set containing the branches and leaves of the crop, a training image set containing the flower branches of the crop, and a training image set containing the fruit parts of the crop to adjust the values ​​of the weight parameter #1 and the weight parameter #2 in each of the K common part neurons; entering the second stage after the first stage; In the second stage, the electronic device copies the K common part neurons in the feature extraction layer into four copies, and constructs the four feature processing layers one by one based on the four copies, the electronic device uses the training and test image sets containing the branches and leaves of the crop to train and test the feature processing layer #1, and during the training of the feature processing layer #1, the K common part neurons contained in the feature processing layer #1 are not regressed, the electronic device uses the training and test image sets containing the flower and branch parts of the crop to train and test the feature processing layer #2, and during the training of the feature processing layer #2 During the process, the K common part neurons included in the feature processing layer #2 are not regressed, the electronic device uses the training and test image sets containing the branches and leaves of the crop to train and test the feature processing layer #3, and during the training of the feature processing layer #3, the K common part neurons included in the feature processing layer #3 are not regressed, the electronic device uses the training and test image sets containing the fruit part of the crop to train and test the feature processing layer #4, and during the training of the feature processing layer #4, the K common part neurons included in the feature processing layer #4 are not regressed; Any common part neuron among the K common part neurons is connected to all other common part neurons among the common part neurons except the common part neuron; The construction of the feature processing layer #1 includes: configuring L1 non-common part neurons connected only to some of the public part neurons in the periphery of the K public part neurons, L1 is an integer greater than 2, and the structure of each non-common part neuron in the L1 non-common part neurons is A matrix, N is an integer greater than 1 and less than M, the The matrix includes weight parameter #1 and weight parameter #2. Matrix: The weight parameter #1 is located in the On the diagonal of the matrix, the weight parameter #2 is located at The matrix is ​​at other positions except the diagonal; The construction of the feature processing layer #2 includes: configuring L2 non-public part neurons connected only to the peripheral public part neurons among the K public part neurons, L2 is an integer greater than 2, and the structure of each non-public part neuron among the L1 non-public part neurons is as follows: The matrix of the feature processing layer #2 before training, In the matrix, the initial value of the weight parameter #1 and the initial value of the weight parameter #2 are reciprocals of each other; The construction of the feature processing layer #3 includes: configuring L3 non-public part neurons connected only to some public part neurons in the periphery of the K public part neurons, L3 is an integer greater than 2, and the structure of each non-public part neuron in the L3 non-public part neurons is as follows: Matrix of The construction of the feature processing layer #2 includes: configuring L4 non-public part neurons connected only to some public part neurons in the periphery of the K public part neurons, L4 is an integer greater than 2, and the structure of each non-public part neuron in the L4 non-public part neurons is as follows: Matrix of Before the four feature processing layers are trained, In the matrix, the initial value of the weight parameter #1 and the initial value of the weight parameter #2 are reciprocals of each other.

7. An electronic device, characterized in that: The electronic device comprises a processor configured to execute the method according to any one of claims 1-5.

Citation Information

Patent Citations

  • Agricultural insurance compensation amount prediction method, device and equipment and readable storage medium

    CN110689191A

  • Field fruit size measuring method

    CN110853080A

  • Intelligent pest and disease identification method and system based on CNN and LSTM

    CN113920426A

  • Method and system for analyzing growth vigor and yield of target crop

    CN114492166A

  • Crop growth period identification method based on parallel detection under computer vision

    CN114627411A