Image-based plant growth cycle monitoring system and method

Abstract

The invention relates to the technical field of intelligent monitoring, and particularly discloses an image-based plant growth cycle monitoring system and method, which is characterized in that plant leaf image sets at different growth stages are acquired in advance, and an image processing technology based on deep learning is introduced; image feature extraction and prototype feature extraction aggregation analysis are carried out on the plant leaf image set of each growth stage to capture plant leaf prototype image features of each growth stage, and then leaf feature matching analysis is carried out on a plant image to be identified by taking the plant leaf prototype image features of each growth stage as a reference to obtain a plant leaf feature matching result; therefore, the current growth stage of the plant can be quickly judged, and targeted fertilization management can be conveniently carried out on the plant. By means of the mode, intelligent monitoring of the plant growth cycle can be achieved, effective guidance is provided for fertilization management of plants, and then scientificity and accuracy of fertilization management are improved.

Description

Technical Field

[0001] The present application relates to the technical field of intelligent monitoring, and more specifically, to an image-based plant growth cycle monitoring system and method. Background Art

[0002] In the modern agricultural production system, scientific and precise fertilization management is the core link to improve crop yield and quality and ensure the sustainable development of agriculture. Traditional fertilization management mainly relies on manual experience judgment or soil nutrient detection, which has many drawbacks. For example, manual experience judgment requires a large amount of manpower and time, and there is a lag in subjective experience, which is likely to lead to untimely fertilization timing; while soil nutrient detection requires destructive sampling and cannot reflect the actual absorption status of plants. According to FAO statistics, due to misjudgment of growth stages, about 27% of nitrogen fertilizers are wasted globally every year.

[0003] In recent years, plant nutrient diagnosis technology based on machine vision has provided a new way for dynamic fertilization. Existing methods mostly evaluate the nutritional status through leaf color (such as SPAD value estimation) for fertilization management, but this method ignores the strong correlation between plant nutrient requirements and growth stages. For example, the nitrogen requirement during the tillering stage of rice is 40-60% higher than that during the heading stage. If fertilization is only based on leaf color, it is easy to cause a mismatch between fertilization recommendations and actual needs, thereby affecting crop growth and yield.

[0004] Since the nutrient requirements of plants vary significantly at different growth stages, the prerequisite for precise fertilization is to accurately control the plant growth cycle. However, most of the existing plant growth cycle monitoring models only construct binary classifiers for specific growth stages and lack the ability of full-cycle continuous monitoring; secondly, the growth of plants is a continuous and dynamic process, and most of the existing methods are based on the analysis of a single static image, and the progressive feature correlation of growth stages is not fully considered in the feature learning process, resulting in a relatively high cross-stage misjudgment rate.

[0005] Therefore, an optimized image-based plant growth cycle monitoring system and method are expected. Summary of the Invention

[0006] To solve the above technical problems, the present application is proposed. Embodiments of the present application provide an image-based plant growth cycle monitoring system and method, which pre-collect image sets of plant leaves at different growth stages, and introduce image processing technology based on deep learning to perform image feature extraction and prototype feature extraction and aggregation analysis on the image sets of plant leaves at each growth stage to capture the prototype image features of plant leaves at each growth stage. Furthermore, based on the prototype image features of plant leaves at each growth stage, leaf feature matching analysis is performed on the plant image to be recognized, so as to quickly determine the growth stage of the plant currently, so as to perform targeted fertilization management on the plant. In this way, intelligent monitoring of the plant growth cycle can be realized, providing effective guidance for the fertilization management of the plant, and further improving the scientificity and accuracy of fertilization management.

[0007] According to one aspect of the present application, there is provided an image-based plant growth cycle monitoring method, which includes: Collecting plant leaf images at different growth stages and segmenting the image sets according to the growth stages to obtain subsets of plant leaf images at the first to fifth growth stages; Performing image feature extraction and prototype feature extraction and aggregation analysis on the subsets of plant leaf images at each growth stage in the subsets of plant leaf images at the first to fifth growth stages to obtain prototype image features of plant leaves at the first to fifth growth stages; Obtaining a plant image to be recognized; Extracting leaf features from the plant image to be recognized to obtain leaf image features to be recognized; Performing image feature matching analysis on the leaf image features to be recognized and the prototype image features of plant leaves at the first to fifth growth stages to obtain a matching analysis result; Based on the matching analysis result, determining the growth stage of the plant image to be recognized.

[0008] According to another aspect of the present application, there is provided an image-based plant growth cycle monitoring system, which includes: An image acquisition module, configured to collect plant leaf images at different growth stages and segment the image sets according to the growth stages to obtain subsets of plant leaf images at the first to fifth growth stages; A plant leaf prototype feature extraction module, configured to perform image feature extraction and prototype feature extraction and aggregation analysis on the subsets of plant leaf images at each growth stage in the subsets of plant leaf images at the first to fifth growth stages to obtain prototype image features of plant leaves at the first to fifth growth stages; A plant image to be recognized acquisition module, configured to obtain a plant image to be recognized; The plant image feature extraction module to be recognized is used to extract leaf features from the plant image to be recognized to obtain the leaf image features to be recognized; The feature matching and analysis module is used to perform image feature matching and analysis on the leaf image features to be recognized and the plant leaf prototype image features in the first to fifth growth stages to obtain a matching analysis result; The growth stage determination module is used to determine the growth stage of the plant image to be recognized based on the matching analysis result.

[0009] Compared with the prior art, the image-based plant growth cycle monitoring system and method provided by this application pre-collects plant leaf image sets in different growth stages, and introduces image processing technology based on deep learning to perform image feature extraction and prototype feature extraction and aggregation analysis on the plant leaf image sets in each growth stage to capture the plant leaf prototype image features in each growth stage. Furthermore, based on the plant leaf prototype image features in each growth stage, leaf feature matching and analysis is performed on the plant image to be recognized, so as to quickly judge the current growth stage of the plant, which is convenient for targeted fertilization management of the plant. In this way, intelligent monitoring of the plant growth cycle can be realized, providing effective guidance for the fertilization management of the plant, and further improving the scientificity and accuracy of fertilization management. Description of the Drawings

[0010] By describing the embodiments of the present application in more detail in conjunction with the drawings, the above and other objects, features and advantages of the present application will become more obvious. The drawings are used to provide a further understanding of the embodiments of the present application, and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation to the present application. In the drawings, the same reference numerals generally represent the same components or steps.

[0011] Figure 1 It is a flowchart of the image-based plant growth cycle monitoring method according to an embodiment of the present application.

[0012] Figure 2 It is a schematic diagram of data flow of the image-based plant growth cycle monitoring method according to an embodiment of the present application.

[0013] Figure 3 It is a flowchart of sub-step S2 of the image-based plant growth cycle monitoring method according to an embodiment of the present application.

[0014] Figure 4 It is a flowchart of sub-step S22 of the image-based plant growth cycle monitoring method according to an embodiment of the present application.

[0015] Figure 5Flowchart of sub-step S221 of the image-based plant growth cycle monitoring method according to an embodiment of the present application.

[0016] Figure 6 Flowchart of sub-step S222 of the image-based plant growth cycle monitoring method according to an embodiment of the present application.

[0017] Figure 7 Flowchart of sub-step S4 of the image-based plant growth cycle monitoring method according to an embodiment of the present application.

[0018] Figure 8 Block diagram of the image-based plant growth cycle monitoring system according to an embodiment of the present application.

[0019] Figure 9 Automatic fertilization flowchart of the automatic plant fertilization management system according to an embodiment of the present application. Detailed implementation manners

[0020] As shown in the present application and the claims, unless the context clearly indicates otherwise, words such as "a", "an", "one", and / or "the" are not specifically singular and may also include the plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0021] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, however, any number of different modules can be used and run on the user terminal and / or the server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0022] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the operations before or below do not necessarily need to be executed precisely in order. On the contrary, various steps can be processed in reverse order or simultaneously as needed. At the same time, other operations can also be added to these processes, or one or more steps can be removed from these processes.

[0023] Next, exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. It should be understood that the present application is not limited by the exemplary embodiments described herein.

[0024] It should be noted that in the present application, all actions of obtaining data are carried out on the premise of complying with the corresponding data protection regulations and policies of the country where it is located and obtaining the authorization given by the owner of the corresponding device.

[0025] In view of the technical problems described in the above background art, the present application proposes an image-based plant growth cycle monitoring method. It pre-collects plant leaf image sets at different growth stages and introduces image processing technology based on deep learning to perform image feature extraction and prototype feature extraction and aggregation analysis on the plant leaf image sets at each growth stage, so as to capture the prototype image features of plant leaves at each growth stage. Furthermore, based on the prototype image features of plant leaves at each growth stage, leaf feature matching analysis is performed on the plant image to be recognized, so as to quickly determine the current growth stage of the plant, facilitating targeted fertilization management of the plant. In this way, intelligent monitoring of the plant growth cycle can be achieved, providing effective guidance for the fertilization management of the plant, and further improving the scientificity and accuracy of fertilization management.

[0026] Figure 1 It is a flowchart of the image-based plant growth cycle monitoring method according to an embodiment of the present application. Figure 2 It is a schematic diagram of data flow of the image-based plant growth cycle monitoring method according to an embodiment of the present application. As Figure 1 and Figure 2 shown, the image-based plant growth cycle monitoring method includes the steps: S1, collecting plant leaf images at different growth stages and segmenting the image sets according to the growth stages to obtain subsets of plant leaf images at the first to fifth growth stages; S2, performing image feature extraction and prototype feature extraction and aggregation analysis on the subsets of plant leaf images at each of the first to fifth growth stages to obtain prototype image features of plant leaves at the first to fifth growth stages; S3, obtaining the plant image to be recognized; S4, extracting leaf features from the plant image to be recognized to obtain the leaf image features to be recognized; S5, performing image feature matching analysis on the leaf image features to be recognized and the prototype image features of plant leaves at the first to fifth growth stages to obtain a matching analysis result; S6, determining the growth stage of the plant image to be recognized based on the matching analysis result.

[0027] In the above image-based plant growth cycle monitoring method, in step S1, plant leaf images at different growth stages are collected and segmented according to the growth stages to obtain subsets of plant leaf images at the first to fifth growth stages. It should be understood that when the plant is in the transition period between adjacent growth stages, the morphological characteristics of the leaves often show a progressive evolution, resulting in the fact that the feature analysis method using a single static image is very likely to misjudge the leaves in the heading stage as those in the jointing stage. Therefore, in this application, the whole growth cycle of the plant from germination to senescence is continuously monitored, and the complete growth cycle is divided into five key stages according to plant physiology standards. The growth stages include the germination stage, the growth stage, the flowering stage, the fruiting stage, and the senescence stage, forming subsets of plant leaf images at the first to fifth growth stages. At the same time, it is ensured that each subset contains a sufficient number of sample images, which can cover the morphological variation range of the leaves in this growth stage and fully reflect the change law of the leaf characteristics in this growth stage, so as to provide a comprehensive data basis for the subsequent identification and analysis of the plant growth stage.

[0028] Specifically, in the actual operation process, according to plant physiology standards, the complete growth cycle is divided into five key stages: the germination stage, the growth stage, the flowering stage, the fruiting stage, and the senescence stage. Each stage has unique morphological characteristics, which are reflected not only in the color of the leaves, but also in the changes in their shape, size, and texture. Therefore, when collecting leaf images, not only the surface details of the leaves should be concerned, but also the possible impacts of environmental factors such as lighting conditions and background interference on the image quality should be considered.

[0029] After selecting the research object, multiple observation points need to be set up to ensure that the growth changes of the plant in different environments can be captured. The selected plant samples are photographed regularly every day using a high-resolution camera, and it is ensured that the shooting angle and distance are kept consistent each time to reduce variable interference during subsequent analysis. At the same time, professional software is used to preliminarily screen the captured pictures, eliminating the blurred or obviously defective photos and retaining high-quality image data. As the plant grows, its leaves will undergo significant morphological changes, especially during the transition period between adjacent growth stages, and this change is particularly obvious. For example, during the transition from the growth stage to the flowering stage, fine wrinkles may appear on the leaf edge, and the color changes from dark green to light green, and these are all important feature information. Therefore, it is particularly important to record the image data at each time point in detail.

[0030] Then, these images are classified and sorted according to the growth stages to form subsets of plant leaf images at the first to fifth growth stages. This process requires strict review of each image to confirm its specific growth stage. Through learning a large number of samples, the system can better understand the transition state between each growth stage, so as to make a more accurate judgment.

[0031] To ensure that each subset contains a sufficient number of sample images to fully reflect the variation law of leaf characteristics at this growth stage, on the one hand, the observation frequency can be increased, and the sampling intensity can be increased at the critical nodes of plant growth; on the other hand, the multi-view shooting technology can be introduced to capture the full view of the leaves from different angles to further enhance the data dimension. In addition, the information obtained by other sensors, such as environmental parameters like temperature and humidity, can be combined to assist in determining the current growth stage of the plant. All these measures work together to provide a solid data foundation for the subsequent identification and analysis of the plant growth stage.

[0032] In the above image-based plant growth cycle monitoring method, in step S2, image feature extraction and prototype feature extraction and aggregation analysis are performed on each subset of plant leaf images at the first to fifth growth stages to obtain the prototype image features of plant leaves at the first to fifth growth stages. It should be understood that since plant growth is a continuous and dynamic process, there are progressive feature correlations within each growth stage. Therefore, in this application, further feature correlation and aggregation analysis are performed on the plant leaf images within each growth stage image subset to integrate and refine the scattered image features to form the prototype image features representing this growth stage, thereby obtaining the prototype image features of plant leaves at the first to fifth growth stages, which serve as an important reference basis for subsequent determination of the growth stage of the plant image to be identified. Among them, Figure 3 is a flowchart of sub-step S2 of the image-based plant growth cycle monitoring method according to an embodiment of the present application. As Figure 3 shown, step S2 includes steps: S21, performing image feature extraction based on dilated convolution coding on each plant leaf image at the first growth stage in the subset of plant leaf images at the first growth stage to obtain a subset of plant leaf image features at the first growth stage; S22, performing prototype feature extraction and aggregation analysis on the subset of plant leaf image features at the first growth stage to obtain the prototype image features of plant leaves at the first growth stage.

[0033] Specifically, in step S21, image feature extraction based on dilated convolution encoding is performed on each of the first growth stage plant leaf images in the subset of the first growth stage plant leaf images to obtain a subset of the first growth stage plant leaf image features. That is, taking the image processing process of the subset of the first growth stage plant leaf images as an example, in order to extract image features such as the color, texture, and shape of the leaves from each plant leaf image in the first growth stage, this application performs dilated convolution encoding on each of the first growth stage plant leaf images in the subset to extract leaf image features, so as to obtain a subset of the first growth stage plant leaf image features. Those of ordinary skill in the art should know that dilated convolution inserts holes in the traditional convolution kernel, enabling the convolution kernel to span more pixels during convolution operations, expanding the receptive field without losing the image resolution, capturing the spatial correlation features between structures such as leaf sheaths and leaf veins, and thus being able to consider both the fine texture of the leaf locally and the overall shape contour of the plant leaf image, improving the accuracy and robustness of image feature extraction.

[0034] Specifically, in step S22, prototype feature extraction and aggregation analysis are performed on the subset of the first growth stage plant leaf image features to obtain the first growth stage plant leaf prototype image features. It should be understood that considering the actual scenario of plant growth monitoring, traditional single-sample feature analysis is difficult to handle the morphological diversity of leaves within the same growth stage. For example, the leaves in the maize germination stage show continuous change features such as the curling angle (15° - 45°) and the degree of leaf sheath wrapping (30% - 80%). Directly using a single leaf feature as the judgment benchmark will increase the misjudgment rate of the model for atypical samples. Therefore, this application further performs essential modeling of the population feature distribution on the subset of the first growth stage plant leaf image features to construct a feature benchmark that can cover the typical morphological changes in this stage, forming the first growth stage plant leaf prototype image features, thereby eliminating the interference of individual sample noise on stage determination. Among them, Figure 4 is a flowchart of sub-step S22 of the image-based plant growth cycle monitoring method according to an embodiment of the present application. As Figure 4 shown, step S22 includes steps: S221, performing image feature independence topological analysis on the subset of the first growth stage plant leaf image features to obtain a first growth stage plant leaf image independence spectral space encoding feature matrix; S222, based on the first growth stage plant leaf image independence spectral space encoding feature matrix, performing feature significance modulation aggregation on the subset of the first growth stage plant leaf image features to obtain the first growth stage plant leaf prototype image features.

[0035] More specifically, in step S221, among them, Figure 5It is a flowchart of sub-step S221 of the image-based plant growth cycle monitoring method according to an embodiment of the present application. As Figure 5 shown, the step S221 includes the steps: S2211, calculating an image feature space independence description operator between any two first growth stage plant leaf image features in a subset of the first growth stage plant leaf image features based on an image feature projection matrix to obtain a first growth stage plant leaf image feature independence spectral space encoding matrix composed of a plurality of plant leaf image feature subspace independence description operators; S2212, performing spectral space activation based on an activation function on the first growth stage plant leaf image feature independence spectral space encoding matrix to obtain the first growth stage plant leaf image independence spectral space encoding feature matrix.

[0036] In a specific example of the present application, the step S2211 is expressed by the formula: where represents a subset of the first growth stage plant leaf image features, , , , and respectively represent the 1st, 2nd, th, th, and th first growth stage plant leaf image features in the subset, is the number of the first growth stage plant leaf image features in the subset, represents calculating the two-norm, and respectively represent different image feature projection matrices, represents the transpose of a vector, represents vector multiplication, represents and the plant leaf image feature subspace independence description operator between.

[0037] That is, by calculating the image feature space independence description operator between the image features of plant leaves in each first growth stage, the coupling relationship of heterogeneous features such as leaf morphology, texture, and color of different leaf images in the image feature projection space is decoupled, and the modal dissociation degree between leaf features is quantified. Here, the image feature space independence description operator represents the independence between the corresponding two plant leaf image features, and the first growth stage plant leaf image feature independence spectrum space encoding matrix formed based on the spatial topological structure arrangement not only retains the relative position information of each leaf image feature in the first growth stage but also reveals the continuous law of leaf feature evolution. Through the capture of this global interaction mode, the model can break through the limitations of traditional single-frame static analysis, achieve a fine division of the growth stage transition period, and thus upgrade the growth cycle monitoring from discrete static image feature determination to continuous dynamic tracking.

[0038] In a specific example of the present application, the step S2212 is expressed by the formula: Wherein, represents the sigmoid activation function, 、 、 and respectively represent the and 、 and 、 and 、 and the independence description operator of the plant leaf image feature subspace between, represents the first growth stage plant leaf image independence spectrum space encoding feature matrix.

[0039] That is, the present application further introduces a normalization function, reconstructs the significance distribution of the feature space through non-linear mapping, adjusts the activation degree of the plant leaf image feature subspace independence description operator, enhances the interaction intensity between plant leaf image features with significant associations, and at the same time suppresses the interference to non-significant associated features. Through the activation operation, the significance information in the first growth stage plant leaf image feature independence spectrum space encoding matrix is strengthened, while irrelevant or redundant information is suppressed. The generated first growth stage plant leaf image independence spectrum space encoding feature matrix not only improves the expression ability of modal features but also provides optimized modal significance information for subsequent node feature modulation.

[0040] More specifically, for the step S222, wherein, Figure 6The flowchart of sub-step S222 of the image-based plant growth cycle monitoring method according to an embodiment of the present application. As Figure 6 shown, the step S222 includes steps: S2221, inputting each first growth stage plant leaf image feature in the subset of the first growth stage plant leaf image features and the first growth stage plant leaf image independent spectral space encoding feature matrix into an image feature saliency modulation module to obtain a subset of independent modulation first growth stage plant leaf image features; S2222, inputting the subset of the independent modulation first growth stage plant leaf image features into a feature dynamic clustering network to obtain the first growth stage plant leaf prototype image features.

[0041] In a specific example of the present application, the step S2221 is expressed by the formula: where represents the feature scale value of and represents the corresponding independent modulation first growth stage plant leaf image feature.

[0042] That is, by using the first growth stage plant leaf image independent spectral space encoding feature matrix to perform feature modulation on each first growth stage plant leaf image feature, each first growth stage plant leaf image feature is made to integrate into the global image feature interaction topological information, so as to re-encode the originally independent leaf image features in the spectral space, forming a feature representation with rich context information, obtaining a subset of independent modulation first growth stage plant leaf image features, and making the continuous representation of the leaf image features within the stage more accurate. By explicitly modeling the synergistic effect between features, the efficiency and accuracy of subsequent clustering analysis are optimized, the computational complexity of feature dynamic clustering is reduced, and advantageous original feature materials are provided for subsequent feature dynamic clustering.

[0043] In a specific example of the present application, the step S2222 includes: calculating the leaf feature stability factor of each independent modulation first growth stage plant leaf image feature based on the feature distribution of each independent modulation first growth stage plant leaf image feature in the subset of independent modulation first growth stage plant leaf image features to obtain a subset of first growth stage plant leaf image feature stability factors; performing Softmax-based normalization processing on the subset of first growth stage plant leaf image feature stability factors to obtain a subset of first growth stage plant leaf image feature dynamic aggregation weight coefficients; and performing weighted aggregation on the subset of independent modulation first growth stage plant leaf image features based on the subset of first growth stage plant leaf image feature dynamic aggregation weight coefficients to obtain the first growth stage plant leaf prototype image feature, which is expressed by the formula: Wherein, represents calculating the feature kurtosis, represents the offset factor, represents the first growth stage plant leaf image feature stability factor of represents the feature mean of the independent modulation first growth stage plant leaf image feature, represents the feature variance of the independent modulation first growth stage plant leaf image feature, represents calculating the expected value, represents the normalization function, represents the first growth stage plant leaf image feature dynamic aggregation weight coefficient of represents the first growth stage plant leaf prototype image feature.

[0044] That is, by constructing a feature dynamic clustering network with context time-series evolution perception, measuring the stability and importance of each independent modulation first growth stage plant leaf image feature, and converting the stability factor into a probability distribution, not only dynamic weight allocation is realized, but also a mapping relationship between feature importance and growth stage evolution is established, so that the signal-to-noise ratio of the feature space is systematically improved, the discriminability of feature representation is enhanced, and the weighted aggregation process forms a global feature prototype representation by fusing the time-series evolution information of multiple leaf image features, enabling the system to capture subtle feature changes in the stage transition period.

[0045] In a preferred example of the present application, based on the feature distributions of each of the independently modulated first growth stage plant leaf image features in the subset of the independently modulated first growth stage plant leaf image features, calculating the leaf feature stability factors of each of the independently modulated first growth stage plant leaf image features to obtain a subset of the first growth stage plant leaf image feature stability factors, including: performing a covariance-orthogonal composite transformation on the independently modulated first growth stage plant leaf image features to obtain an offset factor; calculating the feature kurtosis of the independently modulated first growth stage plant leaf image features, dividing it by two and adding it to the offset factor to obtain the first growth stage plant leaf image feature stability factor, which is expressed by the formula: Wherein, represents the first growth stage plant leaf image feature alignment space matrix, represents the identity matrix, represents the corresponding spatially aligned independently modulated first growth stage plant leaf image features, represents vector subtraction, represents the first growth stage plant leaf image feature covariance matrix, represents the scaling factor, represents the vector inner product.

[0046] That is, the present application takes into account that in plant growth monitoring, when the dynamic clustering network strengthens the key leaf features for feature aggregation by the significant relationship between the independently modulated first growth stage plant leaf image features, the base mismatch is likely to occur between the clustering center and the edge of the feature distribution. Therefore, preferably, the present application dynamically adjusts the mapping intensity by introducing the offset factor for downward alignment. Specifically, first, the first growth stage plant leaf image feature alignment space matrix is obtained through the orthogonal constraint decomposition of the identity matrix, and then the isometric mapping transformation is performed on the independently modulated first growth stage plant leaf image features based on the first growth stage plant leaf image feature alignment space matrix, and then the covariance matrix of the independently modulated first growth stage plant leaf image features is introduced, at In addition to the statistical geometric structure represented by the mean direction and variance representation, adjust the local geometric structure of the plant leaf image features in the first growth stage of the independence modulation, and rotationally / reflectively eliminate the random local differences of the basis through the inner product basification representation based on the orthogonal constraint, so as to achieve downward spatial arrangement alignment, and eliminate the dynamic cross-domain inconsistency caused by emphasizing the significant relationship of the plant leaf image features in the first growth stage of the independence modulation during the feature clustering process.

[0047] In the above image-based plant growth cycle monitoring method, in step S3, obtain the plant image to be recognized. That is, by obtaining the plant image to be recognized, perform leaf feature matching analysis with the plant leaf prototype image features in the first to fifth growth stages to determine the growth stage of the plant image to be recognized.

[0048] In the above image-based plant growth cycle monitoring method, in step S4, extract leaf features from the plant image to be recognized to obtain the leaf image features to be recognized. Among them, Figure 7 is a flowchart of sub-step S4 of the image-based plant growth cycle monitoring method according to an embodiment of the present application. As Figure 7 shown, step S4 includes steps: S41, input the plant image to be recognized into a leaf region of interest recognizer based on the Yolo model to obtain a leaf ROI image to be recognized; S42, perform image feature extraction based on dilated convolution coding on the leaf ROI image to be recognized to obtain the leaf image features to be recognized.

[0049] Specifically, in step S41, input the plant image to be recognized into a leaf region of interest recognizer based on the Yolo model to obtain a leaf ROI image to be recognized. Specifically, since the plant image to be recognized actually collected usually contains a large amount of background information, such as soil, surrounding weeds, planting facilities, etc., these irrelevant information will increase the complexity and computational amount of subsequent feature analysis, and may interfere with the extraction of key features of plant leaves, resulting in inaccurate analysis results. To this end, the present application uses the Yolo model to construct a leaf region of interest recognizer to perform region of interest recognition on the plant image to be recognized. The Yolo (You Only Look Once) model is an advanced object detection algorithm. Based on the principle of deep learning, through learning a large amount of labeled data, it can quickly identify the category and location of the target object in the image. By using the Yolo model to process the plant image to be recognized in the present application, the region where the leaf is located can be quickly and accurately recognized, the plant leaf part can be separated from the complex plant image to be recognized, and the interference of background information can be eliminated, so as to obtain a leaf ROI image to be recognized that only contains leaf information, simplify the subsequent analysis process, and improve the accuracy and efficiency of the analysis.

[0050] Specifically, in step S42, image feature extraction based on dilated convolution coding is performed on the ROI image of the leaf to be recognized to obtain the image features of the leaf to be recognized. It should be understood that in order to effectively match the ROI image of the leaf to be recognized with the image features of the prototype images of plant leaves in the first to fifth growth stages, the present application also uses dilated convolution coding technology to process the ROI image of the leaf to be recognized, so as to utilize the large-scale receptive field characteristics of dilated convolution to capture key features such as the color, texture, shape, and spatial structure of the leaf, thereby obtaining the image features of the leaf to be recognized and providing accurate and comprehensive feature data for subsequent leaf feature matching analysis.

[0051] In the above method for monitoring the plant growth cycle based on images, in step S5, image feature matching analysis is performed on the image features of the leaf to be recognized and the image features of the prototype images of plant leaves in the first to fifth growth stages to obtain a matching analysis result. In a specific example of the present application, the cosine similarity between the image features of the leaf to be recognized and the image features of the prototype images of plant leaves in the first to fifth growth stages is calculated to obtain the matching analysis result. As an index for measuring the consistency of the directions of two vectors, the closer the value of the cosine similarity is to 1, the more consistent the directions of the two vectors are, that is, the higher the similarity between the image features of the leaf to be recognized and the image features of the prototype image of plant leaves in a certain growth stage; the closer the cosine similarity is to 0, the more inconsistent the directions of the two vectors are, that is, the lower the similarity between the image features of the leaf to be recognized and the image features of the prototype image of plant leaves in a certain growth stage. According to the calculated cosine similarity, it can be determined which growth stage the prototype image features of the leaf image to be recognized are closest to, thereby determining the growth stage of the plant image to be recognized.

[0052] In the above method for monitoring the plant growth cycle based on images, in step S6, based on the matching analysis result, the growth stage of the plant image to be recognized is determined. Specifically, if the matching analysis result shows that the cosine similarity between the image features of the leaf to be recognized and the image features of the prototype image of plant leaves in a certain growth stage is the highest and this cosine similarity exceeds a preset similarity threshold, it is determined that the plant image to be recognized is in this growth stage. The preset similarity threshold is an empirical value obtained based on a large amount of training data and actual tests, and is used to ensure the accuracy and reliability of the determination result. If the cosine similarity between the image features of the leaf to be recognized and the image features of the prototype images of plant leaves in all growth stages does not exceed the preset similarity threshold, it is determined that the growth stage of the plant image to be recognized is not clear, and it may be necessary to re-collect the image or use manual review and other auxiliary means for judgment. Through the above method, the present application can realize automatic, rapid, and accurate recognition of the plant growth stage, help managers timely understand the growth situation of plants, take corresponding measures for management and adjustment, and promote the healthy growth of plants and increase yields.

[0053] In summary, the image-based plant growth cycle monitoring method according to the embodiments of the present application is elucidated. It pre-collects a set of plant leaf images at different growth stages, and introduces an image processing technology based on deep learning to perform image feature extraction and prototype feature extraction and aggregation analysis on the set of plant leaf images at each growth stage, so as to capture the prototype image features of plant leaves at each growth stage. Furthermore, based on the prototype image features of plant leaves at each growth stage, leaf feature matching analysis is performed on the plant image to be recognized, so as to quickly judge the current growth stage of the plant, which is convenient for targeted fertilization management of the plant. In this way, intelligent monitoring of the plant growth cycle can be realized, providing effective guidance for the fertilization management of the plant, and further improving the scientificity and accuracy of fertilization management.

[0054] To verify the effectiveness of the method of the present application, a comparative experiment was carried out with the prior art in terms of misjudgment rate, accuracy rate, processing time, and environmental anti-interference ability. The prior art one (Tong Shuyuan, Song Fengbin. Application of SPAD value in nitrogen nutrition diagnosis and recommended fertilization of maize [J]. Systems Science and Comprehensive Studies in Agriculture, 2009, 25(2): 233-238.) evaluates the growth stage and nitrogen nutrition status of maize based on the SPAD value of maize leaves, and recommends the nitrogen fertilizer application rate accordingly, and proposes a variable nitrogen application strategy. SPAD (Soil Plant Analysis Development) measures the absorption rate of leaves to specific wavelengths (such as red light and near-infrared light) through an optical sensor to detect the chlorophyll content of leaves, and then infers whether the plant is nitrogen-deficient. The prior art two (Beijing Research Center for Intelligent Equipment for Agriculture. Control method for integrated water and fertilizer irrigation system and integrated water and fertilizer irrigation system: CN201911046157.7 [P]. 2020-01-17.) analyzes the water supply status, fertilizer supply status and growth period of crops based on the deep network model of ResNet-101, and controls the operation of the integrated water and fertilizer irrigation device accordingly. The comparison results are shown in Table 1: Table 1 Comparison results of the plant growth cycle monitoring method of the present application and the prior art As can be seen from Table 1, the misjudgment rate (3.2%) of the method of the present application is much lower than that of SPAD (18.5%) and ResNet-based image classification (12.7%), mainly due to the prototype feature extraction and aggregation analysis and dynamic clustering technology. The method of the present application reduces the influence of light and background noise (SPAD is easily interfered by leaf color distortion) through comprehensive analysis of key features such as the color, texture, shape, and spatial structure of leaves, and has stronger environmental anti-interference ability.

[0055] Furthermore, a plant growth cycle monitoring system based on images is also provided.

[0056] Figure 8 FIG. is a block diagram of a plant growth cycle monitoring system based on images according to an embodiment of the present application. As Figure 8 shown, the plant growth cycle monitoring system 100 based on images according to an embodiment of the present application includes: an image acquisition module 110, configured to acquire plant leaf images at different growth stages and perform image set segmentation according to the growth stages to obtain subsets of plant leaf images at the first to fifth growth stages; a plant leaf prototype feature extraction module 120, configured to perform image feature extraction and prototype feature extraction and aggregation analysis on each subset of plant leaf images at the first to fifth growth stages in the subsets of plant leaf images at the first to fifth growth stages to obtain plant leaf prototype image features at the first to fifth growth stages; a plant image to be recognized acquisition module 130, configured to acquire a plant image to be recognized; a plant image to be recognized feature extraction module 140, configured to extract leaf features from the plant image to be recognized to obtain a plant leaf image feature to be recognized; a feature matching analysis module 150, configured to perform image feature matching analysis on the plant leaf image feature to be recognized and the plant leaf prototype image features at the first to fifth growth stages to obtain a matching analysis result; and a growth stage determination module 160, configured to determine the growth stage of the plant image to be recognized based on the matching analysis result.

[0057] Here, those skilled in the art can understand that the specific operations of the above-mentioned modules in the plant growth cycle monitoring system based on images have been described in detail in the description of the plant growth cycle monitoring method based on images above, and therefore, the repeated description thereof will be omitted. Figures 1 to 7 of the plant growth cycle monitoring method based on images, and therefore, the repeated description thereof will be omitted.

[0058] Figure 9 FIG. is an automatic fertilization flow chart of an automatic plant fertilization management system according to an embodiment of the present application. After preprocessing the collected images, analyze and extract their features, compare them with the pre-stored data in the database, determine the current growth stage of the plant, and retrieve the data on the required nutrients in the database; calculate the actual chemical fertilizer dosage according to the amount of required nutrients, and finally generate a fertilization control signal to achieve automatic fertilization. As Figure 9 shown, first, a growth cycle monitoring link is performed. In the first step, image acquisition is performed; the system will perform the growth cycle monitoring step, and the camera deployed in the planting area 3 will collect plant images at regular intervals, and the original data will be transmitted to the edge computing node; in the second step, the images are preprocessed, and Gaussian filtering, resolution normalization (and CLAHE contrast enhancement (8×8 block to enhance the visibility of leaf texture) are sequentially performed, and at the same time, the main plant body area is segmented by a lightweight U-Net model to eliminate background interference.

[0059] Step 3: Conduct model classification and data comparison: ① The preprocessed images are input into the improved YOLOv8n classification model, which embeds the CBAM attention module in the original architecture to enhance the leaf feature extraction ability.

[0060] ② The model outputs include growth stage labels (five categories such as germination stage, growth stage, etc.) and confidence values. When the confidence is below 90%, the FAO data calibration process is automatically triggered: call the FAO crop stage API, and use the multilingual BERT model to calculate the semantic similarity between the model output stage and the FAO standard terms (e.g., the cosine similarity between "flowering stage" and "Flowering Phase" reaches 0.92). If the match is successful, the FAO standardized stage name is adopted. The final calibration results are written into the SQL database together with the original data, and the nitrogen, phosphorus, and potassium (hereinafter referred to as NPK) requirement data are output (obtained by associatively querying the FAO nutrient requirement interface).

[0061] ③ If there is an interruption in image transmission or an abnormality in model inference during this period, the system automatically enables the standby MobileNetV2 lightweight model and records the device status code, and at the same time sends an alarm message to the operation and maintenance platform. Then the growth stage information is transmitted to the fertilizer calculation module.

[0062] The system needs to calculate the fertilizer dosage. First, logical judgment is performed based on the obtained growth stage information to determine whether the existing nutrient solution or chemical fertilizer can directly meet the growth requirements of this plant. If it can, the fertilizer dilution ratio calculation stage is directly carried out. If not, the ratio calculation of multiple fertilizers is required: ① According to the elemental demand to the compound demand: Input the amount of pure elements required for the plant growth stage: N (kg), P (kg), K (kg).

[0063] Convert P and K to the corresponding oxide demands: ; ; ② Calculate the compound fertilizer dosage according to the single compound demand: Suppose two compound fertilizers A ( : ) and compound fertilizer B ( ) are selected, and a system of equations is established: Use matrix operations or optimization algorithms to solve for x (the dosage of compound fertilizer A) and y (the dosage of compound fertilizer B). If there is no solution, prompt to adjust the compound fertilizer combination or supplement single-element fertilizers.

[0064] ③ Calculate the dilution solution concentration according to the compound fertilizer dosage.

[0065] The following is an example: The required amounts of nitrogen, phosphorus, and potassium elements obtained by comparison from the database are as follows: Nitrogen (N): 5 mg, Phosphorus (P): 3 mg, Potassium (K): 4 kg; Optional compound fertilizers: Compound fertilizer A: (20 - 10 - 10): containing 20% N, 10% , 10% Compound fertilizer B: (10 - 20 - 10): containing 10% N, 20% , 10% First step, calculate the required amounts of oxides: Required amount = = ≈ 6.98 mg Required amount = = ≈ 4.82 mg Second step, solve the system of equations According to the judgment basis for the solutions of the system of linear equations, it can be known that the system of equations has no solution. Therefore, it is necessary to introduce compound fertilizer 3 containing only one element (only containing 50% ), re - allocate the requirements, first use compound fertilizers A and B to meet the N and requirements, and the remaining is supplemented by compound fertilizer C, and the system of equations is corrected.

[0066] The solution is: x = 10.4 mg, y = 28.2 mg; Then calculate the amount of compound fertilizer 3 providing : The provided by compound fertilizers A + B: The to be supplemented: The amount of compound fertilizer C: Steps for fertilization control: ① Through the previous calculations, the required amounts of each fertilizer are obtained, and the volume of the diluted nutrient solution actually required is obtained through calculation.

[0067] ② Calculate the working time of the valve port: Based on the rated flow rate of the valve and the required volume of nutrient solution, calculate the opening time of the valve port, then compensate for the system delay time, and finally obtain the actual working time.

[0068] ③ Control the valve opening: Adopt a parallel control mode to enable the nutrient solution to be fully mixed, improve the working efficiency at the same time, and use a built-in timer to control the working time.

[0069] ④ Generate a work report: Output the fertilization result according to the application rate of the fertilizer and the working time. If there is a problem, a warning will be sent for manual intervention.

[0070] The basic principles of the present invention have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, advantages, effects, etc. mentioned in the present invention are only examples and not limitations. It cannot be considered that these advantages, advantages, effects, etc. are essential for each embodiment of the present invention. In addition, the specific details of the above embodiments are only for the purpose of illustration and easy understanding, rather than limitations. The above details do not limit the present invention to necessarily adopt the above specific details to implement.

[0071] In the above embodiments, the descriptions of each embodiment have their own focuses. For parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments. In the several embodiments provided by the present invention, it should be understood that the disclosed system and method can be implemented in other ways. For example, the system embodiments described above are only illustrative. For example, the unit division is only a logical function division, and there can be other division methods in actual implementation. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0072] For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and can be implemented in other specific forms without departing from the spirit or basic characteristics of the present invention. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, it is intended to cover all changes falling within the meaning and scope of the equivalent elements of the claims in the present invention. Any associated drawing marks in the claims should not be regarded as limiting the claimed rights.

[0073] In addition, obviously the word "including" does not exclude other units or steps, and the singular does not exclude the plural. The multiple units stated in the system claims can also be implemented by one unit through software or hardware.

[0074] Finally, it should be noted that the above description has been given for purposes of illustration and description. In addition, the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A plant growth cycle monitoring method based on an image, characterized in that: include: Collecting plant leaf images at different growth stages and dividing the image set according to the growth stage to obtain a subset of plant leaf images at the first to fifth growth stages; Performing image feature extraction and prototype feature extraction aggregation analysis on subsets of plant leaf images of each growth stage in the subsets of plant leaf images of the first to fifth growth stages to obtain prototype image features of plant leaves of the first to fifth growth stages; Acquire a plant image to be identified; Extracting leaf features from the to-be-identified plant image to obtain to-be-identified leaf image features; Performing image feature matching analysis on the image features of the leaf to be identified and the prototype image features of the plant leaves in the first to fifth growth stages to obtain a matching analysis result; Based on the matching analysis result, the growth stage of the plant image to be identified is determined.

2. The image-based plant growth cycle monitoring method according to claim 1, characterized in that: The growth stages include germination stage, growth stage, flowering stage, fruiting stage and senescence stage.

3. The image-based plant growth cycle monitoring method according to claim 1, characterized in that: Performing image feature extraction and prototype feature extraction aggregation analysis on a subset of plant leaf images at each growth stage in the subset of plant leaf images at the first to fifth growth stages to obtain prototype image features of plant leaves at the first to fifth growth stages, including: Performing image feature extraction based on dilated convolution coding on each first growth stage plant leaf image in the subset of first growth stage plant leaf images to obtain a subset of first growth stage plant leaf image features; Prototype feature extraction and aggregation analysis is performed on a subset of the image features of the plant leaves in the first growth stage to obtain prototype image features of the plant leaves in the first growth stage.

4. The image-based plant growth cycle monitoring method according to claim 3, characterized in that: Performing prototype feature extraction and aggregation analysis on a subset of the image features of the plant leaves at the first growth stage to obtain prototype image features of the plant leaves at the first growth stage includes: Performing image feature independence topological analysis on a subset of the image features of the plant leaves at the first growth stage to obtain an independence spectral space encoding feature matrix of the plant leaves at the first growth stage; Based on the independent spectral space encoding feature matrix of the plant leaf image at the first growth stage, feature significance modulation aggregation is performed on a subset of the plant leaf image features at the first growth stage to obtain the prototype image features of the plant leaf at the first growth stage.

5. The image-based plant growth cycle monitoring method according to claim 4, characterized in that: Performing image feature independence topological analysis on a subset of the image features of the plant leaves at the first growth stage to obtain an independence spectral space encoding feature matrix of the plant leaves at the first growth stage, comprising: Calculating the image feature space independence description operator between any two first growth stage plant leaf image features in the subset of the first growth stage plant leaf image features based on the image feature projection matrix to obtain a first growth stage plant leaf image feature independence spectral space encoding matrix composed of a plurality of plant leaf image feature subspace independence description operators; The spectral space encoding matrix of the independence of the plant leaf image features at the first growth stage is activated in a spectral space based on an activation function to obtain the spectral space encoding feature matrix of the independence of the plant leaf image at the first growth stage.

6. The image-based plant growth cycle monitoring method according to claim 5, characterized in that: Based on the independent spectral space encoding feature matrix of the plant leaf image at the first growth stage, a subset of the plant leaf image features at the first growth stage is subjected to feature saliency modulation aggregation to obtain the prototype image features of the plant leaf at the first growth stage, including: Input each first growth stage plant leaf image feature in the subset of the first growth stage plant leaf image features and the first growth stage plant leaf image independence spectrum space encoding feature matrix into an image feature saliency modulation module to obtain a subset of the first growth stage plant leaf image features of independence modulation; The subset of the independence-modulated plant leaf image features at the first growth stage is input into a feature dynamic clustering network to obtain prototype image features of the plant leaf at the first growth stage.

7. The image-based plant growth cycle monitoring method according to claim 6, characterized in that: Inputting the subset of the independence-modulated first growth stage plant leaf image features into a feature dynamic clustering network to obtain the first growth stage plant leaf prototype image features, including: Based on the feature distribution of each independent modulated first growth stage plant leaf image feature in the subset of the independent modulated first growth stage plant leaf image features, calculating the leaf feature stability factor of each independent modulated first growth stage plant leaf image feature to obtain a subset of the first growth stage plant leaf image feature stability factor; Performing a Softmax-based normalization process on a subset of the stability factors of the plant leaf image features at the first growth stage to obtain a subset of dynamic aggregation weight coefficients of the plant leaf image features at the first growth stage; Based on a subset of the dynamic aggregation weight coefficients of the plant leaf image features in the first growth stage, a subset of the independent modulated plant leaf image features in the first growth stage is weighted aggregated to obtain the prototype image features of the plant leaves in the first growth stage.

8. The image-based plant growth cycle monitoring method according to claim 7, characterized in that: Based on the feature distribution of each independent modulated first growth stage plant leaf image feature in the subset of the independent modulated first growth stage plant leaf image features, calculating the leaf feature stability factor of each independent modulated first growth stage plant leaf image feature to obtain the subset of the first growth stage plant leaf image feature stability factor, including: Performing a covariance-orthogonal composite transformation on the image features of the plant leaves in the first growth stage of the independence modulation to obtain an offset factor; The characteristic kurtosis of the independence-modulated plant leaf image feature in the first growth stage is calculated, divided by two, and then added to the offset factor to obtain the stability factor of the plant leaf image feature in the first growth stage.

9. The image-based plant growth cycle monitoring method according to claim 8, characterized in that: Extracting leaf features from the to-be-identified plant image to obtain to-be-identified leaf image features includes: Inputting the plant image to be identified into a leaf region of interest identifier based on the Yolo model to obtain a leaf ROI image to be identified; Image feature extraction based on hole convolution coding is performed on the leaf ROI image to be identified to obtain the image features of the leaf to be identified.

10. An image-based plant growth cycle monitoring system, characterized in that: include: An image acquisition module, used for acquiring plant leaf images at different growth stages and dividing the image set according to the growth stage to obtain a subset of plant leaf images at the first to fifth growth stages; A plant leaf prototype feature extraction module, used for performing image feature extraction and prototype feature extraction aggregation analysis on a subset of plant leaf images of each growth stage in the subset of plant leaf images of the first to fifth growth stages to obtain prototype image features of plant leaves of the first to fifth growth stages; A module for acquiring images of plants to be identified, used for acquiring images of plants to be identified; A feature extraction module for the plant image to be identified, used for extracting leaf features from the plant image to be identified to obtain leaf image features to be identified; A feature matching analysis module, used for performing image feature matching analysis on the image features of the leaf to be identified and the image features of the prototype plant leaves in the first to fifth growth stages to obtain a matching analysis result; A growth stage determination module is used to determine the growth stage of the plant image to be identified based on the matching analysis result.

Images