Image-based Plant Growth Cycle Monitoring System and Method

Abstract

This application relates to the field of intelligent monitoring technology. Specifically, it discloses a plant growth cycle monitoring system and method based on images. It pre-collects image sets of plant leaves at different growth stages, and introduces image processing technology based on deep learning to perform image feature extraction and prototype feature extraction and aggregation analysis on the image sets of plant leaves at each growth stage, so as to capture the prototype image features of plant leaves at each growth stage. Furthermore, based on the prototype image features of plant leaves at each growth stage, leaf feature matching analysis is performed on the plant images to be recognized, so as to quickly judge the current growth stage of the plant, facilitating targeted fertilization management of the plant. In this way, intelligent monitoring of the plant growth cycle can be achieved, providing effective guidance for the fertilization management of plants, and further improving the scientificity and accuracy of fertilization management.

Description

Technical Field

[0001] This application relates to the technical field of intelligent monitoring, and more specifically, to an image-based plant growth cycle monitoring system and method. Background Art

[0002] In the modern agricultural production system, scientific and precise fertilization management is the core link to improve crop yield and quality and ensure the sustainable development of agriculture. Traditional fertilization management mainly relies on manual experience judgment or soil nutrient detection, which has many drawbacks. For example, manual experience judgment requires a large amount of manpower and time, has the lag of subjective experience, and is prone to untimely fertilization timing; while soil nutrient detection requires destructive sampling and cannot reflect the actual absorption status of plants. According to FAO statistics, due to misjudgment of growth stages, about 27% of nitrogen fertilizers are inefficiently lost globally every year.

[0003] In recent years, the plant nutrient diagnosis technology based on machine vision has provided a new way for dynamic fertilization. Existing methods mostly evaluate the nutritional status through leaf color (such as SPAD value estimation) for fertilization management, but this method ignores the strong correlation between plant nutrient requirements and growth stages. For example, the nitrogen requirement during the tillering stage of rice is 40 - 60% higher than that during the heading stage. If fertilization is only based on leaf color, it is easy to cause the dislocation between fertilization recommendations and real needs, thus affecting crop growth and yield.

[0004] Since the nutrient requirements of plants vary significantly at different growth stages, the prerequisite for precise fertilization is to accurately control the plant growth cycle. However, most of the existing plant growth cycle monitoring models only construct binary classifiers for specific growth stages and lack the ability of full-cycle continuous monitoring; secondly, the growth of plants is a continuous and dynamic process, and most of the existing methods are based on the analysis of a single static image, and the progressive feature correlation of growth stages is not fully considered during feature learning, resulting in a relatively high cross-stage misjudgment rate.

[0005] Therefore, an optimized image-based plant growth cycle monitoring system and method are expected. Summary of the Invention

[0006] To solve the above technical problems, the present application is proposed. Embodiments of the present application provide an image-based plant growth cycle monitoring system and method, which pre-collect plant leaf image sets at different growth stages, and introduce deep learning-based image processing technology to perform image feature extraction and prototype feature extraction and aggregation analysis on the plant leaf image sets at each growth stage to capture the prototype image features of plant leaves at each growth stage. Furthermore, based on the prototype image features of plant leaves at each growth stage, leaf feature matching analysis is performed on the plant image to be recognized, so as to quickly determine the current growth stage of the plant, facilitating targeted fertilization management of the plant. In this way, intelligent monitoring of the plant growth cycle can be achieved, providing effective guidance for the fertilization management of the plant, and further improving the scientificity and accuracy of fertilization management.

[0007] According to one aspect of the present application, there is provided an image-based plant growth cycle monitoring method, which includes:

[0008] Collect plant leaf images at different growth stages and perform image set segmentation according to the growth stages to obtain subsets of plant leaf images at the first to fifth growth stages;

[0009] Perform image feature extraction and prototype feature extraction and aggregation analysis on the subsets of plant leaf images at each growth stage among the subsets of plant leaf images at the first to fifth growth stages to obtain prototype image features of plant leaves at the first to fifth growth stages;

[0010] Obtain the plant image to be recognized;

[0011] Extract leaf features from the plant image to be recognized to obtain the leaf image features to be recognized;

[0012] Perform image feature matching analysis on the leaf image features to be recognized and the prototype image features of plant leaves at the first to fifth growth stages to obtain a matching analysis result;

[0013] Based on the matching analysis result, determine the growth stage of the plant image to be recognized.

[0014] According to another aspect of the present application, there is provided an image-based plant growth cycle monitoring system, which includes:

[0015] An image acquisition module, configured to collect plant leaf images at different growth stages and perform image set segmentation according to the growth stages to obtain subsets of plant leaf images at the first to fifth growth stages;

[0016] A plant leaf prototype feature extraction module for performing image feature extraction and prototype feature extraction and aggregation analysis on each subset of plant leaf images at the first to fifth growth stages in a subset of plant leaf images at the first to fifth growth stages to obtain plant leaf prototype image features at the first to fifth growth stages;

[0017] An image acquisition module of a plant to be recognized for acquiring an image of the plant to be recognized;

[0018] A feature extraction module of a leaf image of a plant to be recognized for extracting leaf features from the image of the plant to be recognized to obtain leaf image features of the plant to be recognized;

[0019] A feature matching analysis module for performing image feature matching analysis on the leaf image features of the plant to be recognized and the plant leaf prototype image features at the first to fifth growth stages to obtain a matching analysis result;

[0020] A growth stage determination module for determining the growth stage of the image of the plant to be recognized based on the matching analysis result.

[0021] Compared with the prior art, the image-based plant growth cycle monitoring system and method provided by the present application pre-collects a set of plant leaf images at different growth stages, and introduces an image processing technology based on deep learning to perform image feature extraction and prototype feature extraction and aggregation analysis on each set of plant leaf images at each growth stage to capture the plant leaf prototype image features at each growth stage. Furthermore, based on the plant leaf prototype image features at each growth stage, leaf feature matching analysis is performed on the image of the plant to be recognized, so as to quickly determine the current growth stage of the plant, facilitating targeted fertilization management of the plant. In this way, intelligent monitoring of the plant growth cycle can be realized, providing effective guidance for the fertilization management of the plant, and further improving the scientificity and accuracy of fertilization management. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] By describing the embodiments of the present application in more detail in conjunction with the drawings, the above and other objects, features, and advantages of the present application will become more obvious. The drawings are used to provide a further understanding of the embodiments of the present application, and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application, and do not constitute a limitation to the present application. In the drawings, the same reference numerals generally represent the same components or steps.

[0023] Figure 1 FIG. is a flowchart of an image-based plant growth cycle monitoring method according to an embodiment of the present application.

[0024] Figure 2 FIG. is a schematic diagram of data flow of an image-based plant growth cycle monitoring method according to an embodiment of the present application.

[0025] Figure 3 It is a flowchart of sub-step S2 of the image-based plant growth cycle monitoring method according to an embodiment of the present application.

[0026] Figure 4 It is a flowchart of sub-step S22 of the image-based plant growth cycle monitoring method according to an embodiment of the present application.

[0027] Figure 5 It is a flowchart of sub-step S221 of the image-based plant growth cycle monitoring method according to an embodiment of the present application.

[0028] Figure 6 It is a flowchart of sub-step S222 of the image-based plant growth cycle monitoring method according to an embodiment of the present application.

[0029] Figure 7 It is a flowchart of sub-step S4 of the image-based plant growth cycle monitoring method according to an embodiment of the present application.

[0030] Figure 8 It is a block diagram of the image-based plant growth cycle monitoring system according to an embodiment of the present application.

[0031] Figure 9 It is an automatic fertilization flowchart of the automatic plant fertilization management system according to an embodiment of the present application. Detailed implementation manners

[0032] As shown in the present application and the claims, unless the context clearly indicates otherwise, words such as "a", "an", "one" and / or "the" are not specifically singular and may also include the plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0033] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, however, any number of different modules can be used and run on the user terminal and / or the server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0034] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the operations before or below do not necessarily need to be executed precisely in order. On the contrary, as needed, various steps can be executed in reverse order or simultaneously. At the same time, other operations can also be added to these processes, or one or several operations can be removed from these processes.

[0035] Next, exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. It should be understood that the present application is not limited by the exemplary embodiments described herein.

[0036] It should be noted that in the present application, all actions of obtaining data are carried out on the premise of complying with the corresponding data protection regulations and policies of the country where it is located and obtaining authorization from the corresponding device owner.

[0037] To address the technical problems described in the above background art, the present application proposes an image-based plant growth cycle monitoring method. It pre-collects a set of plant leaf images at different growth stages and introduces deep learning-based image processing technology to perform image feature extraction and prototype feature extraction and aggregation analysis on each set of plant leaf images at different growth stages to capture the prototype image features of plant leaves at each growth stage. Furthermore, based on the prototype image features of plant leaves at each growth stage, leaf feature matching analysis is performed on the plant image to be recognized, so as to quickly determine the current growth stage of the plant, facilitating targeted fertilization management of the plant. In this way, intelligent monitoring of the plant growth cycle can be achieved, providing effective guidance for the fertilization management of plants, and thus improving the scientificity and accuracy of fertilization management.

[0038] Figure 1 FIG. is a flowchart of an image-based plant growth cycle monitoring method according to an embodiment of the present application. Figure 2 FIG. is a schematic diagram of data flow of an image-based plant growth cycle monitoring method according to an embodiment of the present application. As Figure 1 and Figure 2 shown, the image-based plant growth cycle monitoring method includes the steps of: S1, collecting plant leaf images at different growth stages and segmenting the image set according to the growth stage to obtain subsets of plant leaf images at the first to fifth growth stages; S2, performing image feature extraction and prototype feature extraction and aggregation analysis on each subset of plant leaf images at the first to fifth growth stages to obtain prototype image features of plant leaves at the first to fifth growth stages; S3, obtaining a plant image to be recognized; S4, extracting leaf features from the plant image to be recognized to obtain leaf image features to be recognized; S5, performing image feature matching analysis on the leaf image features to be recognized and the prototype image features of plant leaves at the first to fifth growth stages to obtain a matching analysis result; S6, based on the matching analysis result, determining the growth stage of the plant image to be recognized.

[0039] In the above-mentioned image-based plant growth cycle monitoring method, in step S1, plant leaf images at different growth stages are collected and segmented according to the growth stages to obtain subsets of plant leaf images at the first to fifth growth stages. It should be understood that when the plant is in the transition period between adjacent growth stages, the morphological characteristics of the leaves often show a progressive evolution, resulting in the fact that the feature analysis method using a single static image is very likely to misjudge the leaves in the heading stage as those in the jointing stage. Therefore, in this application, the whole growth cycle of the plant from germination to senescence is continuously monitored, and the complete growth cycle is divided into five key stages according to plant physiology standards. The growth stages include the germination stage, the growth stage, the flowering stage, the fruiting stage, and the senescence stage, forming subsets of plant leaf images at the first to fifth growth stages. At the same time, it is ensured that each subset contains a sufficient number of sample images, which can cover the morphological variation range of the leaves in this growth stage and fully reflect the change law of the leaf characteristics in this growth stage, so as to provide a comprehensive data basis for the subsequent identification and analysis of plant growth stages.

[0040] Specifically, in the actual operation process, according to plant physiology standards, the complete growth cycle is divided into five key stages: the germination stage, the growth stage, the flowering stage, the fruiting stage, and the senescence stage. Each stage has unique morphological characteristics, which are reflected not only in the color of the leaves, but also in the changes in their shape, size, and texture. Therefore, when collecting leaf images, not only the surface details of the leaves should be concerned, but also the possible impacts of environmental factors such as lighting conditions and background interference on the image quality should be considered.

[0041] After selecting the research object, multiple observation points need to be set up to ensure that the growth changes of the plant in different environments can be captured. The selected plant samples are photographed regularly every day using a high-resolution camera, and the shooting angle and distance are kept consistent each time to reduce variable interference during subsequent analysis. At the same time, professional software is used to preliminarily screen the captured pictures, eliminating blurred or obviously defective photos and retaining high-quality image data. As the plant grows, its leaves will undergo significant morphological changes, especially during the transition period between adjacent growth stages, and this change is particularly obvious. For example, during the transition from the growth stage to the flowering stage, fine wrinkles may appear on the leaf edge, and the color changes from dark green to light green, which are all important feature information. Therefore, it is particularly important to record the image data at each time point in detail.

[0042] Then, these images are classified and sorted according to the growth stages to form subsets of plant leaf images at the first to fifth growth stages. This process requires strict review of each image to confirm its specific growth stage. Through learning a large number of samples, the system can better understand the transition state between each growth stage, so as to make a more accurate judgment.

[0043] In order to ensure that each subset contains a sufficient number of sample images to fully reflect the changing patterns of leaf characteristics at this growth stage, on the one hand, the observation frequency can be increased and sampling efforts can be increased at key nodes of plant growth; on the other hand, multi-view shooting technology can be introduced to capture the full view of the leaves from different angles to further enhance the data dimension. In addition, information obtained by other sensors, such as environmental parameters such as temperature and humidity, can be combined to assist in determining the current growth stage of the plant. All these measures work together to provide a solid data foundation for subsequent plant growth stage identification and analysis.

[0044] In the above-mentioned image-based plant growth cycle monitoring method, the step S2 performs image feature extraction and prototype feature extraction aggregation analysis on the subsets of plant leaf images of each growth stage in the subsets of plant leaf images of the first to fifth growth stages to obtain prototype image features of plant leaves of the first to fifth growth stages. It should be understood that since plant growth is a continuous and dynamic process, there are progressive feature associations in each growth stage. Therefore, the present application further performs feature association aggregation analysis on the plant leaf images in each growth stage image subset to integrate and refine the scattered image features to form prototype image features representing the growth stage, thereby obtaining prototype image features of plant leaves of the first to fifth growth stages as an important reference for subsequent judgment of the growth stage of the plant image to be identified. Among them, Figure 3 FIG. 1 is a flowchart of sub-step S2 of the image-based plant growth cycle monitoring method according to an embodiment of the present application. Figure 3 As shown, the step S2 includes the steps of: S21, performing image feature extraction based on hole convolution coding on each first growth stage plant leaf image in the subset of the first growth stage plant leaf images to obtain a subset of first growth stage plant leaf image features; S22, performing prototype feature extraction and aggregation analysis on the subset of the first growth stage plant leaf image features to obtain the prototype image features of the first growth stage plant leaves.

[0045] Specifically, in step S21, image feature extraction based on dilated convolution encoding is performed on each of the first growth stage plant leaf images in the subset of the first growth stage plant leaf images to obtain a subset of the first growth stage plant leaf image features. That is, taking the image processing process of the subset of the first growth stage plant leaf images as an example, in order to extract image features such as the color, texture, and shape of the leaves from each plant leaf image in the first growth stage, the present application performs dilated convolution encoding on each of the first growth stage plant leaf images in the subset to extract leaf image features, so as to obtain a subset of the first growth stage plant leaf image features. Those of ordinary skill in the art should know that dilated convolution inserts holes in the traditional convolution kernel, enabling the convolution kernel to span more pixels during convolution operations, expanding the receptive field without losing image resolution, capturing the spatial correlation features between structures such as leaf sheaths and leaf veins, and thus being able to consider both the fine texture of the leaf local area and the overall shape contour and other features in the plant leaf image at the same time, improving the accuracy and robustness of image feature extraction.

[0046] Specifically, in step S22, prototype feature extraction and aggregation analysis are performed on the subset of the first growth stage plant leaf image features to obtain the first growth stage plant leaf prototype image features. It should be understood that considering the actual scenario of plant growth monitoring, traditional single-sample feature analysis is difficult to cope with the morphological diversity of leaves within the same growth stage. For example, the leaves in the maize germination stage show continuous variation characteristics such as the curling angle (15° - 45°) and the degree of leaf sheath wrapping (30% - 80%). Directly using a single leaf feature as the judgment benchmark will increase the misjudgment rate of the model for atypical samples. Therefore, the present application further performs essential modeling of the population feature distribution on the subset of the first growth stage plant leaf image features to construct a feature benchmark that can cover the typical morphological changes in this stage, forming the first growth stage plant leaf prototype image features, thereby eliminating the interference of individual sample noise on stage determination. Among them, Figure 4 is a flowchart of sub-step S22 of the image-based plant growth cycle monitoring method according to an embodiment of the present application. As Figure 4 shown, step S22 includes steps: S221, performing image feature independence topological analysis on the subset of the first growth stage plant leaf image features to obtain a first growth stage plant leaf image independence spectral space encoding feature matrix; S222, based on the first growth stage plant leaf image independence spectral space encoding feature matrix, performing feature significance modulation aggregation on the subset of the first growth stage plant leaf image features to obtain the first growth stage plant leaf prototype image features.

[0047] More specifically, in step S221, among them, Figure 5It is a flowchart of sub-step S221 of the image-based plant growth cycle monitoring method according to an embodiment of the present application. As Figure 5 shown, the step S221 includes the steps: S2211, calculating an image feature space independence description operator between any two first growth stage plant leaf image features in the subset of the first growth stage plant leaf image features based on the image feature projection matrix to obtain a first growth stage plant leaf image feature independence spectrum space encoding matrix composed of a plurality of plant leaf image feature subspace independence description operators; S2212, performing spectrum space activation based on an activation function on the first growth stage plant leaf image feature independence spectrum space encoding matrix to obtain the first growth stage plant leaf image independence spectrum space encoding feature matrix.

[0048] In a specific example of the present application, the step S2211 is expressed by the formula:

[0049]

[0050]

[0051] where represents the subset of the first growth stage plant leaf image features, , , , and respectively represent the 1st, 2nd, th, th, and nd first growth stage plant leaf image features in the subset of the first growth stage plant leaf image features, is the number of the first growth stage plant leaf image features in the subset, represents calculating the two-norm, and respectively represent different image feature projection matrices, represents the transpose of a vector, represents vector multiplication, represents and the plant leaf image feature subspace independence description operator between.

[0052] That is, by calculating the image feature space independence description operator between the image features of plant leaves in each first growth stage, the coupling relationship of heterogeneous features such as leaf morphology, texture, and color of different leaf images in the image feature projection space is decoupled, and the modal dissociation degree between leaf features is quantified. Here, the image feature space independence description operator represents the independence between the corresponding two plant leaf image features, and the first growth stage plant leaf image feature independence spectrum space coding matrix formed based on the spatial topological structure arrangement not only retains the relative position information of each leaf image feature in the first growth stage but also reveals the continuous law of leaf feature evolution. Through the capture of this global interaction mode, the model can break through the limitations of traditional single-frame static analysis, achieve a fine division of the growth stage transition period, and thus upgrade the growth cycle monitoring from discrete static image feature determination to continuous dynamic tracking.

[0053] In a specific example of the present application, the step S2212 is expressed by the formula:

[0054]

[0055] Wherein, represents the sigmoid activation function, , , and respectively represent the plant leaf image feature subspace independence description operators between and , and , and , and ; represents the first growth stage plant leaf image independence spectrum space coding feature matrix.

[0056] That is, the present application further introduces a normalization function, reconstructs the significance distribution of the feature space through non-linear mapping, adjusts the activation degree of the plant leaf image feature subspace independence description operator, enhances the interaction intensity between plant leaf image features with significant associations, and simultaneously suppresses the interference to non-significant associated features. Through the activation operation, the significance information in the first growth stage plant leaf image feature independence spectrum space coding matrix is strengthened, while irrelevant or redundant information is suppressed. The generated first growth stage plant leaf image independence spectrum space coding feature matrix not only improves the expression ability of modal features but also provides optimized modal significance information for subsequent node feature modulation.

[0057] More specifically, for the step S222, wherein, Figure 6Flow chart of sub-step S222 of the image-based plant growth cycle monitoring method according to an embodiment of the present application. As Figure 6 shown, the step S222 includes steps: S2221, inputting each first growth stage plant leaf image feature in the subset of the first growth stage plant leaf image features and the first growth stage plant leaf image independent spectral space encoding feature matrix into an image feature saliency modulation module to obtain a subset of independent modulation first growth stage plant leaf image features; S2222, inputting the subset of the independent modulation first growth stage plant leaf image features into a feature dynamic clustering network to obtain the first growth stage plant leaf prototype image features.

[0058] In a specific example of the present application, the step S2221 is expressed by the formula:

[0059]

[0060] where represents the feature scale value of and represents the corresponding independent modulation first growth stage plant leaf image feature.

[0061] That is, by using the first growth stage plant leaf image independent spectral space encoding feature matrix to perform feature modulation on each first growth stage plant leaf image feature, each first growth stage plant leaf image feature is integrated into the global image feature interaction topology information, so as to re-encode the originally independent leaf image features in the spectral space, form a feature representation with rich context information, obtain a subset of independent modulation first growth stage plant leaf image features, and make the continuous representation of the leaf image features within the stage more accurate. By explicitly modeling the synergistic effect between features, the efficiency and accuracy of subsequent clustering analysis are optimized, the computational complexity of feature dynamic clustering is reduced, and advantageous original feature materials are provided for subsequent feature dynamic clustering.

[0062] In a specific example of the present application, the step S2222 includes: calculating the leaf feature stability factor of each independently modulated plant leaf image feature in the first growth stage based on the feature distribution of each independently modulated plant leaf image feature in the subset of the independently modulated plant leaf image features in the first growth stage to obtain a subset of the plant leaf image feature stability factors in the first growth stage; performing Softmax-based normalization processing on the subset of the plant leaf image feature stability factors in the first growth stage to obtain a subset of the dynamic aggregation weight coefficients of the plant leaf image features in the first growth stage; and performing weighted aggregation on the subset of the independently modulated plant leaf image features in the first growth stage based on the subset of the dynamic aggregation weight coefficients of the plant leaf image features in the first growth stage to obtain the prototype image features of the plant leaves in the first growth stage, which is expressed by the formula:

[0063]

[0064] Wherein, represents calculating the feature kurtosis, represents the offset factor, represents the plant leaf image feature stability factor in the first growth stage of, represents the feature mean of the independently modulated plant leaf image features in the first growth stage, represents the feature variance of the independently modulated plant leaf image features in the first growth stage, represents calculating the expected value, represents the normalization function, represents the dynamic aggregation weight coefficient of the plant leaf image feature in the first growth stage of, represents the prototype image features of the plant leaves in the first growth stage.

[0065] That is, by constructing a feature dynamic clustering network with context time-series evolution awareness, measuring the stability and importance of each independently modulated plant leaf image feature in the first growth stage, and converting the stability factor into a probability distribution, not only the dynamic weight allocation is realized, but also the mapping relationship between the feature importance and the growth stage evolution is established, so that the signal-to-noise ratio in the feature space is systematically improved, the discriminability of the feature representation is enhanced, and the weighted aggregation process forms a global feature prototype representation by fusing the time-series evolution information of multiple leaf image features, enabling the system to capture the subtle feature changes in the stage transition period.

[0066] In a preferred example of the present application, based on the feature distributions of each of the independently modulated first growth stage plant leaf image features in the subset of the independently modulated first growth stage plant leaf image features, the leaf feature stability factor of each of the independently modulated first growth stage plant leaf image features is calculated to obtain a subset of the first growth stage plant leaf image feature stability factors, including: performing a covariance-orthogonal composite transformation on the independently modulated first growth stage plant leaf image features to obtain an offset factor; calculating the feature kurtosis of the independently modulated first growth stage plant leaf image features, dividing it by two and adding it to the offset factor to obtain the first growth stage plant leaf image feature stability factor, which is represented by the formula:

[0067]

[0068]

[0069]

[0070]

[0071]

[0072] Wherein, represents the first growth stage plant leaf image feature alignment space matrix, represents the identity matrix, represents corresponding spatially aligned independently modulated first growth stage plant leaf image features, represents vector subtraction, represents the first growth stage plant leaf image feature covariance matrix, represents the scaling factor, represents the vector inner product.

[0073] That is, the present application takes into account that in plant growth monitoring, when the dynamic clustering network strengthens the key leaf features for feature aggregation by the significant relationship between the independently modulated first growth stage plant leaf image features, the base mismatch is likely to occur between the clustering center and the edge of the feature distribution. Therefore, preferably, the present application dynamically adjusts the mapping intensity by introducing the offset factor for downward alignment. Specifically, first, the first growth stage plant leaf image feature alignment space matrix is obtained through the orthogonal constraint decomposition of the identity matrix, and then the isometric mapping transformation is performed on the independently modulated first growth stage plant leaf image features based on the first growth stage plant leaf image feature alignment space matrix, and then the covariance matrix of the independently modulated first growth stage plant leaf image features is introduced, at In addition to the statistical geometric structure represented by the mean direction and the variance representation, the local geometric structure of the plant leaf image features in the first growth stage of the independence modulation is adjusted, and the random local differences of the basis are rotationally / reflectionally eliminated through the inner product basification representation on the basis of the orthogonal constraint, so as to achieve the downward spatial arrangement alignment, so as to eliminate the dynamic cross-domain inconsistency caused by emphasizing the significance relationship of the plant leaf image features in the first growth stage of the independence modulation during the feature clustering process.

[0074] In the above image-based plant growth cycle monitoring method, in step S3, an image of the plant to be recognized is obtained. That is, by obtaining an image of the plant to be recognized, leaf feature matching analysis is performed on it and the leaf prototype image features of the first to fifth growth stages of the plant, so as to judge the growth stage of the image of the plant to be recognized.

[0075] In the above image-based plant growth cycle monitoring method, in step S4, leaf features are extracted from the image of the plant to be recognized to obtain the leaf image features to be recognized. Among them, Figure 7 FIG. is a flowchart of sub-step S4 of the image-based plant growth cycle monitoring method according to an embodiment of the present application. As Figure 7 shown, step S4 includes steps: S41, inputting the image of the plant to be recognized into a leaf region of interest recognizer based on the Yolo model to obtain a leaf ROI image to be recognized; S42, performing image feature extraction based on dilated convolution coding on the leaf ROI image to be recognized to obtain the leaf image features to be recognized.

[0076] Specifically, in step S41, the image of the plant to be recognized is input into a leaf region of interest recognizer based on the Yolo model to obtain a leaf ROI image to be recognized. Specifically, since the actually collected image of the plant to be recognized usually contains a large amount of background information, such as soil, surrounding weeds, planting facilities, etc., these irrelevant information will increase the complexity and computational amount of subsequent feature analysis, and may interfere with the extraction of key features of plant leaves, resulting in inaccurate analysis results. In response to this, the present application uses the Yolo model to construct a leaf region of interest recognizer to perform region of interest recognition on the image of the plant to be recognized. The Yolo (You Only Look Once) model is an advanced object detection algorithm. Based on the principle of deep learning, through learning a large amount of labeled data, it can quickly identify the category and position of the target object in the image. By using the Yolo model to process the image of the plant to be recognized in the present application, the region where the leaf is located can be quickly and accurately recognized, the plant leaf part can be separated from the complex image of the plant to be recognized, and the interference of the background information can be eliminated, so as to obtain a leaf ROI image to be recognized that only contains leaf information, so as to simplify the subsequent analysis process and improve the accuracy and efficiency of the analysis.

[0077] Specifically, in step S42, image feature extraction based on dilated convolution coding is performed on the ROI image of the leaf to be recognized to obtain the image features of the leaf to be recognized. It should be understood that in order to effectively match the ROI image of the leaf to be recognized with the image features of the prototype images of plant leaves in the first to fifth growth stages, the present application also uses dilated convolution coding technology to process the ROI image of the leaf to be recognized, so as to utilize the large-scale receptive field characteristics of dilated convolution to capture key features such as the color, texture, shape, and spatial structure of the leaf, thereby obtaining the image features of the leaf to be recognized and providing accurate and comprehensive feature data for subsequent leaf feature matching analysis.

[0078] In the above image-based plant growth cycle monitoring method, in step S5, image feature matching analysis is performed on the image features of the leaf to be recognized and the image features of the prototype images of plant leaves in the first to fifth growth stages to obtain a matching analysis result. In a specific example of the present application, the cosine similarity between the image features of the leaf to be recognized and the image features of the prototype images of plant leaves in the first to fifth growth stages is calculated to obtain the matching analysis result. As an index for measuring the consistency of the directions of two vectors, the closer the value of the cosine similarity is to 1, the more consistent the directions of the two vectors are, that is, the higher the similarity between the image features of the leaf to be recognized and the image features of the prototype image of plant leaves in a certain growth stage; the closer the cosine similarity is to 0, the more inconsistent the directions of the two vectors are, that is, the lower the similarity between the image features of the leaf to be recognized and the image features of the prototype image of plant leaves in a certain growth stage. According to the calculated cosine similarity, it can be determined which growth stage the prototype image features of the leaf image to be recognized are closest to, thereby determining the growth stage of the plant image to be recognized.

[0079] In the above image-based plant growth cycle monitoring method, in step S6, based on the matching analysis result, the growth stage of the plant image to be recognized is determined. Specifically, if the matching analysis result shows that the cosine similarity between the image features of the leaf to be recognized and the image features of the prototype image of plant leaves in a certain growth stage is the highest and this cosine similarity exceeds a preset similarity threshold, it is determined that the plant image to be recognized is in this growth stage. The preset similarity threshold is an empirical value obtained based on a large amount of training data and actual tests, and is used to ensure the accuracy and reliability of the determination result. If the cosine similarity between the image features of the leaf to be recognized and the image features of the prototype images of plant leaves in all growth stages does not exceed the preset similarity threshold, it is determined that the growth stage of the plant image to be recognized is not clear, and it may be necessary to re-collect the image or use manual review and other auxiliary means for judgment. Through the above method, the present application can achieve automatic, rapid, and accurate identification of the plant growth stage, helping managers to timely understand the growth situation of plants, take corresponding measures for management and adjustment, and promote the healthy growth of plants and increase yields.

[0080] In summary, the image-based plant growth cycle monitoring method based on the embodiments of the present application is elucidated. It pre-collects a set of plant leaf images at different growth stages and introduces an image processing technology based on deep learning to perform image feature extraction and prototype feature extraction and aggregation analysis on the set of plant leaf images at each growth stage to capture the prototype image features of the plant leaves at each growth stage. Furthermore, based on the prototype image features of the plant leaves at each growth stage, leaf feature matching analysis is performed on the plant image to be recognized, so as to quickly determine the current growth stage of the plant, facilitating targeted fertilization management of the plant. In this way, intelligent monitoring of the plant growth cycle can be achieved, providing effective guidance for the fertilization management of the plant, and further improving the scientificity and accuracy of fertilization management.

[0081] To verify the effectiveness of the method of the present application, a comparative experiment was conducted with the prior art in terms of misjudgment rate, accuracy rate, processing time, and environmental anti-interference ability. Prior art one (Tong Shuyuan, Song Fengbin. Application of SPAD value in nitrogen nutrition diagnosis and recommended fertilization of maize [J]. Systems Science and Comprehensive Studies in Agriculture, 2009, 25(2): 233-238.) evaluates the growth stage and nitrogen nutrition status of maize based on the SPAD value of maize leaves, and recommends the nitrogen fertilizer application rate accordingly, proposing a variable nitrogen application strategy. SPAD (Soil Plant Analysis Development) measures the absorption rate of leaves for specific wavelengths (such as red light and near-infrared light) through an optical sensor to detect the chlorophyll content of the leaves, and then infers whether the plant is nitrogen-deficient. Prior art two (Beijing Research Center for Intelligent Equipment for Agriculture. Control method for integrated water and fertilizer irrigation system and integrated water and fertilizer irrigation system: CN201911046157.7 [P]. 2020-01-17.) analyzes the water supply status, fertilizer supply status, and growth period of crops based on a deep network model of ResNet-101, and controls the operation of the integrated water and fertilizer irrigation device accordingly. The comparison results are shown in Table 1:

[0082] Table 1 Comparison results of the plant growth cycle monitoring method of the present application and the prior art

[0083]

[0084] As can be seen from Table 1, the misjudgment rate (3.2%) of the method of the present application is much lower than that of SPAD (18.5%) and ResNet-based image classification (12.7%), mainly due to the prototype feature extraction and aggregation analysis and dynamic clustering technology. The method of the present application reduces the influence of light and background noise (SPAD is easily interfered by leaf color distortion) through comprehensive analysis of key features such as the color, texture, shape, and spatial structure of the leaves, and has stronger environmental anti-interference ability.

[0085] Further, a plant growth cycle monitoring system based on images is also provided.

[0086] Figure 8 It is a block diagram of a plant growth cycle monitoring system based on images according to an embodiment of the present application. As Figure 8 shown, the plant growth cycle monitoring system 100 based on images according to an embodiment of the present application includes: an image acquisition module 110, configured to acquire plant leaf images at different growth stages and perform image set segmentation according to the growth stages to obtain subsets of plant leaf images at the first to fifth growth stages; a plant leaf prototype feature extraction module 120, configured to perform image feature extraction and prototype feature extraction and aggregation analysis on each subset of plant leaf images at the first to fifth growth stages in the subsets of plant leaf images at the first to fifth growth stages to obtain plant leaf prototype image features at the first to fifth growth stages; a plant image to be recognized acquisition module 130, configured to acquire a plant image to be recognized; a plant image to be recognized feature extraction module 140, configured to extract leaf features from the plant image to be recognized to obtain a plant leaf image feature to be recognized; a feature matching analysis module 150, configured to perform image feature matching analysis on the plant leaf image feature to be recognized and the plant leaf prototype image features at the first to fifth growth stages to obtain a matching analysis result; and a growth stage determination module 160, configured to determine the growth stage of the plant image to be recognized based on the matching analysis result.

[0087] Here, those skilled in the art can understand that the specific operations of each module in the above-mentioned plant growth cycle monitoring system based on images have been described in detail in the description of the plant growth cycle monitoring method based on images above, and therefore, the repeated description thereof will be omitted. Figures 1 to 7 It is an automatic fertilization flow chart of an automatic plant fertilization management system according to an embodiment of the present application. After preprocessing the collected images, analyze and extract their features, compare them with the pre-stored data in the database, determine the current growth stage of the plant, and retrieve the data on the required nutrients in the database; calculate the actual chemical fertilizer dosage according to the amount of required nutrients, and finally generate a fertilization control signal to achieve automatic fertilization. As

[0088] Figure 9 shown, first, a growth cycle monitoring link is performed. In the first step, image acquisition is performed; the system will perform the growth cycle monitoring step, and the camera deployed in the planting area 3 will collect plant images at regular intervals, and the original data will be transmitted to the edge computing node; in the second step, the images are preprocessed, and Gaussian filtering, resolution standardization (and CLAHE contrast enhancement (8×8 block to enhance the visibility of leaf texture) are sequentially performed, and at the same time, the main plant body area is segmented by a lightweight U-Net model to eliminate background interference. Figure 9 ​

[0089] The third step is to conduct model classification and data comparison:

[0090] ① The preprocessed image is input into the improved YOLOv8n classification model, which embeds the CBAM attention module in the original architecture to enhance the leaf feature extraction capability.

[0091] ② The model output includes growth stage labels (five categories such as germination period and growth period) and confidence values. When the confidence is lower than 90%, the FAO data calibration process is automatically triggered: the FAO crop stage API is called, and the multilingual BERT model is used to calculate the semantic similarity between the model output stage and the FAO standard terminology (such as the cosine similarity between "flowering phase" and "Flowering Phase" is 0.92). If the match is successful, the FAO standardized stage name is used. The final calibration results are written into the SQL database together with the original data, and the nitrogen, phosphorus and potassium (hereinafter referred to as NPK) demand data is output (obtained by associating and querying the FAO nutrient demand interface).

[0092] ③ If image transmission is interrupted or model reasoning is abnormal during this period, the system automatically activates the backup MobileNetV2 lightweight model and records the device status code, and sends a warning message to the operation and maintenance platform. The growth stage information is then transmitted to the fertilizer calculation module.

[0093] The system needs to calculate the amount of fertilizer. It first makes a logical judgment based on the growth stage information to determine whether the existing nutrient solution or fertilizer can directly meet the growth needs of the plant. If it can, it will directly proceed to the dilution ratio calculation stage of fertilization. If not, it is necessary to calculate the ratio of multiple fertilizers:

[0094] ① According to the element demand and compound demand:

[0095] Enter the amount of pure elements required for the plant growth stage: N (kg), P (kg), K (kg).

[0096] Convert P and K to the corresponding oxide requirements:

[0097] ;

[0098] ;

[0099] ② Calculate the amount of compound fertilizer based on the demand for a single compound:

[0100] Assume that two types of compound fertilizer A are selected ( : ) and compound fertilizer B ( ), establish the equation system:

[0101]

[0102] Solve for x (the amount of compound fertilizer A) and y (the amount of compound fertilizer B) using matrix operations or optimization algorithms. If there is no solution, prompt to adjust the compound fertilizer combination or supplement single-element fertilizers.

[0103] ③ Calculate the dilution solution concentration based on the amount of compound fertilizer used.

[0104] The following is an example: The required amounts of nitrogen, phosphorus, and potassium elements obtained by comparison from the database are:

[0105] Nitrogen (N): 5 mg, Phosphorus (P): 3 mg, Potassium (K): 4 kg;

[0106] Optional compound fertilizers:

[0107] Compound fertilizer A: (20 - 10 - 10): containing 20% N, 10% , 10%

[0108] Compound fertilizer B: (10 - 20 - 10): containing 10% N, 20% , 10%

[0109] First step, calculate the required amounts of oxides:

[0110] Required amount = = ≈ 6.98 mg

[0111] Required amount = = ≈ 4.82 mg

[0112] Second step, solve the system of equations

[0113]

[0114] Judging from the criterion for the solution of the linear system of equations, the system of equations has no solution. Therefore, it is necessary to introduce compound fertilizer 3 containing only one element (containing only 50% ), reallocate the requirements, first use compound fertilizers A and B to meet the N and requirements, and the remaining is supplemented by compound fertilizer C, and the system of equations is corrected.

[0115]

[0116] The solution is:

[0117] x = 10.4 mg, y = 28.2 mg;

[0118] Next, calculate and provide the application rate of compound fertilizer 3

[0119] Provided by compound fertilizer A + B :

[0120]

[0121] To be supplemented :

[0122]

[0123] The application rate of compound fertilizer C:

[0124]

[0125] Control steps for fertilization:

[0126] ① Through the previous calculations, the demand for each fertilizer has been obtained, and the volume of the diluted nutrient solution actually required has been obtained through calculation.

[0127] ② Calculate the valve opening time: Based on the rated flow rate of the valve and the volume of the nutrient solution required, calculate the opening time of the valve opening, and then compensate for the system delay time to finally obtain the actual working time.

[0128] ③ Control the valve opening: Adopt a parallel control mode to enable the nutrient solution to be fully mixed, improve the working efficiency at the same time, and use a built-in timer to control the working time.

[0129] ④ Generate a work report: Output the fertilization result based on the application rate of the fertilizer and the working time. If there is a problem, a warning will be sent for manual intervention.

[0130] The basic principle of the present invention has been described above in combination with specific embodiments. However, it should be noted that the advantages, advantages, effects, etc. mentioned in the present invention are only examples and not limitations. It cannot be considered that these advantages, advantages, effects, etc. are essential for each embodiment of the present invention. In addition, the specific details of the above embodiments are only for the purpose of illustration and easy understanding, rather than limitations. The above details do not limit the present invention to necessarily adopt the above specific details to implement.

[0131] In the above embodiments, the descriptions of the various embodiments have their own emphases. For parts not described or recorded in a certain embodiment, reference may be made to the relevant descriptions of other embodiments. In the several embodiments provided by the present invention, it should be understood that the disclosed system and method can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the unit division is only a logical function division, and there may be other division methods in actual implementation. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0132] For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and without departing from the spirit or basic characteristics of the present invention, the present invention can be implemented in other specific forms. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be embraced by the present invention. Any reference signs in the claims should not be regarded as limiting the claimed rights.

[0133] In addition, it is obvious that the word "including" does not exclude other units or steps, and the singular does not exclude the plural. The multiple units stated in the system claims can also be implemented by one unit through software or hardware.

[0134] Finally, it should be noted that the above description has been given for purposes of illustration and description. In addition, the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. An image-based method for monitoring the plant growth cycle, characterized in that, Including: Collecting plant leaf images at different growth stages and segmenting the image set according to the growth stages to obtain subsets of plant leaf images at the first to fifth growth stages; Performing image feature extraction and prototype feature extraction and aggregation analysis on each subset of plant leaf images at the first to fifth growth stages to obtain prototype image features of plant leaves at the first to fifth growth stages; specifically including: performing image feature extraction based on dilated convolutional coding on each first-stage plant leaf image in the subset of first-stage plant leaf images to obtain a subset of first-stage plant leaf image features; Calculating an image feature space independence description operator between any two first-stage plant leaf image features in the subset of first-stage plant leaf image features based on an image feature projection matrix to obtain a first-stage plant leaf image feature independence spectrum space coding matrix composed of multiple plant leaf image feature subspace independence description operators; Performing spectrum space activation based on an activation function on the first-stage plant leaf image feature independence spectrum space coding matrix to obtain a first-stage plant leaf image independence spectrum space coding feature matrix; Inputting each first-stage plant leaf image feature in the subset of first-stage plant leaf image features and the first-stage plant leaf image independence spectrum space coding feature matrix into an image feature saliency modulation module to obtain a subset of independently modulated first-stage plant leaf image features; Inputting the subset of independently modulated first-stage plant leaf image features into a feature dynamic clustering network to obtain first-stage plant leaf prototype image features; Obtaining a plant image to be recognized; Extracting leaf features from the plant image to be recognized to obtain leaf image features to be recognized; Performing image feature matching analysis on the leaf image features to be recognized and the prototype image features of plant leaves at the first to fifth growth stages to obtain a matching analysis result; Based on the matching analysis result, determining the growth stage of the plant image to be recognized.

2. The image-based plant growth cycle monitoring method according to claim 1, characterized in that, The growth stages include the germination stage, the growth stage, the flowering stage, the fruiting stage, and the senescence stage.

3. The method for monitoring the plant growth cycle based on images according to claim 1, wherein Inputting the subset of independently modulated first-stage plant leaf image features into a feature dynamic clustering network to obtain the first-stage plant leaf prototype image features, including: Calculating the leaf feature stability factor of each independently modulated first-stage plant leaf image feature based on the feature distribution of each independently modulated first-stage plant leaf image feature in the subset of independently modulated first-stage plant leaf image features to obtain a subset of first-stage plant leaf image feature stability factors; Performing normalization processing based on Softmax on the subset of first-stage plant leaf image feature stability factors to obtain a subset of first-stage plant leaf image feature dynamic aggregation weight coefficients; Based on the subset of the dynamic aggregation weight coefficients of the plant leaf image features in the first growth stage, the subset of the plant leaf image features in the first growth stage that are independently modulated is weighted and aggregated to obtain the prototype image features of the plant leaves in the first growth stage.

4. The method for monitoring the plant growth cycle based on images according to claim 3, wherein, Based on the feature distributions of the independently modulated plant leaf image features in the subset of the independently modulated plant leaf image features in the first growth stage, the leaf feature stability factors of the independently modulated plant leaf image features in each are calculated to obtain a subset of the plant leaf image feature stability factors in the first growth stage, including: Performing covariance-orthogonal composite transformation on the independently modulated plant leaf image features in the first growth stage to obtain an offset factor; Calculating the feature kurtosis of the independently modulated plant leaf image features in the first growth stage, dividing it by two and adding it to the offset factor to obtain the plant leaf image feature stability factor in the first growth stage.

5. The method for monitoring the plant growth cycle based on images according to claim 1, characterized in that, Extracting leaf features from the plant image to be recognized to obtain the leaf image features to be recognized, including: Inputting the plant image to be recognized into a leaf region of interest recognizer based on the Yolo model to obtain the leaf ROI image to be recognized; Performing image feature extraction based on dilated convolution coding on the leaf ROI image to be recognized to obtain the leaf image features to be recognized.

6. An image-based plant growth cycle monitoring system, characterized in that, Including: An image acquisition module for acquiring plant leaf images at different growth stages and segmenting the image set according to the growth stage to obtain subsets of plant leaf images in the first to fifth growth stages; A plant leaf prototype feature extraction module for performing image feature extraction and prototype feature extraction and aggregation analysis on each subset of plant leaf images in the subsets of plant leaf images in the first to fifth growth stages to obtain prototype image features of plant leaves in the first to fifth growth stages; specifically including: performing image feature extraction based on dilated convolution coding on each plant leaf image in the subset of plant leaf images in the first growth stage to obtain a subset of plant leaf image features in the first growth stage; Calculating the image feature space independence description operator between any two plant leaf image features in the subset of plant leaf image features in the first growth stage based on the image feature projection matrix to obtain a first growth stage plant leaf image feature independence spectrum space coding matrix composed of a plurality of plant leaf image feature subspace independence description operators; Performing spectrum space activation based on the activation function on the first growth stage plant leaf image feature independence spectrum space coding matrix to obtain a first growth stage plant leaf image independence spectrum space coding feature matrix; Inputting each plant leaf image feature in the subset of plant leaf image features in the first growth stage and the first growth stage plant leaf image independence spectrum space coding feature matrix into an image feature saliency modulation module to obtain a subset of independently modulated plant leaf image features in the first growth stage; Input the subset of the image features of the plant leaves in the first growth stage that modulates independence into the feature dynamic clustering network to obtain the prototype image features of the plant leaves in the first growth stage; An image acquisition module for the plant to be recognized, which is used to acquire the image of the plant to be recognized; A feature extraction module for the plant leaves to be recognized, which is used to extract the leaf features from the image of the plant to be recognized to obtain the image features of the plant leaves to be recognized; A feature matching and analysis module, which is used to perform image feature matching and analysis on the image features of the plant leaves to be recognized and the prototype image features of the plant leaves in the first to fifth growth stages to obtain a matching analysis result; A growth stage determination module, which is used to determine the growth stage of the plant image to be recognized based on the matching analysis result.

Images