Intelligent landscaping maintenance monitoring system and method thereof

Abstract

The invention relates to the field of greening maintenance monitoring, and particularly discloses an intelligent landscaping maintenance monitoring system and method, and the method comprises the steps: firstly obtaining a landscaping maintenance image collected by a camera and greening growth environment influence data collected by a sensor, and then carrying out the deep learning technology, and feature extraction and correlation analysis are carried out on the landscaping maintenance environment and the landscaping maintenance environment, and finally a classification result is obtained through a classifier so as to judge whether the landscaping maintenance environment needs to be adjusted and timely find and adjust problems of the landscaping maintenance environment, so that the efficiency and effect of landscaping maintenance are improved, and intelligent landscaping management is realized.

Description

Technical Field

[0001] The present application relates to the field of greening maintenance monitoring, and more specifically, to a smart garden greening maintenance monitoring system and method thereof. Background Art

[0002] Landscaping refers to the creation and maintenance of artificial landscapes through the planting, layout, and maintenance of plants to provide a beautiful, comfortable, and healthy natural environment. Landscaping not only increases urban green space and improves the ecological environment, but also enhances people's quality of life and well-being. Through proper planning and design, landscaping can create a variety of landscape elements, such as flower beds, lawns, trees, and water features, creating diverse landscape effects.

[0003] Landscaping maintenance requires professional monitoring and maintenance, which involves human resource investment and training costs. Manual monitoring requires hiring professional gardeners or landscapers, who regularly conduct inspections and address issues, potentially increasing maintenance costs. Furthermore, the frequency of manual monitoring is limited by human resources. Landscaping areas are typically large, and manual monitoring requires significant time and effort, so daily or even weekly monitoring may not be possible. This can lead to delayed detection and resolution of some issues, impacting the health and aesthetics of the landscaping.

[0004] Therefore, a smart landscaping maintenance monitoring system and method thereof are desired. Summary of the Invention

[0005] In order to solve the above technical problems, the present application is proposed. The embodiments of the present application provide a smart garden greening maintenance monitoring system and method thereof, which first obtains garden greening maintenance images collected by a camera and greening growth environment impact data collected by a sensor, then uses deep learning technology to perform feature extraction and correlation analysis on the two, and finally obtains classification results through a classifier to determine whether the garden greening maintenance environment needs to be adjusted, so as to timely discover problems in the garden greening maintenance environment and make adjustments, thereby improving the efficiency and effectiveness of garden greening maintenance, thereby realizing intelligent garden greening management.

[0006] According to one aspect of the present application, a smart landscaping maintenance monitoring system is provided, comprising:

[0007] A garden greening maintenance data acquisition module is used to acquire garden greening maintenance images collected by the camera and greening growth environment impact data collected by the sensor, wherein the greening growth environment impact data includes the nutrient solution content of the garden greening at multiple predetermined time points, the air temperature at multiple predetermined time points, and the air humidity at multiple predetermined time points;

[0008] A garden greening maintenance data extraction module is used to extract a garden greening maintenance global feature vector and a greening growth environment Gaussian density map from the garden greening maintenance image collected by the camera and the greening growth environment impact data collected by the sensor;

[0009] A greening maintenance environment feature fusion module is used to construct a garden maintenance fusion feature matrix between the garden greening maintenance global feature vector and the greening growth environment Gaussian density map;

[0010] The greening maintenance environment judgment module is used to judge whether the garden greening maintenance environment needs to be adjusted based on the garden maintenance fusion feature matrix.

[0011] According to another aspect of the present application, a smart landscaping maintenance monitoring method is provided, which includes:

[0012] Acquire landscaping maintenance images captured by a camera and greening growth environment impact data captured by a sensor, wherein the greening growth environment impact data includes nutrient solution content of the landscaping at multiple predetermined time points, air temperature at multiple predetermined time points, and air humidity at multiple predetermined time points;

[0013] Extracting a garden greening maintenance global feature vector and a greening growth environment Gaussian density map from the garden greening maintenance image collected by the camera and the greening growth environment impact data collected by the sensor;

[0014] Constructing a garden maintenance fusion feature matrix between the garden greening maintenance global feature vector and the greening growth environment Gaussian density map;

[0015] Based on the garden maintenance fusion feature matrix, it is determined whether the garden greening maintenance environment needs to be adjusted.

[0016] Compared with the existing technology, the present application provides an intelligent garden greening maintenance monitoring system and method thereof, which first obtains the garden greening maintenance image collected by the camera and the greening growth environment impact data collected by the sensor, and then uses deep learning technology to perform feature extraction and correlation analysis on the two. Finally, the classification result is obtained through the classifier to determine whether the garden greening maintenance environment needs to be adjusted, so as to timely discover the problems of the garden greening maintenance environment and make adjustments, thereby improving the efficiency and effectiveness of garden greening maintenance, thereby realizing intelligent garden greening management. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The above and other purposes, features, and advantages of the present application will become more apparent through a more detailed description of the embodiments of the present application in conjunction with the accompanying drawings. The accompanying drawings are intended to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the drawings, the same reference numerals generally represent the same components or steps.

[0018] Figure 1 This is a block diagram of a smart landscaping maintenance monitoring system according to an embodiment of the present application.

[0019] Figure 2 This is a block diagram of a garden greening maintenance data extraction module in a smart garden greening maintenance monitoring system according to an embodiment of the present application.

[0020] Figure 3 This is a block diagram of a greening growth environment impact feature extraction unit in a smart garden greening maintenance monitoring system according to an embodiment of the present application.

[0021] Figure 4 This is a block diagram of a greening maintenance environment judgment module in a smart garden greening maintenance monitoring system according to an embodiment of the present application.

[0022] Figure 5 Flowchart of a smart landscaping maintenance monitoring method according to an embodiment of the present application.

[0023] Figure 6 is a block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described herein.

[0025] Exemplary Systems

[0026] Figure 1 FIG is a block diagram of a smart landscaping maintenance monitoring system according to an embodiment of the present application. Figure 1As shown, the smart garden greening maintenance monitoring system 100 according to the embodiment of the present application includes: a garden greening maintenance data acquisition module 110, which is used to obtain garden greening maintenance images collected by a camera and greening growth environment impact data collected by a sensor, wherein the greening growth environment impact data includes the nutrient solution content of the garden greening at multiple predetermined time points, the air temperature at multiple predetermined time points, and the air humidity at multiple predetermined time points; a garden greening maintenance data extraction module 120, which is used to extract the garden greening maintenance global feature vector and the greening growth environment Gaussian density map from the garden greening maintenance images collected by the camera and the greening growth environment impact data collected by the sensor; a greening maintenance environment feature fusion module 130, which is used to construct a garden maintenance fusion feature matrix between the garden greening maintenance global feature vector and the greening growth environment Gaussian density map; a greening maintenance environment judgment module 140, which is used to judge whether the garden greening maintenance environment needs to be adjusted based on the garden maintenance fusion feature matrix.

[0027] In the aforementioned intelligent landscaping maintenance monitoring system 100, the landscaping maintenance data acquisition module 110 is configured to acquire landscaping maintenance images captured by cameras and data on the impact of landscaping growth environments collected by sensors. The data on the impact of landscaping growth environments includes the nutrient solution content, air temperature, and air humidity at multiple predetermined time points. It is understood that the effectiveness of landscaping maintenance is significantly impacted by professional monitoring and maintenance. This requires significant human resources and training costs. Professional gardeners or garden workers regularly inspect and address issues, but this increases maintenance costs. Limited human resources may result in insufficient monitoring frequency, leading to delayed detection and resolution of issues, which in turn affects the quality and aesthetics of the landscaping. Considering that images captured by cameras can provide visual information, such as plant appearance, growth status, and changes, image analysis can be used to observe plant growth and whether plants are infested by pests and diseases. The nutrient solution content reflects the availability of nutrients in the soil required by plants and is crucial to their growth and development. Air temperature and humidity have a significant impact on plant growth and health. Air temperature directly affects plant growth rate, metabolism, and nutrient absorption, while humidity affects plant transpiration and water absorption. Therefore, in the technical solution of this application, by comprehensively analyzing this data and utilizing a deep learning network, we can more accurately understand the overall state of landscaping, thereby promptly identifying problems and taking appropriate maintenance measures, thereby maintaining the good condition of landscaping and improving the quality and quality of the landscape.

[0028] In the aforementioned intelligent landscaping maintenance monitoring system 100, the landscaping maintenance data extraction module 120 is configured to extract a global landscaping maintenance feature vector and a Gaussian density map of the landscaping growth environment from the landscaping maintenance images captured by the camera and the landscaping growth environment impact data collected by the sensor. It should be understood that the global landscaping maintenance feature vector can include various feature information extracted from the image, such as plant morphological characteristics, color characteristics, and texture characteristics, reflecting the overall landscaping maintenance status. Converting complex image information into vector form helps simplify the data and extract the most representative features, facilitating subsequent processing and analysis. The Gaussian density map of the landscaping growth environment can reflect the spatial distribution characteristics of the landscaping growth environment, such as the distribution of nutrient solution content, air temperature, and air humidity. Presenting environmental impact data as a Gaussian density map helps intuitively understand environmental changes and distribution patterns, facilitating environmental analysis. By extracting these feature vectors and density maps, the system can better understand the overall landscaping situation and environmental characteristics, providing a more accurate and comprehensive basis for subsequent decision-making and adjustments.

[0029] Figure 2 FIG is a block diagram of a landscaping maintenance data extraction module in a smart landscaping maintenance monitoring system according to an embodiment of the present application. Figure 2 As shown, in a specific embodiment of the present application, the garden greening maintenance data extraction module 120 includes: a garden greening maintenance feature extraction unit 121, which is used to perform feature extraction on the garden greening maintenance image collected by the camera to obtain the garden greening maintenance global feature vector; a greening growth environment impact feature extraction unit 122, which is used to perform feature extraction on the greening growth environment impact data collected by the sensor to obtain the greening growth environment Gaussian density map. It should be understood that the image contains a large amount of visual information, such as the morphology, color, texture, etc. of the plant. Through feature extraction, this information can be converted into usable data to help the system better understand the greening maintenance situation. Considering that image data is usually high-dimensional, feature extraction can convert complex image information into lower-dimensional feature vectors to facilitate subsequent processing and analysis. The garden greening maintenance global feature vector obtained by feature extraction can comprehensively reflect the characteristics of the entire image, rather than being limited to a local area, thereby providing a global perspective on the overall greening maintenance situation.

[0030] Furthermore, the data collected by sensors typically contains information about the spatial distribution of the greenery growth environment. By extracting features and generating Gaussian density maps, the spatial distribution of environmental impacts can be intuitively displayed, making the environmental data more visual and interpretable, helping users better understand the greenery growth environment and take appropriate measures to manage and improve it. Therefore, by extracting features from the greenery growth environment impact data collected by sensors, it is possible to quantitatively analyze the density distribution of the greenery growth environment, thereby identifying possible areas of dense concentration or sparseness, providing a reference for environmental management and adjustment, and making it easier for users to understand and analyze environmental characteristics.

[0031] In a specific embodiment of the present application, the landscape maintenance feature extraction unit 121 includes: passing the landscape maintenance image captured by the camera through a landscape maintenance feature extractor based on a spatial attention mechanism to obtain a landscape maintenance local feature map; and passing the landscape maintenance local feature map through a landscape maintenance feature encoder based on a non-local neural network to obtain the landscape maintenance global feature vector. It should be understood that the feature extractor based on the spatial attention mechanism can help the system focus on important local areas in the image, thereby better capturing local feature information, such as specific parts of plants. By extracting the local feature map, the feature expression of important areas in the image can be enhanced and the interference of irrelevant information in the image can be reduced, allowing the system to more centrally process key local information related to landscape maintenance, thereby more accurately understanding and identifying key information in the landscape maintenance image. The local feature map can help the system perform more fine-grained analysis, such as detecting the growth of green vegetation, thereby achieving more refined landscape maintenance monitoring. Specifically, the convolutional coding part of the landscape maintenance feature extractor based on the spatial attention mechanism is used to perform deep convolution coding on the landscape maintenance image captured by the camera to obtain an initial convolutional feature map; the initial convolutional feature map is input into the spatial attention part of the landscape maintenance feature extractor based on the spatial attention mechanism to obtain a spatial attention map; the spatial attention map is activated by the Softmax function to obtain a spatial attention feature map; the spatial attention feature map and the initial convolutional feature map are multiplied by the position points to obtain the landscape maintenance local feature map. More specifically, the initial convolutional feature map is subjected to average pooling and maximum pooling along the channel dimension to obtain an average feature matrix and a maximum feature matrix; the average feature matrix and the maximum feature matrix are cascaded and channel-adjusted to obtain a channel feature matrix; the channel feature matrix is convolutionally coded using the convolutional layer of the spatial attention feature map to obtain a spatial attention map.

[0032] Furthermore, by converting the local feature map into a global feature vector, the local feature information of each image can be integrated into a global feature representation to better reflect the comprehensive features of the entire image. Among them, the non-local neural network can capture the long-range dependencies between different positions in the image, so as to better understand the overall structure of the image and improve the representation and generalization capabilities of the features. Through such a processing flow, the image data of garden greening maintenance can be better represented and understood, providing more powerful support for subsequent analysis, recognition and decision-making. Specifically, the garden greening maintenance local feature map is input into the first point convolution layer, the second point convolution layer and the third point convolution layer of the garden greening maintenance feature encoder based on the non-local neural network to obtain the first feature map, the second feature map and the third feature map; the position-weighted sum of the first feature map and the second feature map is calculated to obtain an intermediate fusion feature map; the intermediate fusion feature map is input into the Softmax function to normalize the feature values of each position in the intermediate fusion feature map to obtain a normalized intermediate fusion feature map; the position-weighted sum of the normalized intermediate fusion feature map and the third feature map is calculated. The weighted sum is calculated to obtain a re-fused feature map; the re-fused feature map is embedded in a Gaussian similarity function to calculate the similarity between the eigenvalues of each position in the re-fused feature map to obtain a global perception feature map; the global perception feature map is passed through the fourth point convolution layer of the landscaping and maintenance feature encoder based on a non-local neural network to obtain a channel-adjusted global perception feature map; and the position-weighted sum of the channel-adjusted global perception feature map and the landscaping and maintenance local feature map is calculated to obtain a global correlation feature map; the global correlation feature map is pooled to obtain the landscaping and maintenance global feature vector.

[0033] Figure 3 FIG. 1 is a block diagram of a greening growth environment impact feature extraction unit in a smart garden greening maintenance monitoring system according to an embodiment of the present application. Figure 3As shown, in a specific embodiment of the present application, the greening growth environment impact feature extraction unit 122 includes: a greening growth environment impact time arrangement subunit 1221, which is used to arrange the greening growth environment impact data collected by the sensor according to the time dimension to obtain a greening growth environment input vector; a greening growth environment feature encoding subunit 1222, which is used to feature encode the greening growth environment input vector to obtain the greening growth environment Gaussian density map. It should be understood that the greening growth environment is affected by factors such as season, weather, and light, and these factors show a certain change pattern over time. By arranging data according to the time dimension, the changing trend of environmental factors over time can be better captured, and the time correlation between environmental factors can be revealed. Arranging data according to the time dimension can integrate data collected at different times to form a more complete time series data, which is convenient for unified processing and analysis, and improves the availability and application efficiency of the data.

[0034] Furthermore, feature encoding can transform the original high-dimensional input vector into a more representative and abstract feature representation, thereby better capturing the key characteristics and patterns of the greening growth environment. By performing Gaussian density estimation on the feature-encoded data, the characteristic information of the greening growth environment can be presented in the form of a Gaussian distribution, intuitively displaying the density distribution of the environment and facilitating further analysis and understanding. This processing flow can make the characteristic information of the greening growth environment more intuitive, providing stronger support for environmental monitoring, analysis, and decision-making.

[0035] In a specific embodiment of the present application, the greening growth environment feature encoding subunit 1222 includes: passing the greening growth environment input vector through a greening growth environment time series encoder based on a convolutional neural network to obtain a greening growth environment feature vector; and constructing a Gaussian density map of the greening growth environment feature vector to obtain the greening growth environment Gaussian density map. It should be understood that performing a convolution operation on the input vector of the greening growth environment can capture feature information at different locations and time points, which helps to more comprehensively understand the changes and characteristics of the greening growth environment. Through training, the network can learn the representation that can best distinguish different environmental features, thereby improving the expressive power and generalization ability of the features. In addition, the time series encoder can process time series data and retain the time series information in the data. By adding a time series encoder to the convolutional neural network, the information of the time dimension can be effectively utilized to capture the changing patterns of environmental data over time, thereby obtaining a feature representation with more temporal dynamics. By processing the greening growth environment input vector through a time series encoder based on a convolutional neural network, the spatiotemporal features in the environmental data can be better extracted, and a more representative and effective feature vector can be obtained, providing stronger support for subsequent environmental analysis, prediction and decision-making. Specifically, the greening growth environment input vector is fully connected using the fully connected layer of the greening growth environment time series encoder based on a convolutional neural network to extract the high-dimensional implicit features of the eigenvalues of each position in the greening growth environment input vector; and the greening growth environment input vector is one-dimensionally encoded using the one-dimensional convolutional layer of the greening growth environment time series encoder based on a convolutional neural network to extract the high-dimensional implicit correlation features of the correlation between the eigenvalues of each position in the greening growth environment input vector.

[0036] Furthermore, by constructing a Gaussian density map, the spatial distribution of the greening growth environment characteristic vector can be intuitively displayed. The Gaussian density map can provide information about greening density, concentration, and distribution range, helping people to more intuitively understand the spatial structure of the greening growth environment. The Gaussian density map has a smooth property and can continuously represent the density distribution of the greening growth environment characteristic vector in space, avoiding the influence of mutations and noise on the visualization results, making the results more stable and reliable. Specifically, the greening growth environment Gaussian density map is used to construct the greening growth environment characteristic vector using the following Gaussian construction formula; wherein, the Gaussian construction formula is; Among them, μ i represents the greening growth environment feature vector, ∑ i The covariance matrix representing the Gaussian density map of the greening growth environment.

[0037] In the above-mentioned intelligent garden greening maintenance monitoring system 100, the greening maintenance environment feature fusion module 130 is used to construct a garden greening maintenance fusion feature matrix between the garden greening maintenance global feature vector and the greening growth environment Gaussian density map. It should be understood that by fusing the garden greening maintenance global feature vector and the greening growth environment Gaussian density map into one feature matrix, the information of both can be comprehensively utilized to more comprehensively describe the garden greening maintenance situation. This helps to understand the status and needs of garden greening maintenance from different angles. Among them, the garden greening maintenance global feature vector usually contains information about the overall greening maintenance situation, while the greening growth environment Gaussian density map provides information about spatial distribution and density. Fusion of these two types of information can make the feature matrix have richer feature expression capabilities, better reflect the multifaceted characteristics of garden greening maintenance, and improve the performance and generalization ability of the model.

[0038] In a specific embodiment of the present application, the greening maintenance environment feature fusion module 130 includes: performing a Gaussian mixture of the garden greening maintenance global feature vector and the greening growth environment Gaussian density map to obtain a garden greening maintenance Gaussian mixture model; and performing Gaussian discretization on the garden greening maintenance Gaussian mixture model to obtain the garden greening maintenance fusion feature matrix. It should be understood that by combining the global feature vector and spatial distribution information, a more comprehensive description of the garden greening maintenance situation can be achieved. The Gaussian mixture model can simultaneously consider the contributions of both types of information, improving the model's expressive power. Specifically, the Gaussian mixture model has good flexibility and can adapt to different types of data distributions. In the field of garden greening maintenance, there may be different types of maintenance conditions and greening environment distributions, and the Gaussian mixture model can better adapt to this diversity. The Gaussian mixture model can flexibly fit data by adjusting the number and parameters of the mixture components, thereby better capturing the complex characteristics of garden greening maintenance. This facilitates modeling and analysis of garden greening maintenance conditions. By constructing a garden maintenance Gaussian mixture model, we can better integrate global features and spatial information, providing a more accurate and comprehensive reference for the evaluation, planning, and decision-making of garden greening maintenance. Specifically, based on the garden greening maintenance global feature vector and the greening growth environment Gaussian density map, the garden maintenance Gaussian mixture model is constructed using the following Gaussian mixture model formula:

[0039] Wherein, the Gaussian mixture model formula is:

[0040]

[0041] in:

[0042]

[0043] Among them, π kis the weighted coefficient of the Gaussian density map of the greening growth environment, x k is the observed data point between the eigenvalues of the corresponding two positions, μ k is the mean between the eigenvalues of the corresponding two positions, σ k is the variance between the eigenvalues at the corresponding two locations, F k is the eigenvalue at the kth position, represents the sum of each eigenvalue.

[0044] Furthermore, Gaussian mixture models typically contain continuous probability density functions. However, in practical applications, continuous probability distributions need to be converted into discrete forms. Gaussian discretization simplifies the model, making it easier to understand and process. Discretizing Gaussian mixture models can reduce computational complexity and speed up data processing. Gaussian discretization can convert continuous Gaussian mixture models into discrete feature representations, which helps extract features that are easier to understand and apply, providing more intuitive and practical information for subsequent analysis and decision-making.

[0045] In the above-mentioned intelligent garden greening maintenance monitoring system 100, the greening maintenance environment judgment module 140 is used to judge whether the garden greening maintenance environment needs to be adjusted based on the garden maintenance fusion feature matrix. It should be understood that the fusion feature matrix takes into account the characteristics of different aspects and can provide more comprehensive information to help evaluate the overall condition of the garden greening maintenance environment. By comprehensively considering various characteristics, it is possible to more accurately judge whether the maintenance environment needs to be adjusted. Based on the data of the feature matrix, quantitative analysis and comparison can be performed, rather than just subjective judgment. This helps to objectively evaluate the condition of the garden greening maintenance environment and determine whether adjustments are needed. Through the analysis of the feature matrix, problems and room for improvement in the garden greening maintenance environment can be identified more quickly, thereby improving the efficiency and accuracy of decision-making.

[0046] Figure 4 FIG is a block diagram of a greening maintenance environment judgment module in a smart garden greening maintenance monitoring system according to an embodiment of the present application. Figure 4 As shown, in a specific embodiment of the present application, the greening maintenance environment judgment module 140 includes: an optimization unit 141, which is used to perform feature space domain transformation modulation based on intrinsic mapping on the garden maintenance fusion feature matrix to obtain an optimized garden maintenance fusion feature vector; a garden greening maintenance adjustment judgment generation unit 142, which is used to pass the optimized garden maintenance fusion feature matrix through a classifier to obtain a classification result, and the classification result is used to judge whether the garden greening maintenance environment needs to be adjusted.

[0047] It's understandable that constructing a fused feature matrix for landscape maintenance is a key step in assessing the health of landscapes by fusing image features with environmental data. However, this process can overlook or underutilize the internal structural information of features. Because fused feature matrices are typically derived through linear or nonlinear combinations of multiple features, this approach may not fully capture the complex patterns and inherent connections within the original features. For example, local feature maps extracted using a spatial attention mechanism and sensor data arranged along the temporal dimension each represent different types of feature information, each describing the visual characteristics of landscapes and the influence of their growing environment. When these features are encoded into global and environmental feature vectors, and fused using methods such as Gaussian mixture models, this fusion approach may simply achieve a simple numerical superposition, failing to fully consider the underlying deep connections and dynamic variations among features. Therefore, for applications that rely on internal structural information for accurate analysis, this approach may underutilize subtle yet crucial feature information, compromising the accuracy and reliability of the final classification results. In addition, different plant species, soil conditions, and climatic factors may all affect the effectiveness of greening maintenance. These complex interactions are difficult to fully cover through a single fusion strategy, which further exacerbates the problem of neglecting or underutilizing the internal structural information of features. Based on this, in this application, the garden maintenance fusion feature matrix is modulated by a feature space domain transformation based on intrinsic mapping to obtain an optimized garden maintenance fusion feature vector.

[0048] Specifically, the garden maintenance fusion feature matrix is transformed and modulated in the feature space domain based on intrinsic mapping to obtain an optimized garden maintenance fusion feature vector, including: first, expanding the garden maintenance fusion feature matrix into a garden maintenance fusion feature vector, and constructing a pixel-level information interaction structure matrix of the garden maintenance fusion feature vector, which is expressed as follows:

[0049]

[0050] Wherein, V represents the garden maintenance fusion feature vector, v i and v j Respectively represent the eigenvalues of the i-th and j-th positions of the garden maintenance fusion feature vector, d(v i ,v j ) indicates calculating the Euclidean distance, D i,j Represents the eigenvalue of the (i, j) position of the pixel-level information interaction structure matrix.

[0051] That is, by expanding the garden maintenance fusion feature matrix into a garden maintenance fusion feature vector to reduce the feature dimension, and then constructing a pixel-level information interaction structure matrix of the garden maintenance fusion feature vector, the correlation strength or similarity between the feature information of each pixel position in the garden maintenance fusion feature vector is quantified, and the complex dependency structure within the features is explicitly encoded. This not only captures redundant information, but also reveals potential patterns, groupings, and the underlying organizational structure within the garden maintenance fusion feature vector, thereby transforming the garden maintenance fusion feature vector from a discrete unit description to an explicit relationship graph representation containing internal correlation relationships. This enables the garden maintenance fusion feature vector to more accurately reflect the intrinsic connection and deep characteristics between the garden greening maintenance image and the greening growth environment impact data, providing richer and more structured information support for subsequent judgments on whether the garden greening maintenance environment needs to be adjusted.

[0052] Secondly, based on the convolution layer, the pixel-level information interaction structure matrix is subjected to convolution kernel driven feature mining to obtain the garden maintenance fusion feature space domain dynamic response matrix, which is expressed as:

[0053] M=Conv(D)

[0054] Among them, D represents the pixel-level information interaction structure matrix, Conv represents the convolution layer, and M represents the dynamic response matrix of the garden maintenance fusion feature space domain.

[0055] Specifically, through kernel feature extraction at the convolutional layer, the convolution kernel focuses on identifying specific correlation patterns between feature positions within the pixel-level information interaction structure matrix. This allows the extraction of high-level, nonlinear structural information from the raw correlation information, which is useful for determining whether adjustments to the landscape maintenance environment are necessary. This provides rich learned context for subsequent processing. The resulting dynamic response matrix for the landscape maintenance fusion feature space domain encodes complex correlation patterns that go beyond simple correlation strengths, containing richer, higher-level structural information specific to the maintenance environment judgment task. This provides more accurate and semantically deeper feature support for subsequent fusion feature-based maintenance environment judgments.

[0056] Then, the pixel-level information interaction structure matrix is subjected to spectral domain decomposition analysis to obtain a set of potential component coding vectors of the garden maintenance fusion feature, which is expressed as follows:

[0057]

[0058] Where T represents the transpose of the vector, Λ represents the diagonal matrix, λ1 and λ m They represent the first and mth eigenvalues of the diagonal matrix respectively, U represents the set of potential component encoding vectors of the garden maintenance fusion feature, x1, x2, x mRepresent the first, i-th and m-th garden maintenance fusion feature latent component encoding vectors respectively.

[0059] That is, through spectral decomposition, the pixel-level information interaction structure matrix is decomposed into a set of initial feature eigencomponent encoding vectors, so that these vectors form an orthogonal basis of the linear space represented by the association matrix, thereby capturing the main change direction or inherent pattern of data association, thereby refining the core components that constitute the overall association structure, and providing a basic representation that can reflect the key correlation structure of the features for subsequent analysis. The generated set of latent component encoding vectors of the garden maintenance fusion features is used as an orthogonal basis, effectively capturing the main change direction and inherent pattern in the pixel-level information interaction structure matrix, simplifying the complex correlation structure into a set of orthogonal basic atomic patterns. It can essentially characterize the core correlation characteristics between garden maintenance images and growth environment data features, providing more concise and representative feature information for judging whether the maintenance environment needs to be adjusted.

[0060] Next, each garden maintenance fusion feature latent component coding vector in the set of garden maintenance fusion feature latent component coding vectors is input into the adaptive attention-guided saliency modulation module to obtain a set of garden maintenance fusion feature explicit component modulation coding vectors, which can be expressed as follows:

[0061] Y=Transformer{[x1,x2,…,x m ]}=[y1,y2,…,y m ]

[0062] Among them, Transformer represents a sequence model based on the self-attention mechanism, Y represents the set of modulation coding vectors of the dominant component of the garden maintenance fusion feature, y1, y2, y m They represent the first, i-th and m-th garden maintenance fusion feature explicit component modulation coding vectors respectively.

[0063] Specifically, the self-attention mechanism calculates the interdependencies between the encoding vectors of the latent components of the garden maintenance fusion features, learning which latent components of the garden maintenance fusion features are more "significant" or require special attention within the context of the current overall feature structure. This allows for dynamic and adaptive focusing on the underlying structural components within the features, enabling the model to flexibly enhance or suppress different latent components of the garden maintenance fusion features based on the global structural context, making their representations more suitable for the current data instance or maintenance environment judgment task requirements. The resulting set of modulation encoding vectors of the explicit components of the garden maintenance fusion features, after dynamic evaluation and adaptive adjustment, provides more targeted and effective feature support for subsequent judgments, improving the system's recognition accuracy and response efficiency for maintenance environment issues.

[0064] Then, each garden maintenance fusion feature explicit component modulation coding vector in the set of garden maintenance fusion feature explicit component modulation coding vectors is mapped to the garden maintenance fusion feature spatial domain dynamic response matrix to obtain a set of garden maintenance fusion feature latent component kernel state mask coding vectors, which is expressed as follows:

[0065]

[0066] in, represents matrix multiplication, S represents the characteristic scale of the dynamic response matrix of the garden maintenance fusion feature space domain, y i represents the modulation coding vector of the explicit component of the i-th garden maintenance fusion feature, L represents the length of the kernel state mask coding vector of the latent component of the garden maintenance fusion feature, z i Represents the kernel mask encoding vector of the i-th garden maintenance fusion feature potential component.

[0067] That is, through this generalized projection interaction, the global structural backbone is aware of and affected by local complex correlation patterns, achieving a deep fusion of global decoupled structural information and local data-driven correlation patterns, thereby generating an intermediate representation that not only retains the basic structural information but also incorporates the complex contextual dependencies mined by deep learning, namely, the kernel mask encoding vector of the latent component of the garden maintenance fusion feature, which effectively integrates the global structural backbone and the local complex correlation pattern, and contains both basic structural information such as the main change direction or inherent pattern of data correlation, and high-level nonlinear structural information beyond simple second-order statistics mined by the convolutional layer, providing more comprehensive and semantically deeper feature support for subsequent judgment of whether the garden greening maintenance environment needs to be adjusted, and further improving the pertinence and effectiveness of feature representation for maintenance environment issues.

[0068] Finally, the set of kernel mask encoding vectors of the potential components of the garden maintenance fusion feature is fused to obtain the optimized garden maintenance fusion feature vector, which is expressed as follows:

[0069] V'=Concat{z1,z2,…,z m}

[0070] Among them, Concat represents the cascade function, z1, z2, z m They represent the first, i-th and m-th garden maintenance fusion feature potential component kernel mask encoding vectors respectively, and V' represents the optimized garden maintenance fusion feature vector.

[0071] That is, a cascade approach is adopted to effectively aggregate the different structural perspective information contained in the kernel mask encoding vectors of the potential components of each garden maintenance fusion feature, as well as the information after saliency adjustment and kernel domain context injection, to form a unified, enhanced optimized garden maintenance fusion feature vector that can retain the independent information of each component and have compact comprehensive characteristics, providing comprehensive and integrated feature support for the subsequent judgment of whether the garden greening maintenance environment needs to be adjusted.

[0072] In summary, the embodiment of the present application first obtains the garden greening maintenance image collected by the camera and the greening growth environment impact data collected by the sensor, and then uses deep learning technology to perform feature extraction and correlation analysis on the two. Finally, the classification result is obtained through the classifier to determine whether the garden greening maintenance environment needs to be adjusted, so as to timely discover the problems of the garden greening maintenance environment and make adjustments, thereby improving the efficiency and effectiveness of garden greening maintenance, thereby realizing intelligent garden greening management.

[0073] As described above, the smart garden greening maintenance monitoring system 100 according to the embodiments of the present application can be implemented in various terminal devices, such as a server deployed with a smart garden greening maintenance monitoring algorithm. In one example, the smart garden greening maintenance monitoring system 100 can be integrated into a terminal device as a software module and / or a hardware module. For example, the smart garden greening maintenance monitoring system 100 can be a software module in the operating system of the terminal device, or it can be an application developed for the terminal device; of course, the smart garden greening maintenance monitoring system 100 can also be one of the many hardware modules of the terminal device.

[0074] Alternatively, in another example, the smart landscaping maintenance monitoring system 100 and the terminal device may also be separate devices, and the smart landscaping maintenance monitoring system 100 may be connected to the terminal device via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

[0075] Exemplary Methods

[0076] Figure 5 Flowchart of the intelligent landscaping maintenance monitoring method according to the embodiment of the present application. Figure 5As shown, the smart garden greening maintenance monitoring method according to the embodiment of the present application includes: S110, obtaining the garden greening maintenance image collected by the camera and the greening growth environment impact data collected by the sensor, wherein the greening growth environment impact data includes the nutrient solution content of the garden greening at multiple predetermined time points, the air temperature at multiple predetermined time points, and the air humidity at multiple predetermined time points; S120, extracting the garden greening maintenance global feature vector and the greening growth environment Gaussian density map from the garden greening maintenance image collected by the camera and the greening growth environment impact data collected by the sensor; S130, constructing a garden maintenance fusion feature matrix between the garden greening maintenance global feature vector and the greening growth environment Gaussian density map; S140, judging whether the garden greening maintenance environment needs to be adjusted based on the garden maintenance fusion feature matrix.

[0077] Here, those skilled in the art will appreciate that the specific operations of each step in the above-mentioned smart landscaping maintenance monitoring method have been described in detail in the reference to Figures 1 to 4 The description of the intelligent landscaping maintenance monitoring system has been introduced in detail, and therefore, its repeated description will be omitted.

[0078] Exemplary electronic devices

[0079] Below, reference Figure 6 To describe the electronic device according to the embodiment of the present application.

[0080] like Figure 6 As shown, the electronic device 10 includes an input device 11, an input interface 12, a central processing unit 13, a memory 14, an output interface 15, an output device 16, and a bus 17. The input interface 12, the central processing unit 13, the memory 14, and the output interface 15 are interconnected via the bus 17, and the input device 11 and the output device 16 are connected to the bus 17 via the input interface 12 and the output interface 15, respectively, and are further connected to other components of the electronic device 10.

[0081] Specifically, the input device 11 receives input information from the outside and transmits the input information to the central processing unit 13 through the input interface 12; the central processing unit 13 processes the input information based on the computer-executable instructions stored in the memory 14 to generate output information, stores the output information temporarily or permanently in the memory 14, and then transmits the output information to the output device 16 through the output interface 15; the output device 16 outputs the output information to the outside of the electronic device 10 for user use.

[0082] In one embodiment, Figure 6The electronic device 10 shown can be implemented as a network device, which may include: a memory configured to store programs; a processor configured to run the programs stored in the memory to execute any one of the smart landscaping maintenance monitoring methods described in the above embodiments.

[0083] According to an embodiment of the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, an embodiment of the present application includes a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program comprising program code for executing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network and / or installed from a removable storage medium.

[0084] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As is well known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable, and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those skilled in the art that communication media typically embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0085] It is understood that the above embodiments are merely exemplary embodiments for illustrating the principles of the present application, and the present application is not limited thereto. Those skilled in the art may make various modifications and improvements without departing from the spirit and substance of the present application, and such modifications and improvements are also considered to be within the scope of protection of the present application.

Claims

1. A smart landscaping maintenance monitoring system, characterized by: include: A garden greening maintenance data acquisition module is used to acquire garden greening maintenance images collected by the camera and greening growth environment impact data collected by the sensor, wherein the greening growth environment impact data includes the nutrient solution content of the garden greening at multiple predetermined time points, the air temperature at multiple predetermined time points, and the air humidity at multiple predetermined time points; A garden greening maintenance data extraction module is used to extract a garden greening maintenance global feature vector and a greening growth environment Gaussian density map from the garden greening maintenance image collected by the camera and the greening growth environment impact data collected by the sensor; A greening maintenance environment feature fusion module is used to construct a garden maintenance fusion feature matrix between the garden greening maintenance global feature vector and the greening growth environment Gaussian density map; The greening maintenance environment judgment module is used to judge whether the garden greening maintenance environment needs to be adjusted based on the garden maintenance fusion feature matrix.

2. The intelligent landscaping maintenance monitoring system according to claim 1 is characterized in that: The landscaping maintenance data extraction module includes: A garden greening maintenance feature extraction unit is used to extract features from the garden greening maintenance image captured by the camera to obtain the garden greening maintenance global feature vector; The greening growth environment impact feature extraction unit is used to extract features from the greening growth environment impact data collected by the sensor to obtain the greening growth environment Gaussian density map.

3. The intelligent landscaping maintenance monitoring system according to claim 2 is characterized in that: The garden greening maintenance feature extraction unit includes: The garden greening maintenance image captured by the camera is passed through a garden greening maintenance feature extractor based on a spatial attention mechanism to obtain a garden greening maintenance local feature map; The garden greening and maintenance local feature map is passed through a garden greening and maintenance feature encoder based on a non-local neural network to obtain the garden greening and maintenance global feature vector.

4. The intelligent landscaping maintenance monitoring system according to claim 3 is characterized in that: The greening growth environment impact feature extraction unit includes: a greening growth environment impact time arrangement subunit, configured to arrange the greening growth environment impact data collected by the sensor according to a time dimension to obtain a greening growth environment input vector; The greening growth environment feature encoding subunit is used to perform feature encoding on the greening growth environment input vector to obtain the greening growth environment Gaussian density map.

5. The intelligent landscaping maintenance monitoring system according to claim 4 is characterized in that: The greening growth environment characteristic coding subunit includes: Passing the greening growth environment input vector through a greening growth environment temporal encoder based on a convolutional neural network to obtain a greening growth environment feature vector; The greening growth environment feature vector is subjected to Gaussian density map construction to obtain the greening growth environment Gaussian density map.

6. The intelligent landscaping maintenance monitoring system according to claim 5 is characterized in that: The greening maintenance environment feature fusion module includes: Performing Gaussian mixing on the garden greening maintenance global feature vector and the greening growth environment Gaussian density map to obtain a garden greening maintenance Gaussian mixture model; The garden maintenance Gaussian mixture model is Gaussian discretized to obtain the garden maintenance fusion feature matrix.

7. The intelligent landscaping maintenance monitoring system according to claim 6 is characterized in that: The greening maintenance environment judgment module includes: An optimization unit, configured to perform feature space domain transformation modulation on the garden maintenance fusion feature matrix based on intrinsic mapping to obtain an optimized garden maintenance fusion feature vector; The garden greening maintenance adjustment judgment generation unit is used to pass the optimized garden greening maintenance fusion feature vector through a classifier to obtain a classification result, and the classification result is used to judge whether the garden greening maintenance environment needs to be adjusted.

8. The intelligent landscaping maintenance monitoring system according to claim 7 is characterized in that: The optimization unit comprises: Expanding the garden maintenance fusion feature matrix into a garden maintenance fusion feature vector, and constructing a pixel-level information interaction structure matrix of the garden maintenance fusion feature vector; Based on the convolution layer, convolution kernel driven feature mining is performed on the pixel-level information interaction structure matrix to obtain a garden maintenance fusion feature space domain dynamic response matrix; Performing spectral domain decomposition analysis on the pixel-level information interaction structure matrix to obtain a set of potential component coding vectors of garden maintenance fusion features; Inputting each garden maintenance fusion feature latent component coding vector in the set of garden maintenance fusion feature latent component coding vectors into an adaptive attention-guided saliency modulation module to obtain a set of garden maintenance fusion feature explicit component modulation coding vectors; Mapping each garden maintenance fusion feature explicit component modulation coding vector in the set of garden maintenance fusion feature explicit component modulation coding vectors to the garden maintenance fusion feature spatial domain dynamic response matrix to obtain a set of garden maintenance fusion feature latent component kernel state mask coding vectors; The set of kernel state mask coding vectors of the potential components of the garden maintenance fusion feature is fused to obtain the optimized garden maintenance fusion feature vector.

9. A smart landscaping maintenance monitoring method, characterized in that: include: Acquire landscaping maintenance images captured by a camera and greening growth environment impact data captured by a sensor, wherein the greening growth environment impact data includes nutrient solution content of the landscaping at multiple predetermined time points, air temperature at multiple predetermined time points, and air humidity at multiple predetermined time points; Extracting a garden greening maintenance global feature vector and a greening growth environment Gaussian density map from the garden greening maintenance image collected by the camera and the greening growth environment impact data collected by the sensor; Constructing a garden maintenance fusion feature matrix between the garden greening maintenance global feature vector and the greening growth environment Gaussian density map; Based on the garden maintenance fusion feature matrix, it is determined whether the garden greening maintenance environment needs to be adjusted.

10. The intelligent landscaping maintenance and monitoring method according to claim 9, characterized in that: Extracting a garden greening maintenance global feature vector and a greening growth environment Gaussian density map from the garden greening maintenance image collected by the camera and the greening growth environment impact data collected by the sensor, including: Performing feature extraction on the garden greening maintenance image captured by the camera to obtain the garden greening maintenance global feature vector; Feature extraction is performed on the greening growth environment impact data collected by the sensor to obtain the greening growth environment Gaussian density map.

Images