Landscaping maintenance monitoring and early warning method and system based on computer vision

Abstract

The invention discloses a landscaping maintenance monitoring and early warning method and system based on computer vision, particularly relates to the field of computer-aided identification, and is used for solving the problem that the existing landscaping maintenance depends on manual patrol and is difficult to find abnormal plant growth in time. The method comprises the following steps: acquiring a plant optical image, segmenting the plant optical image, extracting features such as shadow area proportion, compactness and gray average, and establishing a multi-scale shadow feature sequence; a shadow feature prediction model is trained by combining the environment and phenological data in the sliding time window; performing longitudinal and transverse residual calculation on the real-time observation features and the model prediction result, and fusing to generate an abnormal confidence score; enabling the abnormal confidence coefficient to correspond to a canopy projection region, and recognizing an abnormal region; and finally, the vegetation index difference ratio, texture and edge features of the abnormal region are extracted, the abnormal type is judged through a rule base, an early warning report is generated, and intelligent identification and efficient early warning of garden plant abnormity are realized.

Description

Technical Field

[0001] This invention relates to the field of computer-aided recognition technology, and more specifically, to a method and system for monitoring and early warning of landscaping maintenance based on computer vision. Background Technology

[0002] In the current process of garden and greening maintenance, the monitoring of plant growth status and early warning of abnormalities still mainly rely on manual inspection and experience judgment. This method is not only time-consuming and labor-intensive, but also often makes it difficult to detect early diseases, water shortages or pests in large-scale green areas. As a result, once the abnormal situation spreads, more manpower and resources need to be invested to remedy it.

[0003] In large-scale green spaces within urban landscapes, manual inspections suffer from insufficient coverage, poor timeliness, and strong subjectivity, lacking an objective, continuous, and quantifiable means of maintenance monitoring. Meanwhile, while existing sensor-based point-based monitoring can provide localized data, it struggles to comprehensively cover the entire green space, failing to achieve holistic monitoring of plant communities and anomaly localization, often resulting in isolated data and low spatial resolution. In recent years, computer vision technology has rapidly developed in agricultural and ecological environment monitoring. By processing and analyzing plant optical images, it can extract the spatial and optical characteristics of plant canopies without physical contact, offering new possibilities for monitoring plant growth.

[0004] However, research on related technologies in the field of landscape greening maintenance remains insufficient. In particular, there is a lack of systematic methods for multi-scale growth characteristic modeling, dynamic prediction, and anomaly identification of plant communities, making it difficult to meet the dual requirements of real-time performance and accuracy in actual maintenance management. Therefore, there is an urgent need for a computer vision-based solution that organically combines image features with anomaly identification to construct an automated landscape greening maintenance monitoring and early warning system, thereby achieving efficient, continuous, and forward-looking management in landscape greening maintenance. Summary of the Invention

[0005] In order to overcome the above-mentioned defects of the prior art, the embodiments of the present invention provide a computer vision-based method and system for monitoring and early warning of landscaping maintenance to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A computer vision-based method for monitoring and early warning of landscaping maintenance includes the following steps:

[0008] S1. Segment the optical images of plants in the garden greening into canopy projection areas, extract the canopy shadow area ratio, compactness and grayscale mean features, and establish a multi-scale shadow feature sequence.

[0009] S2. Using historical environmental and phenological data within a set sliding time window as input and shadow features as output, train a shadow feature prediction model.

[0010] S3. Compare the real-time shadow features of different canopy projection regions with the output of the shadow feature prediction model to calculate the longitudinal residual, and convert the position of the real-time shadow features in the population baseline distribution into the lateral residual. Then, fuse the two types of residuals to generate anomaly confidence scores.

[0011] S4. Spatial correspondence between the comprehensive anomaly confidence score and the plant canopy projection area, and identify anomaly areas within the canopy projection area;

[0012] S5. Calculate the vegetation index difference ratio, texture and edge features of the abnormal area, identify the abnormal type according to the preset rule base and generate an early warning report.

[0013] In a preferred embodiment, step S1, which involves segmenting the optical image of plants within the landscape into canopy projection areas and extracting features such as canopy shadow area ratio, compactness, and grayscale mean, to establish a multi-scale shadow feature sequence, specifically includes:

[0014] Semantic segmentation is performed on the optical images of plants in the garden green space to separate the plant canopy projection area from the ground background area, and the canopy projection area is divided into grids according to the actual location;

[0015] For the segmented canopy projection areas, calculate the ratio of the total number of canopy pixels in each canopy projection area to the total number of pixels in the entire plant optical image to obtain the shadow area ratio;

[0016] Calculate the ratio of the area of ​​the minimum bounding rectangle of the canopy projection region to the total number of its own pixels to obtain the shadow shape compactness;

[0017] Calculate the average grayscale value of all pixels within the canopy projection area to obtain the average grayscale value of the shadow.

[0018] Extract the shadow area ratio change sequence sampled at fixed time intervals from sunrise to sunset, and construct the shadow area diurnal variation curve as intra-diurnal scale feature;

[0019] The maximum daily shadow area percentage and the daily average shadow gray value are extracted as daytime scale features.

[0020] The maximum daily shaded area percentage is correlated with the daily growth accumulated temperature data to form a sequence of shaded features changing with growth accumulated temperature as a seasonal scale feature.

[0021] Intraday, interday, and seasonal scale features are integrated into a multi-scale shadow feature sequence.

[0022] In a preferred embodiment, step S2, using historical environmental and phenological data within a set sliding time window as input and shadow features as output, specifically includes training the shadow feature prediction model:

[0023] Historical environmental and phenological data extracted within a sliding time window of a set length are arranged in chronological order to construct a training dataset for training the shadow feature prediction model.

[0024] The input consists of a combination of environmental data and phenological period data arranged in a time series, and the corresponding shadow features are used as the model output.

[0025] The model employs a recurrent neural network with an encoder-decoder structure, where the encoder combines and compresses the input environmental and phenological data into a feature vector that reflects the historical state, and the decoder predicts the shadow features of future time steps based on this feature vector.

[0026] During training, the true shadow features of the previous time step are used as auxiliary inputs for the prediction of the next time step. The training is carried out iteratively in a recursive manner until the mean square error between the prediction result and the true observation value is minimized.

[0027] In a preferred embodiment, step S3 involves comparing the real-time shadow features of different canopy projection regions with the output of the shadow feature prediction model to calculate the longitudinal residual, and converting the position of the real-time shadow features in the population baseline distribution into the lateral residual. The fusion of these two types of residuals to generate anomaly confidence scores specifically includes:

[0028] The shadow features extracted from real-time observations are compared with the predicted output values ​​of the shadow feature prediction model to calculate the residuals and obtain the longitudinal residual sequence.

[0029] Calculate the percentile of the shadow feature extracted from real-time observation in the probability density function of the corresponding population baseline distribution, convert the percentile into a standard score and use the absolute value as the lateral residual;

[0030] Calculate the mean and standard deviation of the data in the longitudinal residual sequence, and convert the current longitudinal residual value into a standard score with the mean and standard deviation as parameters;

[0031] The weighted summation and normalization of the transformed longitudinal residual standard scores and the group lateral residual standard scores yield the comprehensive anomaly confidence score.

[0032] In a preferred embodiment, the population baseline distribution is established by extracting the shadow features of similar plant populations under the same time period and environmental conditions, and calculating the probability density function of the corresponding shadow feature set.

[0033] In a preferred embodiment, step S4 involves spatially mapping the comprehensive anomaly confidence score to the plant canopy projection area, and identifying anomaly areas within the canopy projection area specifically includes:

[0034] Spatial correspondence is established between the comprehensive anomaly confidence score and the gridded plant canopy projection area.

[0035] Based on a preset abnormal confidence threshold, the canopy projection region with a confidence score exceeding the abnormal confidence threshold is identified and marked as an abnormal region.

[0036] Image morphological closing operations are used to process abnormal regions and fill internal holes to generate a binary mask of the abnormal regions.

[0037] The geometric center coordinates of the abnormal region are calculated based on the mask and used as the spatial location. The binary mask of the abnormal region is combined with the spatial location coordinates and output to form a spatialized anomaly labeling result.

[0038] In a preferred embodiment, step S5, which involves calculating the vegetation index difference ratio, texture, and edge features of the abnormal area, and determining the anomaly type and generating an early warning report based on a preset rule base, specifically includes:

[0039] Acquire optical images of the plants in the anomalous area at the current time and at several similar times in the past;

[0040] Calculate the vegetation index in the visible light band of the abnormal area on the plant optical image, including the excess green index and the red-green-blue vegetation index;

[0041] Calculate the ratio of the current vegetation index to the vegetation index of the plant optical image at the same time in the past to obtain the vegetation index difference ratio;

[0042] Extract the internal texture features of the abnormal region in the current plant optical image, and calculate the contrast and entropy values ​​of its gray-level co-occurrence matrix;

[0043] Extract the mean gradient magnitude of the boundary of the abnormal region as the edge sharpness feature;

[0044] The vegetation index difference ratio, internal texture contrast, internal texture entropy value and edge sharpness features are input into a preset anomaly type discrimination rule library;

[0045] Based on the anomaly type identification results, a monitoring and early warning report containing spatial location and anomaly type labels is generated.

[0046] In a preferred embodiment, the anomaly type discrimination method is specifically as follows:

[0047] Constructed based on historical sample feature data, the sample feature data includes vegetation index difference ratio, texture contrast, texture entropy value and edge sharpness feature index of abnormal areas;

[0048] Based on the statistical distribution of historical sample feature data labeled with different anomaly types, threshold ranges and discrimination conditions for each anomaly type are generated, and a discrimination rule base is constructed.

[0049] The sample feature data extracted in real time within the abnormal region is matched in the discrimination rule base. If the threshold range and discrimination conditions corresponding to a certain abnormal type are met, the label of that abnormal type is output.

[0050] On the other hand, the present invention provides a computer vision-based monitoring and early warning system for landscaping maintenance, comprising:

[0051] The image processing module is used to perform semantic segmentation on optical images of garden plants, extract the canopy shadow area ratio, compactness and grayscale mean features, and establish a multi-scale shadow feature sequence.

[0052] The prediction modeling module is used to train a shadow feature prediction model and generate prediction results within a sliding time window, using historical environmental and phenological data as input and shadow features as output.

[0053] The anomaly detection module is used to compare real-time observation features with prediction results, calculate longitudinal and lateral residuals and fuse them to generate a comprehensive anomaly confidence score, thereby realizing the identification of anomaly regions.

[0054] The feature analysis module is used to calculate vegetation index difference ratio, texture contrast, texture entropy value and edge sharpness features for abnormal areas, and to identify the abnormal type in combination with the rule base.

[0055] The early warning output module is used to combine the abnormal location, abnormal type and judgment result to generate a monitoring early warning report, and output it to the maintenance management terminal for subsequent decision-making.

[0056] The technical effects and advantages of the computer vision-based monitoring and early warning method and system for landscaping maintenance in this invention are as follows:

[0057] By segmenting and extracting multi-scale shadow features from optical images of garden plants, and combining this with a time-series prediction model trained on historical environmental and phenological data, dynamic modeling and trend prediction of plant community growth processes are achieved, effectively avoiding the shortcomings of traditional manual inspections that rely on experience and have delayed responses. Simultaneously, real-time observation features are combined with prediction outputs to calculate longitudinal and lateral residuals and generate a comprehensive anomaly confidence score, reducing false alarms while ensuring detection sensitivity and improving the accuracy of anomaly identification. Through spatial mapping, anomaly results are mapped to canopy projection areas, enabling precise location of anomalies. Furthermore, by combining vegetation index difference ratios, texture, and edge features, anomaly type discrimination is performed, allowing the system to distinguish between different types of maintenance problems such as diseases, water shortages, or pests.

[0058] The method of this invention not only supports automated inspection of large-scale green areas, but also enables dynamic comparison and continuous tracking of multi-temporal images, possessing high efficiency and real-time capability. The system can generate structured early warning reports containing location, type, and confidence level, providing managers with intuitive, quantitative, and actionable decision-making support, thereby significantly improving the scientific nature, foresight, and resource utilization efficiency of landscaping maintenance. Attached Figure Description

[0059] Figure 1 This is a schematic diagram of a computer vision-based monitoring and early warning method for landscaping maintenance according to the present invention.

[0060] Figure 2 This is a schematic diagram of the structure of a computer vision-based monitoring and early warning system for landscaping maintenance according to the present invention. Detailed Implementation

[0061] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0062] Example 1

[0063] Figure 1 This invention presents a computer vision-based method for monitoring and early warning of landscaping maintenance, comprising the following steps:

[0064] S1. Segment the optical images of plants in the garden greening into canopy projection areas, extract the canopy shadow area ratio, compactness and grayscale mean features, and establish a multi-scale shadow feature sequence.

[0065] S2. Using historical environmental and phenological data within a set sliding time window as input and shadow features as output, train a shadow feature prediction model.

[0066] S3. Compare the real-time shadow features of different canopy projection regions with the output of the shadow feature prediction model to calculate the longitudinal residual, and convert the position of the real-time shadow features in the population baseline distribution into the lateral residual. Then, fuse the two types of residuals to generate anomaly confidence scores.

[0067] S4. Spatial correspondence between the comprehensive anomaly confidence score and the plant canopy projection area, and identify anomaly areas within the canopy projection area;

[0068] S5. Calculate the vegetation index difference ratio, texture and edge features of the abnormal area, identify the abnormal type according to the preset rule base and generate an early warning report.

[0069] In step S1, the optical images of plants in the garden greening are segmented into canopy projection areas, and the canopy shadow area ratio, compactness and grayscale mean features are extracted to establish a multi-scale shadow feature sequence.

[0070] Optical images of plants within a landscaped area are acquired. These images are color images captured under natural light conditions using fixed or mobile acquisition devices. Pixel-by-pixel semantic segmentation is performed on these images to separate the plant canopy projection region from background areas such as roads, lawns, and bare soil. Segmentation is typically based on differences in pixel color, brightness, and spatial relationships with surrounding pixels. A trained segmentation model classifies the input image, labeling each pixel as either "canopy" or "background." After separation, the canopy projection region is further divided into a regularized grid. The grid is created by mapping the entire canopy region to a pre-defined regular grid according to its actual spatial coordinates. For example, a 1m x 1m physical area corresponds to several pixel blocks in the image, ensuring that each grid unit has a unique spatial correspondence. This aims to establish a unified spatial index for each canopy region, facilitating subsequent unit-by-unit feature extraction and comparison.

[0071] After segmenting and meshing the canopy and background, pixel statistics are performed on the corresponding canopy shadow area within each grid cell. The method is as follows: determine which pixels in all grid cells are segmented and marked as canopy, and record the number of canopy pixels in that grid; then, using the total number of pixels in the entire plant optical image as the denominator, calculate the ratio of canopy pixels to the total number of pixels. This ratio is defined as the shadow area percentage, reflecting the canopy coverage. In practice, if the entire plant optical image contains 1000 pixels, and 60 pixels in a certain grid are marked as canopy in the canopy shadow area, then the shadow area percentage of that grid is 0.06. This calculation needs to be performed grid-by-grid, ultimately forming a spatialized shadow percentage distribution matrix.

[0072] Simultaneously, the shape characteristics of each canopy shadow region are geometrically described. Specifically, a minimum bounding rectangle is constructed for each canopy region; that is, a rectangle boundary is found that completely encompasses all canopy pixels in the region, ensuring the rectangle's area is minimized. The side length of the rectangle is determined by extracting the maximum and minimum coordinate range of the outer contour points of the canopy region. After obtaining the rectangle's area, it is compared with the actual total number of pixels in the canopy region, and the ratio, i.e., the shadow morphology compactness, is calculated. If the ratio is close to 1, it indicates that the canopy shape tends to be regular and compact, with relatively flat region boundaries; if the ratio is much greater than 1, it indicates that the canopy region has a large number of irregular edges or internal gaps.

[0073] The segmented canopy region is converted pixel by pixel into grayscale values, which are taken from the luminance channel of the color image or obtained through weighted RGB components. The grayscale values ​​of all pixels in the canopy region are statistically analyzed, summed, and then divided by the total number of pixels to obtain the mean grayscale value of the region. The mean grayscale value reflects the overall brightness level of the canopy region, indirectly indicating the light conditions and health status of the plant canopy.

[0074] The complete observation period from sunrise to sunset is extracted as the sampling interval for intraday data. This observation period is aligned based on the solar altitude and azimuth angles within different cycles. Image acquisition and processing are performed at fixed time intervals within the observation period, with the shadow area percentage of the canopy projection region acquired every 30 minutes. For each sampled image, a preliminary image segmentation and canopy region extraction process is performed, followed by calculation of the shadow area percentage of the canopy region at the current moment. As time progresses, a series of shadow area percentage values ​​arranged chronologically are obtained; these values, along with time labels, form a complete sequence. These sequences are plotted as a curve with time on the horizontal axis and shadow area percentage on the vertical axis, yielding the diurnal variation curve of the shadow area. This curve reflects the optical projection variation of the plant canopy within a daily light cycle.

[0075] After constructing the intraday sampling sequence, representative statistical features are extracted from this sequence to characterize the overall situation of the day. Specifically, the maximum value is selected from the shadow area percentage sequence corresponding to the diurnal variation curve, representing the highest performance of plant canopy coverage for that day. This value typically corresponds to the period of strongest light around noon and can represent the plant's extreme response under all-day light conditions. Simultaneously, the average grayscale value of the canopy region calculated for each sampling process throughout the day is statistically analyzed, summed, and then divided by the number of samplings to obtain a daily average grayscale value. This value serves as a diurnal-scale characteristic reflecting the overall optical brightness level, providing a stable observational indicator unaffected by local fluctuations.

[0076] To reflect the optical and growth responses of plants at seasonal scales, the maximum daily shaded area percentage was correlated with the corresponding accumulated temperature data for that date. The accumulated temperature data was obtained by subtracting the baseline temperature from the average daily temperature and then summing the results. In practice, the maximum daily shaded area percentage was recorded first in chronological order, followed by the corresponding accumulated temperature value. These two data items were then matched to generate a pairing sequence that accumulates over time. This process created a trend dataset of shaded features changing with accumulated temperature. Over time, this trend sequence reflects the overall evolutionary trajectory of the plant's projection characteristics throughout the growing season. After extracting features at the intraday, interday, and seasonal levels, the features at different scales were integrated to form a multi-scale shaded feature sequence.

[0077] In S2, the shadow feature prediction model is trained by taking historical environmental and phenological data within a set sliding time window as input and shadow features as output.

[0078] A fixed sliding window span is set for each time window. Within each window, environmental and phenological data related to garden plants are extracted from an existing monitoring database. The default range is 30 days to 1 year, with specific settings determined based on the phenological growth cycle of the plant type and the periodicity of the environmental data. Environmental data includes parameters directly related to growth conditions such as temperature, humidity, light intensity, and precipitation. Phenological data records the plant's growth stages at different times, such as germination, leaf expansion, flowering, and fruiting. Data extraction is strictly arranged in chronological order to ensure that each input sample reflects the true temporal evolution. Simultaneously, the extracted shadow features within this time period are used as the output target sequence, establishing a correspondence with the input data to construct a complete training dataset. Each sample contains both an input sequence and a corresponding output sequence. The construction of the training dataset is continuously performed using a sliding window, generating a large number of samples with each slide of a fixed step size, ensuring the stability and generalization ability of the model training.

[0079] After the training dataset is constructed, it is organized into a format that can be directly fed into the model. The input consists of a combination of environmental and phenological period data in time series form. The input vector at each time step contains multi-dimensional environmental factors and phenological stage codes. Within the entire time window, the input data is arranged in sequence, maintaining an order consistent with actual time. The output is a sequence of shadow features observed within the same time period. These shadow features correspond to the input data time step by time, forming the output of the training pairs. A recurrent neural network structure of encoder-decoder is chosen, which is suitable for handling time series prediction tasks. The encoder receives all time series inputs within the sliding window, sequentially feeding the time-by-time input vectors into the network units. Through the recurrent structure, information is accumulated and gradually compressed into a fixed-length feature vector, called the hidden state, which represents the historical state and evolution trend of the entire time window. After being processed by the encoder, the input sequence is compressed into a vector that carries long-term memory features of information such as temperature, humidity, light intensity, and phenological stage. The decoder receives this hidden state and generates shadow feature sequences for future time steps in a progressive prediction manner. Each time step of the decoder is updated based on the prediction and hidden state of the previous step until a complete prediction result is generated. Through this encoding and decoding mechanism, the model can generate output features that conform to temporal logic while preserving the historical features of the input sequence, thereby improving the continuity and accuracy of the prediction.

[0080] A recursive training strategy is employed during training. When predicting the output at the next time step, the decoder relies not only on the hidden state but also on the reference data from the previous time step. During training, the decoder receives the actual observed shadow features as input from the previous time step, rather than the predicted value from the previous step. This avoids model divergence due to the accumulation of prediction errors. Throughout the generation of the entire prediction sequence, the decoder learns the temporal dependencies using the true values ​​at each time step. The model's loss function is set as the mean squared error between the predicted and true values, calculated by averaging the squared differences between the predicted shadow features and the actual observed features at each time step. The model iteratively updates its parameters using the backpropagation algorithm, reducing the error value with each iteration until the convergence criterion is met. The convergence criterion is set to be no higher than 0.001 and the coefficient of determination no lower than 0.9. The model is considered converged when this condition is met or the maximum number of iterations is reached.

[0081] In step S3, the real-time shadow features of different canopy projection regions are compared with the output of the shadow feature prediction model to calculate the longitudinal residual, and the position of the real-time shadow features in the population baseline distribution is converted into the lateral residual. The two types of residuals are fused to generate an anomaly confidence score.

[0082] The shadow features extracted from real-time observations are compared with the output of a pre-trained shadow feature prediction model. Real-time shadow features refer to multi-scale shadow parameters obtained through segmentation and analysis of currently acquired plant optical images, including statistical results at different time scales such as shadow area ratio, compactness, and mean grayscale. These observations are arranged sequentially on the time axis and can be directly correlated one-to-one with the corresponding feature values ​​output by the prediction model at the same time point. The residuals are calculated by directly subtracting the predicted values ​​from the observed values, resulting in a difference sequence that reflects the degree of deviation of the current plant growth status from the model prediction. By calculating over all time points, a complete longitudinal residual sequence can be obtained.

[0083] After calculating the longitudinal residuals, a transverse residual is established to measure the deviation between the plant population in a single area and the overall population level. This relies on a pre-established population baseline distribution, which is statistically formed by analyzing the shadow characteristics of similar plant populations under the same time period and environmental conditions (with tolerance ranges set based on environmental data for the region). Specifically, features such as shadow area percentage, morphological compactness, and mean grayscale value are extracted from a large number of plants. These data are then aggregated, and a probability density estimation method is used to construct the population baseline distribution. For example, in a park lawn area, researchers acquire optical images of 100 lawn plants in the same area under stable environmental conditions of 25℃ and 60% relative humidity, and extract their canopy projection areas through semantic segmentation. Subsequently, the shadow area percentage of each plant is calculated sequentially, resulting in a set of numerical samples, such as 0.32, 0.35, 0.37, 0.30, 0.41, etc., totaling 100 data points. A kernel density estimation algorithm is used, employing a Gaussian kernel with a bandwidth parameter set to 0.02, to calculate the probability density function of this sample set. A Gaussian kernel was used as the kernel function, with a bandwidth parameter set to 0.02. The calculation yielded a continuous probability distribution curve, with a peak value around 0.34, indicating that the majority of the shaded area of ​​the lawn plants is concentrated in the range of 0.32-0.36, with the tail extending to 0.28-0.42. This distribution curve represents the baseline distribution of the shaded area proportion for the population. The same method was applied to the compactness and gray-scale mean features. The above calculations yielded a continuous probability distribution curve.

[0084] The shadow features extracted from real-time observations are substituted into the corresponding probability distribution curve to determine their position within the overall distribution, i.e., their percentile. For example, if the shadow area percentage of a plant falls at the 90th percentile, the current feature value deviates significantly from the normal range of the population. By referencing the distribution mean and variance, the percentiles are converted into a more intuitive numerical expression, transforming them into standard scores to obtain a value reflecting the degree of deviation. Finally, the absolute value of this standard score is taken as the horizontal residual, used to represent the relative degree of abnormality of the plant within the population background.

[0085] Before fusing the longitudinal and transverse residuals, the longitudinal residuals are standardized. Specifically, data from the most recent time window in the longitudinal residual sequence are selected, and their mean and standard deviation are calculated. These are then used as parameters to convert the current longitudinal residual value into a standardized score. The standardized scores of the longitudinal and transverse residuals are then summed according to a preset weighting method. The weights are flexibly set based on the model's prediction accuracy and the corresponding environmental data changes. The result is then normalized to a value range of 0 to 1. This value is the comprehensive anomaly confidence score, indicating the probability that the current plant exhibits anomalies in both its longitudinal growth trend and transverse population comparison.

[0086] In step S4, the comprehensive anomaly confidence score is spatially correlated with the plant canopy projection area to identify anomaly areas within the canopy projection area.

[0087] The overall anomaly confidence score is spatially mapped to the gridded plant canopy projection area. This spatial mapping is based on mapping the overall anomaly confidence score to each grid cell. This mapping process is accomplished by establishing a correspondence between the confidence score and the pixel region. Specifically, when calculating the anomaly confidence score, independent features are calculated for each canopy grid cell, thus each cell receives a unique score. After mapping all scores back to image space, a confidence distribution map is formed, where each grid cell's pixel region corresponds to a set of score values. Next, an anomaly confidence threshold is set. This threshold is flexibly set based on the overall distribution of anomaly confidence calculation results across the entire garden; for example, 0.7 is used as an example value. When the confidence score of a certain grid cell exceeds this threshold, the cell is identified as an anomaly area and then distinguished by a prominent marker in the canopy projection area.

[0088] After initial identification of anomalous regions, further image-level processing is performed to ensure that the boundaries and shapes of these regions meet the requirements of subsequent calculations. Specifically, image morphological closing operations are applied to the anomalous regions. Closing is an image manipulation method that involves first dilating and then eroding, suitable for filling small holes within the target area. During implementation, dilation is performed on the pixel regions marked as anomalous, slightly expanding the boundaries of the regions to fill small gaps between boundaries or small internal holes. Subsequently, erosion is performed on the dilated result to shrink the boundaries back to a reasonable range, thus eliminating fragmentation caused by noise or discrete pixels while maintaining the overall shape. Through closing processing, a binary mask with a complete internal structure and smooth boundaries is obtained. This mask uses 0 and 1 pixel values ​​to distinguish between the background and anomalous regions, providing standardized input for subsequent geometric feature calculations. When filling holes, the maximum fillable area is set to no more than 3 pixels to avoid covering anomalous features such as insect-damaged perforations during noise removal.

[0089] After obtaining the binary mask, the geometric features of the anomalous region are calculated based on the mask to form a spatial annotation result. The geometric center point of the masked region is extracted. The geometric center point is calculated by summing the coordinates of all pixels belonging to the anomalous region and then dividing by the total number of pixels to obtain the spatial coordinates of the center point. This coordinate can be directly mapped to the georeferenced reference system of the original image to label the specific location of the anomalous region. Next, the physical area of ​​the anomalous region is calculated. Specifically, the total number of pixels within the mask is counted, and the area conversion factor represented by each pixel in the actual scene is used for calculation. For example, if a single pixel corresponds to 0.5 square centimeters, the actual area can be obtained through multiplication. Finally, the binary mask of the anomalous region, the coordinates of the geometric center point, and the area index are combined and output to form a complete spatial anomaly annotation result.

[0090] In step S5, the vegetation index difference ratio, texture and edge features of the abnormal area are calculated, and the abnormality type is identified and an early warning report is generated according to the preset rule base.

[0091] Acquire plant optical images of the anomalous area at the current time and at several past times of the same duration. "Same times" refers to fixed time points after daily alignment, ensuring consistent illumination angles and environmental conditions and avoiding feature deviations caused by changes in illumination. To ensure continuity of comparative analysis, a fixed historical sampling length is set, for example, selecting images from the same time points over the past 14 days as comparative data. Within these images, registration operations are used to strictly align the positions of the anomalous area across different images, ensuring that each calculation targets the same spatial region. After alignment, vegetation indices in the visible light band of the plant optical images for the anomalous area are calculated, including the excess green index and the red-green-blue vegetation index. The excess green index is calculated based on the comparison of the green channel value with the red and blue channels, reflecting the degree of vegetation enhancement in the green band. The red-green-blue vegetation index combines the proportional relationship of the three channels to measure overall vegetation activity. By calculating these indices in the current and historical images respectively, a time-series-based set of vegetation indices can be obtained.

[0092] After extracting vegetation indices in the visible light band, the vegetation index difference ratio is calculated to measure the magnitude of change between the current anomalous area and historical levels. The difference ratio is calculated by comparing the vegetation index obtained from the current image with the vegetation index at several past moments. For example, the current excess green index is divided by the average excess green index over the past week to obtain the excess green index difference ratio. The same method applies to the red-green-blue vegetation indices. The difference ratio reflects the degree of abnormality in plant conditions. For example, if the difference ratio is significantly lower than 1, it indicates a significant decrease in current vegetation greenness, which may be related to diseases or water shortages. To avoid interference from extreme values ​​when calculating the difference ratio, a statistical rule is set, such as removing outliers from historical indices and retaining only data distributed within the range of mean ± 2 standard deviations, then calculating their average as a reference benchmark. The final difference ratio is stored in numerical form and can be directly used for joint discrimination with other features to ensure the objectivity and stability of anomaly detection.

[0093] Texture and edge features of anomalous regions are extracted to supplement the assessment of vegetation health. Texture features within anomalous regions are extracted from the current image, specifically through calculations based on the gray-level co-occurrence matrix (GLCM). The GLCM is a statistical tool describing the spatial relationships between pixel gray values, reflecting the complexity of details within a region. In this implementation, contrast and entropy are selected as texture features. Contrast represents the magnitude of gray value changes; a higher value indicates more significant texture differences within the region. Entropy represents the complexity of gray-level distribution; a higher value indicates more disordered texture. Subsequently, the sharpness features of the anomalous region boundaries are calculated by extracting the gradient magnitude of boundary pixels and calculating their mean. A higher gradient magnitude indicates a clear and sharp boundary, while a lower value indicates a blurred boundary or signs of wilting and drooping. Through comprehensive analysis of contrast, entropy, and edge sharpness, multi-dimensional feature support can be provided for anomaly type identification. This process ensures that judgments rely not only on vegetation indices but also incorporate supplementary indicators at the texture and geometric levels, thereby improving the comprehensiveness and accuracy of anomaly identification.

[0094] The aforementioned multi-dimensional feature indicators are input into a pre-defined anomaly type discrimination rule base. These feature indicators include vegetation index difference ratio, internal texture contrast, internal texture entropy value, and edge sharpness feature. The vegetation index difference ratio reflects the degree of difference in vegetation spectral characteristics between the current time and the same historical time in the anomaly area; internal texture contrast and entropy value measure the uniformity and complexity of the internal structure of the area, respectively, and can reflect whether there are local lesions or signs of pests in the plants; the edge sharpness feature focuses on the morphological characteristics of the boundary of the anomaly area. When plants wilt or wither, their edge sharpness often decreases.

[0095] The rule base is constructed based on a large amount of historical sample feature data. These samples were labeled with specific anomaly types by landscape maintenance personnel or plant experts during collection, such as disease, wilting due to water shortage, or insect damage. When building the rule base, indicators such as vegetation index difference ratio, texture contrast, texture entropy value, and edge sharpness of each type of anomaly sample are extracted to form a complete set of sample feature vectors. Then, statistical distribution analysis is performed on these sample feature vectors to obtain the mean, standard deviation, and percentile intervals of each indicator. Based on these statistical results, threshold intervals and discrimination conditions for each anomaly type can be generated. For example, leaf disease samples are characterized by a vegetation index difference ratio significantly lower than the normal threshold (e.g., below 0.8) and a high internal texture contrast (e.g., above 30), indicating decreased leaf greenness and the appearance of irregular spots. Water-deficiency wilting samples are characterized by a red-green-blue vegetation index difference ratio lower than a set threshold (e.g., 0.85) and an average edge sharpness lower than a preset value (e.g., below 15), indicating overall plant dehydration leading to blurred and drooping edges. Insect-damaged perforation samples are often characterized by an internal texture entropy value higher than the insect-damaged feature threshold (e.g., above 4.5), while the abnormal area is smaller than the set perforation area threshold (e.g., below 20 square centimeters), indicating the presence of small, more random perforations in localized areas. Through this method, each type of anomaly corresponds to a set of threshold intervals or combinations of conditions, which are stored in a rule base, forming a correspondence between anomaly types and feature indicators.

[0096] In the actual discrimination process, the real-time sample feature data extracted from the abnormal area at the current moment is matched one by one with the threshold intervals defined in the rule base. Specifically, each real-time feature is checked to see if it falls within the discrimination criteria range of a certain abnormality type. If so, the corresponding abnormality type label discrimination result is output. After the abnormality type discrimination is completed, the results are combined with the spatial location to generate a complete monitoring and early warning report. The early warning report should include the spatial coordinates of the abnormal area, the abnormality type label, the abnormal area index, and the corresponding comprehensive abnormality confidence score. The report is output in the form of structured data, which can not only intuitively display the abnormality distribution on the visualization interface, but also provide maintenance personnel with decision-making references for greening maintenance at the data level.

[0097] Example 2

[0098] The difference between Embodiment 2 and Embodiment 1 is that this embodiment introduces a computer vision-based monitoring and early warning system for landscaping maintenance.

[0099] Figure 2 A schematic diagram of a computer vision-based monitoring and early warning system for landscape greening maintenance is provided. The system includes:

[0100] The image processing module is used to perform semantic segmentation on optical images of garden plants, extract the canopy shadow area ratio, compactness and grayscale mean features, and establish a multi-scale shadow feature sequence.

[0101] The prediction modeling module is used to train a shadow feature prediction model and generate prediction results within a sliding time window, using historical environmental and phenological data as input and shadow features as output.

[0102] The anomaly detection module is used to compare real-time observation features with prediction results, calculate longitudinal and lateral residuals and fuse them to generate a comprehensive anomaly confidence score, thereby realizing the identification of anomaly regions.

[0103] The feature analysis module is used to calculate vegetation index difference ratio, texture contrast, texture entropy value and edge sharpness features for abnormal areas, and to identify the abnormal type in combination with the rule base.

[0104] The early warning output module is used to combine the abnormal location, abnormal type and judgment result to generate a monitoring early warning report, and output it to the maintenance management terminal for subsequent decision-making.

[0105] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters and thresholds in the formulas are set by those skilled in the art according to the actual situation.

[0106] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0107] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0108] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0109] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0110] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0111] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0112] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0113] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0114] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A computer vision-based method for monitoring and early warning of landscaping maintenance, characterized in that, Includes the following steps: S1. Segment the optical images of plants in the garden greening into canopy projection areas, extract the canopy shadow area ratio, compactness and grayscale mean features, and establish a multi-scale shadow feature sequence. S2. Using historical environmental and phenological data within a set sliding time window as input and shadow features as output, train a shadow feature prediction model. S3. Compare the real-time shadow features of different canopy projection regions with the output of the shadow feature prediction model to calculate the longitudinal residual, and convert the position of the real-time shadow features in the population baseline distribution into the lateral residual. Then, fuse the two types of residuals to generate anomaly confidence scores. S4. Spatial correspondence between the comprehensive anomaly confidence score and the plant canopy projection area, and identify anomaly areas within the canopy projection area; S5. Calculate the vegetation index difference ratio, texture and edge features of the abnormal area, identify the abnormal type according to the preset rule base and generate an early warning report.

2. The method for monitoring and early warning of landscaping maintenance based on computer vision according to claim 1, characterized in that, In step S1, the optical images of plants within the landscape are segmented into canopy projection regions, and the characteristics of canopy shadow area ratio, compactness, and grayscale mean are extracted to establish a multi-scale shadow feature sequence, specifically including: Semantic segmentation is performed on the optical images of plants in the garden green space to separate the plant canopy projection area from the ground background area, and the canopy projection area is divided into grids according to the actual location; For the segmented canopy projection areas, calculate the ratio of the total number of canopy pixels in each canopy projection area to the total number of pixels in the entire plant optical image to obtain the shadow area ratio; Calculate the ratio of the area of ​​the minimum bounding rectangle of the canopy projection region to the total number of its own pixels to obtain the shadow shape compactness; Calculate the average grayscale value of all pixels within the canopy projection area to obtain the average grayscale value of the shadow. Extract the shadow area ratio change sequence sampled at fixed time intervals from sunrise to sunset, and construct the shadow area diurnal variation curve as intra-diurnal scale feature; The maximum daily shadow area percentage and the daily average shadow gray value are extracted as daytime scale features. The maximum daily shaded area percentage is correlated with the daily growth accumulated temperature data to form a sequence of shaded features changing with growth accumulated temperature as a seasonal scale feature. Intraday, interday, and seasonal scale features are integrated into a multi-scale shadow feature sequence.

3. The method for monitoring and early warning of landscaping maintenance based on computer vision according to claim 1, characterized in that, In step S2, the training of the shadow feature prediction model, using historical environmental and phenological data within a set sliding time window as input and shadow features as output, specifically includes: Historical environmental and phenological data extracted within a sliding time window of a set length are arranged in chronological order to construct a training dataset for training the shadow feature prediction model. The input consists of a combination of environmental data and phenological period data arranged in a time series, and the corresponding shadow features are used as the model output. The model employs a recurrent neural network with an encoder-decoder structure, where the encoder combines and compresses the input environmental and phenological data into a feature vector that reflects the historical state, and the decoder predicts the shadow features of future time steps based on this feature vector. During training, the true shadow features from the previous time step are used as auxiliary inputs for the prediction at the next time step. The training is iterated in a recursive manner until the mean square error between the prediction result and the true observation is minimized.

4. The method for monitoring and early warning of landscaping maintenance based on computer vision according to claim 1, characterized in that, In step S3, the vertical residual is calculated by comparing the real-time shadow features of different canopy projection regions with the output of the shadow feature prediction model, and the position of the real-time shadow features in the population baseline distribution is transformed into the horizontal residual. The anomalous confidence score is generated by fusing the two types of residuals. Specifically, this includes: The shadow features extracted from real-time observations are compared with the predicted output values ​​of the shadow feature prediction model to calculate the residuals and obtain the longitudinal residual sequence. Calculate the percentile of the shadow feature extracted from real-time observation in the probability density function of the corresponding population baseline distribution, convert the percentile into a standard score and use the absolute value as the lateral residual; Calculate the mean and standard deviation of the data in the longitudinal residual sequence, and convert the current longitudinal residual value into a standard score with the mean and standard deviation as parameters; The weighted summation and normalization of the transformed longitudinal residual standard scores and the group lateral residual standard scores yield the comprehensive anomaly confidence score.

5. The method for monitoring and early warning of landscaping maintenance based on computer vision according to claim 4, characterized in that, The population baseline distribution is established by extracting the shadow features of similar plant populations under the same time period and environmental conditions, and calculating the probability density function of the corresponding shadow feature set.

6. The method for monitoring and early warning of landscaping maintenance based on computer vision according to claim 1, characterized in that, In step S4, the comprehensive anomaly confidence score is spatially mapped to the plant canopy projection area. Identifying anomaly areas within the canopy projection area specifically includes: Spatial correspondence is established between the comprehensive anomaly confidence score and the gridded plant canopy projection area. Based on a preset abnormal confidence threshold, the canopy projection region with a confidence score exceeding the abnormal confidence threshold is identified and marked as an abnormal region. Image morphological closing operations are used to process abnormal regions and fill internal holes to generate a binary mask of the abnormal regions. The geometric center coordinates of the abnormal region are calculated based on the mask and used as the spatial location. The binary mask of the abnormal region is combined with the spatial location coordinates and output to form a spatialized anomaly labeling result.

7. The method for monitoring and early warning of landscaping maintenance based on computer vision according to claim 1, characterized in that, In step S5, calculating the vegetation index difference ratio, texture, and edge features of the abnormal area, and determining the anomaly type and generating an early warning report based on a preset rule base specifically includes: Acquire optical images of the plants in the anomalous area at the current time and at several similar times in the past; Calculate the vegetation index in the visible light band of the abnormal area on the plant optical image, including the excess green index and the red-green-blue vegetation index; Calculate the ratio of the current vegetation index to the vegetation index of the plant optical image at the same time in the past to obtain the vegetation index difference ratio; Extract the internal texture features of the abnormal region in the current plant optical image, and calculate the contrast and entropy values ​​of its gray-level co-occurrence matrix; Extract the mean gradient magnitude of the boundary of the abnormal region as the edge sharpness feature; The vegetation index difference ratio, internal texture contrast, internal texture entropy value and edge sharpness features are input into a preset anomaly type discrimination rule library; Based on the anomaly type identification results, a monitoring and early warning report containing spatial location and anomaly type labels is generated.

8. The method for monitoring and early warning of landscaping maintenance based on computer vision according to claim 7, characterized in that, The specific method for determining the anomaly type is as follows: Constructed based on historical sample feature data, the sample feature data includes vegetation index difference ratio, texture contrast, texture entropy value and edge sharpness feature index of abnormal areas; Based on the statistical distribution of historical sample feature data labeled with different anomaly types, threshold ranges and discrimination conditions for each anomaly type are generated, and a discrimination rule base is constructed. The sample feature data extracted in real time within the abnormal region is matched in the discrimination rule base. If the threshold range and discrimination conditions corresponding to a certain abnormal type are met, the label of that abnormal type is output.

9. A computer vision-based monitoring and early warning system for landscape greening maintenance, used to implement the computer vision-based monitoring and early warning method for landscape greening maintenance as described in any one of claims 1-8, characterized in that, include: The image processing module is used to perform semantic segmentation on optical images of garden plants, extract the canopy shadow area ratio, compactness and grayscale mean features, and establish a multi-scale shadow feature sequence. The prediction modeling module is used to train a shadow feature prediction model and generate prediction results within a sliding time window, using historical environmental and phenological data as input and shadow features as output. The anomaly detection module is used to compare real-time observation features with prediction results, calculate longitudinal and lateral residuals and fuse them to generate a comprehensive anomaly confidence score, thereby realizing the identification of anomaly regions. The feature analysis module is used to calculate vegetation index difference ratio, texture contrast, texture entropy value and edge sharpness features for abnormal areas, and to identify the abnormal type in combination with the rule base. The early warning output module is used to combine the abnormal location, abnormal type and judgment result to generate a monitoring early warning report, and output it to the maintenance management terminal for subsequent decision-making.

Images