Landscaping maintenance monitoring and early warning system and method

Abstract

The invention discloses a landscaping maintenance monitoring and early warning system and method, and belongs to the technical field of landscaping maintenance monitoring. The system comprises a sensor module which collects garden environment parameters related to plant health in real time; the image acquisition module is responsible for regularly acquiring high-resolution garden panoramic images; the image data processing module is used for dividing the garden panoramic image into a plurality of small blocks with fixed sizes and extracting a feature vector of each small block; the sensor data processing module is used for performing spatial mapping on sensor data and the small image blocks; the feature fusion module is used for generating a comprehensive feature vector of each small block; the anomaly detection module is used for calculating the anomaly probability of each area in the image and mapping the anomaly probability back to the garden panoramic image; the anomaly determination module inputs the regional image data with the high anomaly probability into a CNN model, identifies the anomaly type, and deduces the cause of the anomaly according to the sensor data; and the abnormity early warning module is used for marking an abnormal area on the garden panoramic image.

Description

Technical Field

[0001] The present application relates to the technical field of greening maintenance monitoring, and more specifically, to a garden greening maintenance monitoring and early warning system and method. Background Art

[0002] With the acceleration of modern urbanization and the improvement of people's requirements for living environment, the status of gardening and greening in the urban ecosystem is becoming more and more important. Gardening and greening can not only beautify the environment and enhance the image of the city, but also improve air quality, reduce temperature, absorb noise and other ecological benefits. However, gardening and greening maintenance involves many aspects such as plant growth monitoring, pest control, irrigation management, soil nutrient regulation, etc., and is affected by many factors such as season, climate, soil environment, etc. Traditional greening maintenance work usually relies on manual experience, requires high skills of maintenance personnel, and it is difficult to guarantee real-time and accuracy. This manual-dependent method has low maintenance efficiency in scenes with large maintenance areas or complex plant species, and it is difficult to detect problems in time and take effective intervention measures.

[0003] In recent years, with the rapid development of Internet of Things technology, sensor technology and big data analysis technology, garden maintenance monitoring methods based on sensor networks have gradually attracted attention. By deploying multiple types of sensors in green areas, parameters such as plant growth, soil moisture, temperature, and light intensity can be collected, and trend analysis and status evaluation can be combined with historical data to achieve accurate monitoring and intelligent early warning of garden greening. For example, soil moisture sensors can help determine soil moisture content and plan irrigation rationally; temperature and light sensors can optimize plant maintenance measures in combination with seasonal changes; and pest and disease sensors can achieve early warning of pests and diseases. However, most of the current garden greening monitoring systems are single data collection modes, which fail to achieve multi-dimensional fusion and comprehensive analysis of data, and cannot dynamically evaluate the health status of plants in real time. In addition, the adaptability and intelligence level of the system still need to be improved.

[0004] In the existing garden monitoring system, data collection, transmission, processing and analysis are mostly carried out in a decentralized manner, and a complete closed-loop intelligent maintenance management system has not been formed. In addition, in the face of sudden environmental changes or pests and diseases, the existing system has limited response speed, making it difficult to achieve real-time dynamic warning and preventive maintenance adjustments. The lack of a unified scheduling management and early warning mechanism leads to low efficiency in garden greening maintenance management, increased labor costs, and easy impact on plant health and landscape effects. Therefore, designing a garden greening maintenance system with multi-parameter monitoring, intelligent analysis and real-time early warning functions is of great practical significance for improving the maintenance efficiency and quality of garden greening and optimizing resource allocation.

[0005] To sum up, how to build an intelligent maintenance system that can efficiently and accurately monitor the status of landscaping and provide timely warnings has become a technical problem that needs to be solved urgently. Summary of the invention

[0006] In order to overcome a series of defects in the prior art, the purpose of this application is to provide a garden greening maintenance monitoring and early warning system for the above-mentioned problems, including the following modules.

[0007] The sensor module collects garden environment parameters related to plant health in real time through sensors deployed in a grid form.

[0008] The image acquisition module is responsible for regularly collecting high-resolution panoramic images of the garden to obtain a comprehensive picture of the plant growth status.

[0009] The image data processing module divides the garden panoramic image into multiple small blocks of fixed size and extracts the feature vector of each small block through linear transformation and position coding.

[0010] The sensor data processing module spatially maps the sensor data with small image patches and uses an interpolation algorithm to supplement the sparse areas of the sensor grid to generate a continuous environmental parameter distribution map.

[0011] The feature fusion module fuses the image features of each small block with the corresponding sensor data to generate a comprehensive feature vector for each small block.

[0012] The anomaly detection module uses the ViT model to learn the global context features of small image patches, calculates the anomaly probability of each area in the image and maps it back to the panoramic image of the garden.

[0013] The anomaly determination module inputs the image data of the area with high anomaly probability into the CNN model, identifies the anomaly type, and infers the cause of the anomaly based on the sensor data.

[0014] The abnormal warning module marks the abnormal area on the panoramic image of the garden and provides warning information on the abnormal type and cause.

[0015] Furthermore, the sensor module includes the following components.

[0016] Soil moisture sensor, used to monitor the moisture content in the soil.

[0017] Temperature sensors to monitor ambient and soil temperatures.

[0018] Light sensors are used to detect light intensity to assess plant photosynthesis conditions.

[0019] Nutrient sensor for detecting nitrogen, phosphorus and potassium content in soil.

[0020] Furthermore, the garden panoramic image is obtained through the following steps.

[0021] The aerial images are geometrically corrected to eliminate the perspective distortion caused by the UAV flight. The formula is: I'=T(I), where I={I1,I2,…,I N} represents the aerial image sequence, T represents the geometric transformation function, I'={I1',I2',…,I N '} indicates that the rectified image sequence is the rectified image, and N indicates the number of aerial images.

[0022] Aerial images taken at different angles and heights are registered to ensure spatial consistency, including: obtaining matching point sets through feature point matching algorithm; calculating the transformation matrix H, expressed as: p'=Hp, where p is the point coordinate in the image, H is the transformation matrix, and p' is the transformed coordinate.

[0023] The corrected images are stitched into a panoramic image of the garden, and the feathering technique is used to process the edges to avoid obvious seams. At the same time, the weighted average method is used to merge the overlapping areas, which can be expressed as: I final (u,v)=∑ r=1 N w r I r '(u,v), where w r is the weight within the splicing area, I final (u, v) represents the pixel value of the stitched garden panoramic image at position (u, v); I r '(u,v) represents the pixel value of the rth corrected image at position (u,v).

[0024] Furthermore, the garden panoramic image is divided into a plurality of small blocks of fixed size, and a feature vector of each small block is extracted through linear transformation and position coding, including the following steps.

[0025] Divide the garden panoramic image into Z large regions R k , expressed as: I final =∪ k=1 Z R k , where I final Represents the stitched panoramic image of the garden.

[0026] The large area R k Further divided into m×n small blocks P i,j , for feature extraction by the ViT model, the size of each small block is set to S×S pixels, and the formula is expressed as: R k =∪ i=1 mk∪ j=1 nk P i,j , where S represents the side length of the small block; mk and nk represent the length of the large area R k The number of tiles to divide into rows and columns.

[0027] For each small block P i,j Apply linear transformation and position encoding to generate a fixed-dimensional feature vector, which is expressed as: F patch (i,j)=Linear(P i,j )+PE(i,j), where F patch (i,j) is the feature vector of the small block; Linear(P i,j ) represents the small block P i,j The applied linear transformation operation converts the image information into a feature vector; PE(i,j) is the corresponding small block position code; (i,j) is the position index of the small block, indicating the position of the small block in the entire large area or image.

[0028] Furthermore, the sensor data are spatially mapped with the image patches, and an interpolation algorithm is used to supplement the sparse areas of the sensor grid to generate a continuous environmental parameter distribution map, including the following steps.

[0029] The sensor data is preprocessed, including data cleaning and filtering, to remove outliers and noise, and the data is labeled according to timestamp, sensor type and location. The preprocessed sensor data is represented as: s ={d s,1 ,d s,2 ,…,d s,J}, where D s is the sensor data set collected before the sth aerial photography, J represents the number of all sensor data collected before the sth aerial photography, d s,q Represents the measurement value of the qth sensor collected before the sth aerial photography.

[0030] The data is smoothed by calculating the average value of two consecutive aerial photography time periods. The formula is: ˉ q =(d s,q +d s+1,q ) / 2, where d ˉ q is the average data value of the qth sensor in the interval between the sth and s+1th aerial photography; d s+1,q Represents the measurement value of the qth sensor collected before the s+1th aerial photography.

[0031] The smoothed sensor data D ˉ ={d ˉ 1,dˉ 2,…,d ˉ J} are assigned to the same grid as the image patch, where: x,y When there is an overlapping relationship, it is expressed as: x,y =d ˉ q if (x, y) falls within the coverage of q; when the sensor grid is sparse, the interpolation method is used to smoothly extend the sensor value to multiple adjacent small blocks to form a transition area, which is expressed as: d ˉ x,y =Interpolation({d ˉ q for neighbors of P x,y}), where d ˉ x,y Represents the environmental parameter value after interpolation, located in the image block P x,y The position of the image patch P; x and y represent the image patch P x,y Coordinates in two-dimensional space.

[0032] The processed sensor data is marked by timestamp, sensor type and location to obtain the final sensor data set D ˉ final , expressed as: D ˉ final ={(d ˉ x,y ,P x,y ) for each patch P x,y}, and then perform subsequent fusion analysis with small image block features.

[0033] Furthermore, the image small block features and the corresponding sensor data feature fusion are realized by direct splicing fusion or dynamic weighted fusion, wherein: in the direct splicing fusion method, the image features and sensor information are integrated into an extended input feature, and the specific steps include: the small block feature extracted by ViT is denoted as F patch (i,j), the corresponding sensor data is recorded as d ˉ x,y , for F patch (i,j) and d ˉ x,y Direct concatenation is performed to form an extended feature vector, the formula is: F combined (i,j)=Concat(F patch (i,j),d ˉ x,y), where Concat means concatenating two feature vectors in the feature dimension. The concatenated feature F combined (i, j) is sent to the subsequent network layer for processing; in the dynamic weighted fusion method, the correlation between the image and the sensor data is integrated into the features through the self-attention mechanism. The specific steps include: patch (i,j) is regarded as Query, sensor data d ˉ x,y As Key and Value, for each image block P i,j , calculate its correlation with all sensor data, the formula is: A i,j =softmax(F patch (i,j)·D ˉ / d k 1 / 2 ), where d k is the dimension of the Key vector, A i,j represents the correlation weight between the image patch and the sensor data; using the calculated A i,j Perform weighted fusion on sensor data to obtain weighted features: F fused (i,j)=A i,j ·D ˉ , the final output comprehensive feature is expressed as: F combined '(i,j)=F patch (i,j)+F fused (i,j), where F fused (i, j) is the feature obtained by weighted fusion of image patches and sensor data through the self-attention mechanism; F combined '(i,j) is the final fusion feature, which combines the influence of small image patch features and sensor data.

[0034] Furthermore, the ViT model is used to learn the global context features of small image patches, calculate the abnormal probability of each area in the image and map it back to the garden panoramic image, including the following steps.

[0035] For each small block P i,j , calculate the self-attention value, the formula is: A i,j '=softmax(Q·K T / d k 1 / 2 )·V, where Q, K, and V are the Query, Key, and Value feature matrices respectively; A i,j ' represents the image block P i,j The attention output of K T represents the transpose of the matrix K.

[0036] Through the fully connected layer, we can classify each small block to determine whether it is abnormal. The formula is: y^ i,j =sigmoid(W fc ·A i,j '+b fc ), where W fc is the weight of the fully connected layer, b fc is the bias term, y^ i,j Represents a small block P i,j The probability of abnormality.

[0037] According to the detection results, the anomaly probability is mapped back to the panoramic image of the garden. The formula is: Anomaly Map=reshape({y^ i,j}), where the reshape operation reassembles the abnormal probability of the small blocks into the same dimension as the original image; Anomaly Map is the final generated abnormal probability map, which reflects the degree of abnormality of each small block in the image.

[0038] Furthermore, the image data of the area with high abnormal probability is input into a convolutional neural network (CNN) model to identify the abnormal type and infer the cause of the abnormality based on the sensor data, including the following steps.

[0039] The anomaly probability map y^ obtained from the anomaly detection module i,j , according to the set threshold τ, the area with abnormal probability higher than the threshold is screened out. The formula is: abnormal area = {(i',j')|y^ i,j >τ}, where (i',j') represents the coordinates of the small patch of the abnormal area.

[0040] Use clustering algorithm to cluster abnormal small blocks and find the area where abnormal points are concentrated. The clustering results generate several abnormal clusters C1, C2, ..., C μ , each cluster contains a set of adjacent abnormal patches, where μ is the number of clusters.

[0041] For each abnormal cluster, calculate the minimum bounding box B, which is expressed as: B = (x min ,y min ,x max ,y max ), where (x min ,y min ) represents the coordinate of the upper left corner of the minimum bounding box, (x max ,y max ) represents the coordinates of the lower right corner of the minimum bounding box.

[0042] Use the bounding box B to crop I to obtain the image part I containing the abnormal aggregation area crop , expressed as: Icrop =I[x min :x max ,y min :y max ], where I[x min :x max ,y min :y max ] means to crop a sub-region through the given coordinate range.

[0043] If I crop When the size does not meet the input requirements of CNN, a scaling and padding operation is performed to scale the cropped image to the fixed size required by CNN; if scaling causes image distortion, zero padding or boundary expansion is used to expand the cropped area to the size required by CNN.

[0044] The cropped and resized image I crop Input into CNN, the formula is: t^ i,j =CNN(I crop ), where t^ i,j It is the abnormal type of CNN output.

[0045] A normal value or a normal range is set for each sensor data, wherein the normal value is obtained based on historical data, agricultural experience or experimental data.

[0046] For the detected abnormal small blocks, extract the corresponding sensor grid G ij ,If a grid contains at least one abnormal small block, the grid is considered to be an abnormal grid. For the sensor data of the abnormal grid, its deviation from the normal value is calculated, and the specific abnormal cause is analyzed.

[0047] Furthermore, the abnormal area is marked on the panoramic image of the garden, and warning information of the abnormal type and cause is given, including the following steps.

[0048] Create an all-zero matrix or background image with the same size as the garden panoramic image as the annotation image.

[0049] Traverse each small block (i, j), mark it according to its abnormal probability, and adjust the transparency or color intensity of the mark according to the size of the abnormal probability to reflect the severity of the abnormality.

[0050] The generated annotation map is superimposed with the panoramic image of the garden to form the final early warning map.

[0051] For areas where the probability of abnormality exceeds the threshold τ, the identified abnormality type is connected with the corresponding small block coordinates of the abnormal area, and the abnormal cause analyzed by the sensor data is connected with the small block coordinates of the abnormal area within the detection range of the corresponding sensor.

[0052] The abnormal small block coordinates, abnormal type and abnormal cause of sensor data are reported to relevant personnel as early warning information.

[0053] Compared with the prior art, the present application has the following beneficial effects.

[0054] This application uses grid-deployed sensors and image acquisition modules to monitor the garden environment in real time, uses data processing and feature fusion technology combined with ViT and CNN models to detect abnormal areas, and generates early warning information, thereby achieving comprehensive monitoring and intelligent early warning of garden plant health. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 This is a structural schematic diagram of a garden landscaping maintenance monitoring and early warning system disclosed in an embodiment of the present application.

[0056] Figure 2 A flow chart of a landscaping maintenance monitoring and early warning method disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical scheme and advantages of the implementation of the present invention clearer, the technical scheme in the embodiment of the present invention will be described in more detail below in conjunction with the drawings in the embodiment of the present invention. In the drawings, the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The described embodiments are part of the embodiments of the present invention, not all of the embodiments.

[0058] Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in the field without making any creative work shall fall within the scope of protection of the present invention.

[0059] The embodiments and directional terms described below with reference to the accompanying drawings are exemplary and intended to be used to explain the present invention, but should not be construed as limiting the present invention.

[0060] like Figure 1 As shown, a landscaping maintenance monitoring and early warning system includes the following modules.

[0061] The sensor module collects garden environment parameters related to plant health in real time through sensors deployed in a grid form.

[0062] The image acquisition module is responsible for regularly collecting high-resolution panoramic images of the garden to obtain a comprehensive picture of the plant growth status.

[0063] The image data processing module divides the garden panoramic image into multiple small blocks of fixed size and extracts the feature vector of each small block through linear transformation and position coding.

[0064] The sensor data processing module spatially maps the sensor data with small image patches and uses an interpolation algorithm to supplement the sparse areas of the sensor grid to generate a continuous environmental parameter distribution map.

[0065] The feature fusion module fuses the image features of each small block with the corresponding sensor data to generate a comprehensive feature vector for each small block.

[0066] The anomaly detection module uses the ViT model to learn the global context features of small image patches, calculates the anomaly probability of each area in the image and maps it back to the panoramic image of the garden.

[0067] The anomaly determination module inputs the image data of the area with high anomaly probability into the CNN model, identifies the anomaly type, and infers the cause of the anomaly based on the sensor data.

[0068] The abnormal warning module marks the abnormal area on the panoramic image of the garden and provides warning information on the abnormal type and cause.

[0069] In summary, the garden greening maintenance monitoring and early warning system integrates multi-source data acquisition, intelligent image processing and deep learning technology, and realizes real-time monitoring of the garden health status and abnormal early warning. Through the sensor network to collect environmental data, drone images to obtain global information, and combined with ViT and CNN models to analyze abnormalities, a comprehensive perception of garden vegetation problems is achieved. The collaborative work of each module ensures the efficiency and accuracy from data acquisition, processing to early warning output, providing systematic support for garden maintenance management. This intelligent monitoring system not only improves the scientificity and accuracy of management, but also provides important guarantees for timely discovery of potential problems and ensuring the long-term health of garden ecology.

[0070] As an optional solution of the present application, the sensor module includes the following components: a soil moisture sensor for monitoring the moisture content in the soil; a temperature sensor for monitoring the ambient temperature and soil temperature; a light sensor for detecting light intensity to evaluate plant photosynthesis conditions; a nutrient sensor for detecting nitrogen, phosphorus, and potassium content in the soil. These sensor components are arranged in a grid in the garden area to ensure uniform coverage. The detection range of each sensor should take into account different plant species and growth conditions. These sensors regularly collect environmental data and store it in a local cache.

[0071] As an optional solution of the present application, the image acquisition module adopts drone image acquisition equipped with a high-definition camera and a stable flight system to ensure stable shooting under various weather conditions. The selected drone should have sufficient flight time and load capacity to meet the coverage requirements of the garden area. According to the area and shape of the garden, the flight route of the drone is planned using map software to ensure that each area is covered. When designing the route, factors such as flight altitude, speed and shooting interval also need to be considered. Set up regular flight missions and shoot under suitable weather conditions to obtain clear, high-resolution garden images. The high-definition camera requires a resolution of 4K or higher to ensure that the image details are fully captured. During the flight, the captured images will be transmitted in real time and preliminarily analyzed or stored. Finally, the captured images are stored on a cloud platform or local server.

[0072] As an optional solution of the present application, obtaining the panoramic image of the garden includes the following steps.

[0073] The aerial images are geometrically corrected to eliminate the perspective distortion caused by the UAV flight. The formula is: I'=T(I), where I={I1,I2,…,I N} represents the aerial image sequence, T represents the geometric transformation function, I'={I1',I2',…,I N '} indicates that the rectified image sequence is the rectified image, and N indicates the number of aerial images.

[0074] Aerial images taken at different angles and heights are registered to ensure spatial consistency, specifically including: first, obtaining a set of matching points through a feature point matching algorithm; then calculating the transformation matrix H, expressed as: p'=Hp, where p is the point coordinate in the image, H is the transformation matrix, and p' is the transformed coordinate.

[0075] The corrected images are stitched into a panoramic image of the garden, and the feathering technique is used to process the edges to avoid obvious seams. At the same time, the weighted average method is used to merge the overlapping areas, which can be expressed as: I final (u,v)=∑ r=1 N w r I r '(u,v), where w r is the weight within the splicing area, I final (u, v) represents the pixel value of the stitched garden panoramic image at position (u, v); I r '(u,v) represents the pixel value of the rth corrected image at position (u,v).

[0076] As an optional solution of the present application, the garden panoramic image is divided into a plurality of small blocks of fixed size, and a feature vector of each small block is extracted through linear transformation and position coding, including the following steps.

[0077] In order to avoid the memory burden caused by processing too large images, the garden panoramic image is divided into Z large areas R k , each area is regarded as a processing unit, the number of areas Z and the division method can be selected according to the garden characteristics or vegetation types to ensure that the division is reasonable, which can be expressed as: I final =∪ k=1 Z R k , where I final represents the spliced ​​panoramic image of the garden. In the subsequent anomaly detection, the region R k Independent processing to reduce computational complexity.

[0078] For each large region R k Or a smaller panoramic image of the garden, further divided into m×n small blocks P i,j , for feature extraction by the ViT (Vision Transformer) model, the size of each small block is set to S×S pixels, and the formula is expressed as: R k =∪ i=1 mk ∪ j=1 nk P i,j or I final =∪ i=1 m ∪ j=1 n P i,j , where m represents the number of rows of the small block, n represents the number of columns of the small block, S represents the side length of the small block; mk and nk represent the large area R k The number of tiles to divide into rows and columns.

[0079] For each small block P i,j Apply linear transformation and position encoding to generate a fixed-dimensional feature vector, which is expressed as: F patch (i,j)=Linear(P i,j )+PE(i,j), where F patch (i,j) is the feature vector of the small block; Linear(P i,j ) represents the small block P i,j The applied linear transformation operation converts the image information into a feature vector; PE(i,j) is the corresponding small block position code; (i,j) is the position index of the small block, indicating the position of the small block in the entire large area or image.

[0080] As an optional solution of the present application, the sensor data is spatially mapped with the image blocks, and an interpolation algorithm is used to supplement the sparse areas of the sensor grid to generate a continuous environmental parameter distribution map, including the following steps.

[0081] The sensor data is preprocessed, including data cleaning and filtering, to remove outliers and noise, and the data is labeled according to timestamp, sensor type and location. The preprocessed sensor data is represented as: s ={d s,1 ,d s,2 ,…,d s,J}, where D s is the sensor data set collected before the sth aerial photography, J represents the number of all sensor data collected before the sth aerial photography, d s,q Represents the measurement value of the qth sensor collected before the sth aerial photography.

[0082] The data is smoothed by calculating the average value of two consecutive aerial photography time periods. The formula is: ˉ q =(d s,q +d s+1,q ) / 2, where d ˉ q is the average data value of the qth sensor in the interval between the sth and s+1th aerial photography; d s+1,q Represents the measurement value of the qth sensor collected before the s+1th aerial photography.

[0083] The smoothed sensor data D ˉ ={d ˉ 1,d ˉ 2,…,d ˉ J} is assigned to the same grid as the image patch, such as the area covered by sensor q and the image patch P x,y If there is an overlapping relationship, it is expressed as: x,y =d ˉ q if (x, y) falls within the coverage of q; if the sensor grid is sparse, the sensor value is smoothly extended to multiple adjacent small blocks using interpolation to form a transition region, which is expressed as: ˉ x,y =Interpolation({d ˉ q for neighbors of P x,y}), where d ˉ x,y Represents the environmental parameter value after interpolation, located in the image block Px,y The position of the image patch P; x and y represent the image patch P x,y Coordinates in two-dimensional space.

[0084] The processed sensor data is labeled by timestamp, sensor type and location to obtain the final sensor data set D ˉ final , specifically expressed as: D ˉ final ={(d ˉ x,y ,P x,y ) for each patch P x,y}, and then perform subsequent fusion analysis with small image block features.

[0085] As an optional solution of the present application, the image small block features and the corresponding sensor data feature fusion are realized by direct splicing fusion or dynamic weighted fusion. In the direct splicing fusion method, the image features and sensor information are integrated into an extended input feature. Specifically, the small block feature extracted by ViT is denoted as F patch (i,j), the corresponding sensor data is recorded as d ˉ x,y , for F patch (i,j) and d ˉ x,y Direct concatenation is performed to form an extended feature vector, the formula is: F combined (i,j)=Concat(F patch (i,j),d ˉ x,y ), where Concat means concatenating two feature vectors in the feature dimension. The concatenated feature F combined (i, j) is sent to the subsequent network layer for processing. In the dynamic weighted fusion method, the correlation between the image and the sensor data is integrated into the features through the self-attention mechanism. Specifically, the small block feature F patch (i,j) is regarded as Query, sensor data d ˉ x,y As Key and Value, for each image block P i,j , calculate its correlation with all sensor data, the formula is: A i,j =softmax(F patch (i,j)·D ˉ / d k 1 / 2 ), where d k is the dimension of the Key vector, A i,j represents the correlation weight between the image patch and the sensor data; using the calculated Ai,j Perform weighted fusion on sensor data to obtain weighted features: F fused (i,j)=A i,j ·D ˉ , the final output comprehensive feature is expressed as: F combined '(i,j)=F patch (i,j)+F fused (i,j), where F fused (i, j) is the feature obtained by weighted fusion of image patches and sensor data through the self-attention mechanism; F combined '(i,j) is the final fusion feature, which combines the influence of small image patch features and sensor data.

[0086] As an optional solution of the present application, the ViT model is used to learn the global context features of small image blocks, calculate the abnormal probability of each area in the image and map it back to the garden panoramic image, including the following steps.

[0087] For each small block P i,j , calculate the self-attention value, the formula is: A i,j '=softmax(Q·K T / d k 1 / 2 )·V, where Q, K, and V are the Query, Key, and Value feature matrices respectively; A i,j ' represents the image block P i,j The attention output of K T represents the transpose of the matrix K.

[0088] Through the fully connected layer, we can classify each small block to determine whether it is abnormal. The formula is: y^ i,j =sigmoid(W fc ·A i,j '+b fc ), where W fc is the weight of the fully connected layer, b fc is the bias term, y^ i,j Represents a small block P i,j The probability of abnormality.

[0089] According to the detection results, the anomaly probability is mapped back to the panoramic image of the garden. The formula is: Anomaly Map=reshape({y^ i,j}), where the reshape operation reassembles the abnormal probability of the small blocks into the same dimension as the original image; Anomaly Map is the final generated abnormal probability map, which reflects the degree of abnormality of each small block in the image.

[0090] As an optional solution of the present application, the image data of the area with high abnormal probability is input into a convolutional neural network (CNN) model to identify the abnormality type and infer the cause of the abnormality based on the sensor data, including the following steps.

[0091] The anomaly probability map y^ obtained from the anomaly detection module i,j , according to the set threshold τ, the area with abnormal probability higher than the threshold is screened out. The formula is: abnormal area = {(i',j')|y^ i,j >τ}, where (i',j') represents the coordinates of the small patch of the abnormal area.

[0092] Use clustering algorithms (such as K-means) to cluster abnormal small blocks and find the areas where abnormal points are concentrated. The clustering results generate several abnormal clusters C1, C2, ..., C μ , each cluster contains a set of adjacent abnormal patches, where μ is the number of clusters.

[0093] For each abnormal cluster, calculate the minimum bounding box, the range of which is determined by the coordinates of the small blocks in the cluster, such as the range of the abnormal points in the cluster is (x min ,y min ) to (x max ,y max ), then the bounding box B is expressed as: B=(x min ,y min ,x max ,y max ), where (x min ,y min ) represents the coordinate of the upper left corner of the minimum bounding box, (x max ,y max ) represents the coordinates of the lower right corner of the minimum bounding box.

[0094] Use the bounding box B to crop I to obtain the image part I containing the abnormal aggregation area crop , expressed as: I crop =I[x min :x max ,y min :y max ], where I[x min :x max ,y min :y max ] means to crop a sub-region through the given coordinate range.

[0095] If I cropIf the size does not meet the input requirements of CNN, a scaling and padding operation is performed to scale the cropped image to the fixed size required by CNN; if scaling causes image distortion, zero padding or boundary expansion is used to expand the cropped area to the size required by CNN.

[0096] The cropped and resized image I crop Input into CNN, the formula is: t^ i,j =CNN(I crop ), where t^ i,j is the type of anomaly output by the CNN, such as disease type, nutrient deficiency, pest type, and physical damage. The CNN model should be trained to perform classification based on the input image data.

[0097] A normal value or a normal range is set for each sensor data, such as humidity, temperature, light and nutrients, wherein the normal value can be obtained based on historical data, agricultural experience or experimental data.

[0098] For the detected abnormal small blocks, extract the corresponding sensor grid G ij ,If a grid contains at least one abnormal small block, the grid is considered to be an abnormal grid. For the sensor data of the abnormal grid, its deviation from the normal value is calculated, and the specific abnormal cause is analyzed.

[0099] As an optional solution of the present application, the abnormal area is marked on the panoramic image of the garden, and warning information of the abnormal type and cause is given, including the following steps.

[0100] Create an all-zero matrix or background image with the same size as the garden panoramic image as the annotation image.

[0101] Traverse each small block (i, j), mark it according to its abnormal probability, and adjust the transparency or color intensity of the mark according to the size of the abnormal probability to reflect the severity of the abnormality.

[0102] The generated annotation map is superimposed with the panoramic image of the garden to form the final early warning map.

[0103] For areas where the probability of abnormality exceeds the threshold τ, the identified abnormality type is connected with the corresponding small block coordinates of the abnormal area, and the abnormal cause analyzed by the sensor data is connected with the small block coordinates of the abnormal area within the detection range of the corresponding sensor.

[0104] The abnormal small block coordinates, abnormal type and abnormal cause of sensor data are reported to relevant personnel as early warning information.

[0105] like Figure 2 As shown, a landscaping maintenance monitoring and early warning method includes the following steps.

[0106] Step S1, real-time collection of garden environment parameters, including soil moisture, light intensity, ambient temperature and soil nutrient content.

[0107] Step S2, deploying drones to perform scheduled aerial photography tasks to obtain panoramic images of the garden to ensure complete coverage of all vegetation in the monitoring area.

[0108] Step S3, dividing the garden panoramic image into multiple small blocks, performing linear transformation on each image block and embedding position coding information to generate a feature vector of fixed dimension.

[0109] Step S4, mapping the data collected by the sensor to the grid corresponding to the image block, and using the interpolation method to smoothly extend the sensor values ​​to multiple adjacent blocks for the sparse sensor coverage area, generating a data smooth transition area to ensure the spatial continuity of the environmental data.

[0110] Step S5, deeply fuse the image feature vector of each small block with the corresponding sensor data to construct a comprehensive feature vector containing complete environmental information.

[0111] Step S6, input the comprehensive feature vector into the ViT model, analyze the relationship between image blocks through the multi-head self-attention mechanism, calculate the abnormal probability distribution of each area and map the abnormal probability to the panoramic image of the garden.

[0112] Step S7, input the image data of the high abnormal probability area into the CNN model to identify the specific abnormality type and infer the cause of the abnormality based on the sensor data.

[0113] Step S8, marking the regional abnormality probability results onto the panoramic image of the garden to intuitively display the health status of each area in the garden, and output the abnormality type and cause of the abnormal area.

[0114] In this embodiment, step S1 collects soil moisture, light intensity, ambient temperature and soil nutrient content in real time by deploying multiple sensors, thereby establishing a dynamic monitoring basis for the garden microenvironment. The multiple environmental parameters obtained in real time provide data guarantee for subsequent comprehensive analysis. These data can help analyze the soil moisture distribution, light distribution and temperature changes in the garden, thereby evaluating the growth environment and health status of the plant. At the same time, the monitoring of soil nutrient content can reveal the nutritional status of the plant and help predict the growth potential and disease resistance of the plant. This data collection step realizes real-time and continuous data collection by rationally configuring the sensor network, providing a high-quality data basis for garden management.

[0115] In this embodiment, step S2 provides comprehensive visual data for subsequent image segmentation and feature extraction. UAV aerial photography is flexible and has comprehensive coverage. It is suitable for a wide range of garden areas and can solve the limitation that ground monitoring is difficult to fully cover. The high-definition images obtained by the timed aerial photography of the drone can provide a macroscopic grasp of the overall situation of the garden, making it easier to identify the spatial distribution, density and growth status of vegetation. Real-time acquisition of panoramic images helps to promptly detect signs of vegetation diseases or decline, providing intuitive reference for managers.

[0116] In this embodiment, the innovation of step S3 is to realize the standardized representation of the image block, so that the subsequent analysis can focus on the spatial features of each image block. The embedding of linear transformation and position coding further improves the expression ability of image information in the model, ensuring that the model can retain spatial information when processing images, making it easier for the model to identify subtle changes in specific areas. This image block division and coding method makes subsequent feature extraction more accurate and ensures the accuracy and stability of image analysis results.

[0117] In this embodiment, step S4 ensures the spatial continuity of environmental parameters and avoids the situation where data is broken or blank in sparse areas. Through interpolation and smoothing, the deficiencies of sensor data can be effectively filled, making the distribution of environmental parameters in the image more coherent, which is very important for the construction and subsequent analysis of comprehensive feature vectors. The smooth transition processing of environmental data improves the integrity of the data and ensures the comprehensiveness and accuracy of anomaly detection.

[0118] In this embodiment, step S5 enables the image information and environmental data to complement each other, thereby forming a multi-dimensional understanding of each small image block. Through deep fusion, it is possible to fully utilize information from different data sources, improve the depth of understanding of the health status of plants, and facilitate subsequent model analysis. The generation of comprehensive feature vectors makes the relationship between environmental data and image features closer, which helps to improve the accuracy of anomaly detection.

[0119] In this embodiment, step S6 inputs the comprehensive feature vector into the ViT (Vision Transformer) model, uses the multi-head self-attention mechanism to analyze the relationship between image blocks, and calculates the abnormal probability distribution of each region. The multi-head self-attention mechanism of the ViT model can capture the global information between image blocks, so that the model can better identify the spatial correlation of the health status of vegetation. By analyzing the relationship between regions, the ViT model can identify potential problems in the plant growth environment and calculate the abnormal probability. This mechanism provides an accurate and high-resolution probability distribution map for anomaly detection, thereby improving the detection ability of abnormal areas.

[0120] In this embodiment, step S7 inputs the image data into a CNN (convolutional neural network) model to identify the specific abnormality type and infer the cause of the abnormality in combination with the sensor data. The CNN model performs well in image recognition and can accurately classify abnormal conditions of plants, such as pests and diseases, insufficient water or insufficient light. Combined with the sensor data, the cause of the abnormality can be further analyzed to help managers make accurate diagnostic decisions. By identifying the abnormality type and analyzing the cause, scientific decision support can be provided, making it easier for managers to take targeted intervention measures.

[0121] In this embodiment, step S8 marks the regional abnormal probability and abnormal type on the panoramic image of the garden, so that the health status of the garden is clear at a glance, helping managers to quickly identify and locate abnormal areas. Through intuitive image display, the health status and abnormal conditions of each area can be quickly understood. This image display method effectively improves the visualization of monitoring results, allowing managers to more intuitively grasp the overall health status of the garden. This intuitive display is not only convenient for daily management, but also can provide clearer guidance for subsequent decision-making.

[0122] In summary, the garden greening maintenance monitoring and early warning method effectively improves the intelligent level of garden monitoring through real-time collection of environmental data, block processing and fusion of drone aerial images, intelligent analysis and anomaly recognition of deep models. The design of each step is based on multimodal fusion of data and image recognition algorithm of deep learning, which ensures the accuracy, comprehensiveness and efficiency of monitoring and early warning. Through this method, garden managers can understand the health status of vegetation more accurately, take timely measures to effectively prevent possible vegetation problems, and ensure the long-term health of garden greening.

[0123] Finally, it should be pointed out that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them. Although the present invention has been described in detail with reference to the above embodiments, a person skilled in the art should understand that the technical solutions described in the above embodiments can still be modified, or some of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A landscaping maintenance monitoring and early warning system, characterized in that: It includes the following modules: The sensor module collects garden environment parameters related to plant health in real time through sensors deployed in a grid format; The image acquisition module is responsible for regularly collecting high-resolution panoramic images of the garden to obtain a full picture of the plant growth status; The image data processing module divides the garden panoramic image into multiple small blocks of fixed size, and extracts the feature vector of each small block through linear transformation and position coding; The sensor data processing module spatially maps the sensor data with small image patches and uses an interpolation algorithm to supplement the sparse areas of the sensor grid to generate a continuous environmental parameter distribution map; The feature fusion module fuses the image features of each small block with the corresponding sensor data to generate a comprehensive feature vector for each small block; The anomaly detection module uses the ViT model to learn the global context features of small image patches, calculates the anomaly probability of each area in the image and maps it back to the panoramic image of the garden; The anomaly determination module inputs the image data of the area with high anomaly probability into the CNN model, identifies the anomaly type, and infers the cause of the anomaly based on the sensor data; The abnormal warning module marks the abnormal area on the panoramic image of the garden and provides warning information on the abnormal type and cause.

2. A garden greening maintenance monitoring and early warning system according to claim 1, characterized in that: The sensor module includes the following components: Soil moisture sensors, which monitor the moisture content in the soil; Temperature sensors to monitor ambient and soil temperatures; Light sensors, used to detect light intensity to assess plant photosynthesis conditions; Nutrient sensor for detecting nitrogen, phosphorus and potassium content in soil.

3. A garden greening maintenance monitoring and early warning system according to claim 1, characterized in that: Obtain a panoramic image of the garden by following these steps: The aerial images are geometrically corrected to eliminate the perspective distortion caused by the UAV flight. The formula is: I'=T(I), where I={I1,I2,…,I N } represents the aerial image sequence, T represents the geometric transformation function, I'={I1',I2',…,I N '} indicates that the corrected image sequence is the corrected image, and N indicates the number of aerial images; Aerial images taken at different angles and heights are registered to ensure spatial consistency, including: obtaining matching point sets through feature point matching algorithm; calculating the transformation matrix H, expressed as: p'=Hp, where p is the point coordinate in the image, H is the transformation matrix, and p' is the transformed coordinate; The corrected images are stitched into a panoramic image of the garden, and the feathering technique is used to process the edges to avoid obvious seams. At the same time, the weighted average method is used to merge the overlapping areas, which can be expressed as: I final (u,v)=∑ r=1 N w r I r '(u,v), where w r is the weight within the splicing area, I final (u, v) represents the pixel value of the stitched garden panoramic image at position (u, v); I r '(u,v) represents the pixel value of the rth corrected image at position (u,v).

4. A garden greening maintenance monitoring and early warning system according to claim 3, characterized in that: The garden panoramic image is divided into multiple small blocks of fixed size, and the feature vector of each small block is extracted through linear transformation and position encoding, including the following steps: Divide the garden panoramic image into Z large regions R k , expressed as: I final =∪ k=1 Z R k , where I final represents the stitched panoramic image of the garden; The large area R k Further divided into m×n small blocks P i,j , for feature extraction by the ViT model, the size of each small block is set to S×S pixels, and the formula is expressed as: R k =∪ i=1 mk ∪ j=1 nk P i,j , where S represents the side length of the small block; mk and nk represent the length of the large area R k The number of blocks divided in row and column directions; For each small block P i,j Apply linear transformation and position encoding to generate a fixed-dimensional feature vector, which is expressed as: F patch (i,j)=Linear(P i,j )+PE(i,j), where F patch (i,j) is the feature vector of the small block; Linear(P i,j ) represents the small block P i,j The applied linear transformation operation converts the image information into a feature vector; PE(i,j) is the corresponding small block position code; (i,j) is the position index of the small block, indicating the position of the small block in the entire large area or image.

5. A garden greening maintenance monitoring and early warning system according to claim 1, characterized in that: The sensor data is spatially mapped with the image patches, and the interpolation algorithm is used to supplement the sparse areas of the sensor grid to generate a continuous environmental parameter distribution map, including the following steps: The sensor data is preprocessed, including data cleaning and filtering, to remove outliers and noise, and the data is labeled according to timestamp, sensor type and location. The preprocessed sensor data is represented as: s ={d s,1 ,d s,2 ,…,d s,J }, where D s is the sensor data set collected before the sth aerial photography, J represents the number of all sensor data collected before the sth aerial photography, d s,q represents the measurement value of the qth sensor collected before the sth aerial photography; The data is smoothed by calculating the average value of two consecutive aerial photography time periods. The formula is: ˉ q =(d s,q +d s+1,q ) / 2, where d ˉ q is the average data value of the qth sensor in the interval between the sth and s+1th aerial photography; d s+1,q represents the measurement value of the qth sensor collected before the s+1th aerial photography; The smoothed sensor data D ˉ ={d ˉ 1,d ˉ 2,…,d ˉ J } are assigned to the same grid as the image patch, where: x,y When there is an overlapping relationship, it is expressed as: x,y =d ˉ q if (x, y) falls within the coverage of q; when the sensor grid is sparse, the interpolation method is used to smoothly extend the sensor value to multiple adjacent small blocks to form a transition area, which is expressed as: ˉ x,y =Interpolation({d ˉ q for neighbors of P x,y }), where d ˉ x,y Represents the environmental parameter value after interpolation, located in the image block P x,y The position of the image patch P; x and y represent the image patch P x,y Coordinates in two-dimensional space; The processed sensor data is marked by timestamp, sensor type and location to obtain the final sensor data set D ˉ final , expressed as: D ˉ final ={(d ˉ x,y ,P x,y ) for each patch P x,y }, and then perform subsequent fusion analysis with small image block features.

6. A garden greening maintenance monitoring and early warning system according to claim 1, characterized in that: The image small block features and the corresponding sensor data feature fusion are realized by direct splicing fusion or dynamic weighted fusion, where: In the direct splicing fusion method, the image features and sensor information are integrated into an extended input feature. The specific steps include: The small block feature extracted by ViT is denoted as F patch (i,j), the corresponding sensor data is recorded as d ˉ x,y , for F patch (i,j) and d ˉ x,y Direct concatenation is performed to form an extended feature vector, the formula is: F combined (i,j)=Concat(F patch (i,j),d ˉ x,y ), where Concat means concatenating two feature vectors in the feature dimension. The concatenated feature F combined (i, j) is sent to the subsequent network layer for processing; in the dynamic weighted fusion method, the correlation between the image and the sensor data is integrated into the features through the self-attention mechanism. The specific steps include: patch (i,j) is regarded as Query, sensor data d ˉ x,y As Key and Value, for each image block P i,j , calculate its correlation with all sensor data, the formula is: A i,j =softmax(F patch (i,j)·D ˉ / d k 1 / 2 ), where d k is the dimension of the Key vector, A i,j represents the correlation weight between the image patch and the sensor data; using the calculated A i,j Perform weighted fusion on sensor data to obtain weighted features: F fused (i,j)=A i,j ·D ˉ , the final output comprehensive feature is expressed as: F combined '(i,j)=F patch (i,j)+F fused (i,j), where F fused (i, j) is the feature obtained by weighted fusion of image patches and sensor data through the self-attention mechanism; F combined '(i,j) is the final fusion feature, which combines the influence of small image patch features and sensor data.

7. A garden greening maintenance monitoring and early warning system according to claim 1, characterized in that: The ViT model is used to learn the global context features of small image patches, calculate the abnormal probability of each area in the image and map it back to the panoramic image of the garden, including the following steps: For each small block P i,j , calculate the self-attention value, the formula is: A i,j '=softmax(Q·K T / d k 1 / 2 )·V, where Q, K, and V are the Query, Key, and Value feature matrices respectively; A i,j ' represents the image block P i,j The attention output of K T represents the transpose of the matrix K; Through the fully connected layer, we can classify each small block to determine whether it is abnormal. The formula is: y^ i,j =sigmoid(W fc ·A i,j '+b fc ), where W fc is the weight of the fully connected layer, b fc is the bias term, y^ i,j Represents a small block P i,j The probability of abnormality; According to the detection results, the anomaly probability is mapped back to the panoramic image of the garden. The formula is: Anomaly Map=reshape({y^ i,j }), where the reshape operation reassembles the abnormal probability of the small blocks into the same dimension as the original image; AnomalyMap is the final generated abnormal probability map, which reflects the degree of abnormality of each small block in the image.

8. The garden greening maintenance monitoring and early warning system according to claim 1 is characterized in that: The image data of the area with high abnormal probability is input into the convolutional neural network (CNN) model to identify the abnormal type and infer the cause of the abnormality based on the sensor data, including the following steps: The anomaly probability map y^ obtained from the anomaly detection module i,j , according to the set threshold τ, the area with abnormal probability higher than the threshold is screened out. The formula is: abnormal area = {(i',j')|y^ i,j >τ}, where (i', j') represents the coordinates of the small patch of the abnormal area; Use clustering algorithm to cluster abnormal small blocks and find the area where abnormal points are concentrated. The clustering results generate several abnormal clusters C1, C2, ..., C μ , each cluster contains a set of adjacent abnormal patches, where,μ,is the number of clusters; For each abnormal cluster, calculate the minimum bounding box B, which is expressed as: B = (x min ,y min ,x max ,y max ), where (x min ,y min ) represents the coordinate of the upper left corner of the minimum bounding box, (x max ,y max ) represents the coordinates of the lower right corner of the minimum bounding box; Use the bounding box B to crop I to obtain the image part I containing the abnormal aggregation area crop , expressed as: I crop =I[x min :x max ,y min :y max ], where I[x min :x max ,y min :y max ] means cutting out a sub-region through the given coordinate range; If I crop When the size does not meet the input requirements of CNN, a scaling and padding operation is performed to scale the cropped image to the fixed size required by CNN; if scaling causes image distortion, zero padding or boundary expansion is used to expand the cropped area to the size required by CNN; The cropped and resized image I crop Input into CNN, the formula is: t^ i,j =CNN(I crop ), where t^ i,j is the abnormal type of CNN output; A normal value or a normal range is set for each sensor data, wherein the normal value is obtained based on historical data, agricultural experience or experimental data; For the detected abnormal small blocks, extract the corresponding sensor grid G ij ,If a grid contains at least one abnormal small block, the grid is considered to be an abnormal grid. For the sensor data of the abnormal grid, its deviation from the normal value is calculated, and the specific abnormal cause is analyzed.

9. The garden greening maintenance monitoring and early warning system according to claim 1 is characterized in that: The abnormal area is marked on the panoramic image of the garden, and warning information of the abnormal type and cause is given, including the following steps: Create an all-zero matrix or background image with the same size as the garden panoramic image as a labeling image; Traverse each small block (i, j), mark it according to its abnormal probability, and adjust the transparency or color intensity of the mark according to the size of the abnormal probability to reflect the severity of the abnormality; The generated annotation map is superimposed with the panoramic image of the garden to form the final warning map; For areas where the probability of abnormality exceeds the threshold τ, the identified abnormality type is connected with the corresponding small block coordinates of the abnormal area, and the abnormal cause analyzed by the sensor data is connected with the small block coordinates of the abnormal area within the detection range of the corresponding sensor; The abnormal small block coordinates, abnormal type and abnormal cause of sensor data are reported to relevant personnel as early warning information.

10. A method for monitoring and early warning of garden greening maintenance, implemented based on a garden greening maintenance monitoring and early warning system according to any one of claims 1 to 9, characterized in that: It includes the following steps: Step S1, real-time collection of garden environment parameters, including soil moisture, light intensity, ambient temperature and soil nutrient content; Step S2, deploying drones to perform scheduled aerial photography tasks to obtain panoramic images of the garden to ensure complete coverage of all vegetation in the monitoring area; Step S3, dividing the garden panoramic image into a plurality of small blocks, performing a linear transformation on each image block and embedding position coding information to generate a feature vector of a fixed dimension; Step S4, mapping the data collected by the sensor to the grid corresponding to the image block, and using the interpolation method to smoothly extend the sensor value to multiple adjacent blocks for the sparse sensor coverage area, generating a data smooth transition area, and ensuring the spatial continuity of the environmental data; Step S5, deeply fuse the image feature vector of each small block with the corresponding sensor data to construct a comprehensive feature vector containing complete environmental information; Step S6, input the comprehensive feature vector into the ViT model, analyze the relationship between image blocks through the multi-head self-attention mechanism, calculate the abnormal probability distribution of each area and map the abnormal probability to the panoramic image of the garden; Step S7, inputting the image data of the high abnormal probability area into the CNN model to identify the specific abnormal type and infer the cause of the abnormality based on the sensor data; Step S8, marking the regional abnormality probability results onto the panoramic image of the garden to intuitively display the health status of each area in the garden, and output the abnormality type and cause of the abnormal area.

Images