A municipal greening maintenance calculation and analysis method and system based on image recognition
Patent Information
- Application Number
- CN202610971167.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-01
- Publication Date
- 2026-09-25
AI Technical Summary
[0003]然而现有的基于图像识别的市政绿化养护系统，还存在需要海量人工标注数据，成本较高，且海量的人工标注容易出现错误，从而导致错误标注严重影响养护系统中算法模型的精度等问题
[0053]与现有技术相比，本发明提供的一种基于图像识别的市政绿化养护计算分析方法及系统，通过对先对市政图像进行聚类，再关联标签，大大降低了图像对人工标注的依赖，且聚类过程保留了簇的特征信息，使得模型的输出具备追溯性，可解释性更强。
Smart Images
Figure CN122821218A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of municipal greening maintenance technology, specifically to a calculation and analysis method and system for municipal greening maintenance based on image recognition. Background Technology
[0002] Municipal green space maintenance refers to the continuous and professional management, maintenance, and cultivation of green vegetation in urban public spaces (such as roadside green spaces, parks, squares, street gardens, and riverside green spaces). Its fundamental goal is to maintain and enhance the ecological, landscape, and recreational value of urban green spaces. With the development of image recognition technology, municipal green space maintenance is shifting from a traditional model relying on human experience to one based on intelligent perception, analysis, decision-making, and execution using image recognition, thereby improving maintenance efficiency.
[0003] However, existing municipal greening maintenance systems based on image recognition still have problems such as requiring massive amounts of manually labeled data, high costs, and the large amount of manual labeling being prone to errors, which seriously affect the accuracy of the algorithm model in the maintenance system. Summary of the Invention
[0004] The purpose of this invention is to provide a calculation and analysis method and system for municipal greening maintenance based on image recognition, so as to solve the above-mentioned shortcomings in the prior art.
[0005] To achieve the above objectives, the present invention provides the following technical solution: a calculation and analysis method for municipal greening maintenance based on image recognition, comprising the following steps:
[0006] S1. Acquire municipal images, preprocess and segment the municipal images to obtain multiple municipal image blocks, and extract the first feature for each municipal image block to generate the first municipal image block vector;
[0007] Preprocessing includes geometric correction, radiometric correction, and multi-source registration. Municipal images can be acquired using drones equipped with multispectral sensors (RGB+NIR+RedEdge), street view acquisition vehicles (binocular stereo camera + IMU), and IoT fixed-point cameras, ensuring a spatial resolution better than 5cm / pixel.
[0008] In the first feature extraction, the extracted features include color features, texture features, and structural features. For color features, image patches can be converted to HSV and CIELAB spaces, and the mean and standard deviation of hue and saturation, as well as the green-red channel difference histogram (e.g., 16 bins), can be calculated according to the set superpixels (e.g., 32×32). Texture features can be obtained by calculating the contrast, entropy, and homogeneity of the gray-level co-occurrence matrix (GLCM), and the statistical histogram of local binary mode (LBP). Structural features can be extracted using a lightweight convolutional network (e.g., the first 7 layers of EfficientNet-B0, pre-trained on ImageNet), and further, the dimensionality can be compressed using principal component analysis. Then, these features are concatenated and normalized using the L2 norm to obtain the first municipal image patch vector.
[0009] S2. Collect all first municipal image block vectors and perform first cluster analysis to generate multiple greening species vector clusters;
[0010] S3. Associate each greening type vector cluster with the corresponding greening type label to generate a vector cluster type label lookup table;
[0011] S4. Take the first municipal image block vector as input, and the probability corresponding to the greening type label associated with each greening type vector cluster as output. Make the sum of squared distances between each first municipal image block vector in the same greening type vector cluster and the first centroid of the corresponding cluster the smallest. Set the first centroid distance between the corresponding clusters of different greening type vector clusters to be greater than the first inter-cluster threshold as the first loss function, and train the first model.
[0012] S5. Perform step S1 on the newly acquired municipal images to obtain a new first municipal image block vector. Input the new first municipal image block vector into the trained first model and output the probability corresponding to each greening type label to generate greening type probability data.
[0013] S6. Select the highest probability and greening type label from the first greening type probability data to obtain the corresponding image block greening type data.
[0014] Furthermore, S1-S6 also includes the following steps:
[0015] S1. Perform second feature extraction on each municipal image block to generate a second municipal image block vector;
[0016] In the second feature extraction, the extracted features include spectral information features, local anomaly features, and detailed textures. For spectral information features, these can be obtained by calculating the normalized vegetation index, normalized difference red edge index, and visible light atmospheric impedance index. Local anomaly features can be obtained by using Difference of Gaussian (DoG) bandpass filtering to highlight necrotic spots, insect holes, and discoloration, calculating the mean and extreme values of the filter response, or extracting edge density (such as the Canny edge ratio). Detailed textures can be obtained by extracting the gradient direction histogram to highlight local texture distortions such as leaf defects and lesions. Similarly, these features are concatenated and normalized using the L2 norm to obtain the second municipal image block vector.
[0017] S2. Perform a second clustering analysis on the second municipal image block vector corresponding to the image block within each greening species vector cluster to obtain multiple species defect vector clusters; the second clustering analysis can use algorithms such as hierarchical agglomerative clustering (HAC) and HDBSCAN clustering.
[0018] S3. Associate each type of defect vector cluster with the corresponding type of defect label to generate a vector cluster type defect label lookup table; when associating clusters with labels, appropriate labels can be selected and associated with corresponding vector clusters based on the experience of experts in relevant fields.
[0019] S4. Using the second municipal image block vector as input and the probability corresponding to the type defect label associated with each type of defect vector cluster as output, the second model is trained by minimizing the sum of squared distances between each second municipal image block vector within the same type of defect vector cluster and the second centroid of the corresponding cluster, and setting a second cluster threshold as the second loss function to ensure that the distance between the second centroids of corresponding clusters of different types of defect vector clusters is greater than the second cluster threshold. The first / second model can be selected as needed, such as MLP, XGBoost, LightGBM, etc.
[0020] S5. Perform step S1 on the newly acquired municipal images to obtain a new second municipal image block vector. Input the new second municipal image block vector into the trained second model and output the probability corresponding to various types of defect labels to generate type defect probability data.
[0021] S6. Select the highest probability and the type defect label from the type defect probability data to obtain the corresponding image block type defect data.
[0022] Furthermore, the segmentation of the municipal image in S1 includes the following steps:
[0023] A sliding window is used to set a sliding step size to slide on the municipal image. The sliding window extracts the internal image at each position and generates the corresponding municipal image block. The side length of the sliding window is greater than the sliding step size, so that adjacent municipal image blocks have partial overlap (such as 10%-60% overlap), thereby avoiding the critical target being cut off by the boundary.
[0024] Furthermore, S2 includes the following steps:
[0025] S2.1a. Based on the municipal greening GIS database, for each type of greening species (such as camphor, ginkgo, lawn, shrub, etc.), the first municipal image block vector with the number of seeds of each species is selected for labeling.
[0026] S2.2 Analyze the clustering effectiveness index (profile coefficient and Davies-Bouldin index) of all first municipal image block vectors as a function of the number of first clusters K, and determine the initial K value. K is greater than or equal to the number of categories with greening type labels. For example, take the minimum K value when the profile coefficient reaches a peak of 85%-95% as the initial K, and then fine-tune it according to the prior number of categories.
[0027] S2.3 For each type of greening label, calculate the mean of all labeled first municipal image block vectors as the first centroid of the greening type vector cluster corresponding to the greening type label. Then, randomly select zero to multiple first municipal image block vectors from the unlabeled first municipal image block vectors as the first centroids, so that the number of first centroids reaches K, that is, each greening type vector cluster has one first centroid.
[0028] S2.4. Assign all labeled first municipal image block vectors to the greening type vector clusters where the corresponding greening type labels are located; for all unlabeled first municipal image block vectors, calculate the cosine similarity to each first centroid, and assign the unlabeled first municipal image block vectors to the greening type vector clusters corresponding to the first centroids with the largest cosine similarity; after the assignment is completed, if there is a greening type vector cluster without a first municipal image block vector, then randomly select a first municipal image block vector from the unlabeled first municipal image block vectors to update the corresponding first centroid;
[0029] S2.5 Calculate the mean of all first municipal image block vectors within each greening type vector cluster to obtain the new first centroid of the corresponding greening type vector cluster;
[0030] S2.6 Determine whether the change in the average cosine distance between all new first centroids and all first centroids is less than the set first convergence threshold. If yes, output the new first centroids and the corresponding greening type vector clusters. If no, update the corresponding first centroids with the new first centroids and return to S2.4.
[0031] Furthermore, if the final output contains a greening type vector cluster with a number of first municipal image block vectors less than the set minimum vector number threshold for the first cluster, then the greening type vector cluster is deleted (except for the cluster with labeled first municipal image block vectors), and the unlabeled samples in it are reassigned to the second most similar cluster, and the mean of each cluster is recalculated as the centroid of the final output.
[0032] In addition, the first cluster analysis can also use the Mini-Batch K-Means algorithm, the BIRCH algorithm, or the Gaussian Mixture Model (GMM), etc.
[0033] Furthermore, S2 also includes the following steps:
[0034] S2.1b. For each type of defect label (such as leaf spot, nutrient deficiency, pests, wilt, etc.), select the second municipal image block vector with the number of defective seeds for labeling.
[0035] Furthermore, the number of seed types and the number of defect seeds can be set to 5-10, that is, the number of first / second municipal image block vectors corresponding to each type of greening label and type defect label is 5-10.
[0036] Furthermore, S6 includes the following steps:
[0037] S6.1 Select the highest probability and greening type label from the first greening type probability data, and determine whether the highest probability is greater than or equal to the set probability upper limit threshold.
[0038] S6.2 If so, then generate image patch greening type data based on the highest probability and greening type label;
[0039] S6.3 If not, determine whether the maximum probability is less than the set upper probability threshold and greater than or equal to the set lower probability threshold.
[0040] S6.4 If so, then generate image block greening type data to be reviewed based on the highest probability and greening type label. After the review is approved, generate image block greening type data.
[0041] S6.5 If not, the corresponding image block greening type data will show that the image block has no greening.
[0042] S6.6 Select the highest probability and type defect label from the type defect probability data, and compare it with the set upper probability threshold and the set lower probability threshold according to steps S6.1 and S6.3 to generate image block type defect data. If it is greater than or equal to the set upper probability threshold, the image block type defect data is generated based on the highest probability and type defect label; if it is less than the set lower probability threshold, the image block type defect data shows that the image block has no defects; otherwise, the image block type defect pending review data is generated based on the highest probability and type defect label. After the review is approved, the image block type defect data is generated.
[0043] Furthermore, defect judgment can be performed on image blocks without greenery.
[0044] Furthermore, the method also includes the following steps:
[0045] S7.1 Collect the geographic coordinate data corresponding to each municipal image block and generate the geographic coordinate data of the image block;
[0046] S7.2 Collect image block greening type data, image block type defect data and image block geographic coordinate data for each municipal image block to form an image block three-dimensional data group;
[0047] S7.3 Collect the geographic coordinates of each target area, divide the image block geographic coordinate data into a target area three-dimensional data group, generate a target area greening distribution and defect data, and send it to the municipal greening maintenance personnel so that the municipal greening maintenance personnel can find out the greening problems in the target area based on the target area greening distribution and defect data and take action.
[0048] A computational analysis system for municipal greening maintenance based on image recognition includes an image acquisition module, a storage module, an intelligent chip, and a data transmission module;
[0049] The image acquisition module is used to acquire municipal images and store them in the storage module;
[0050] The storage module is also used to store computer programs;
[0051] The smart chip is used to run the computer program and execute a calculation and analysis method for municipal greening maintenance based on image recognition.
[0052] The data transmission module is used to send the intelligent chip's processing and analysis results to municipal greening maintenance personnel and system operation and maintenance personnel.
[0053] Compared with existing technologies, the present invention provides a computational analysis method and system for municipal greening maintenance based on image recognition. By first clustering municipal images and then associating them with labels, the reliance on manual annotation of images is greatly reduced. Moreover, the clustering process retains the characteristic information of the clusters, making the output of the model traceable and more interpretable.
[0054] By performing two different feature extractions on municipal images, and then clustering the features of greening types and defects based on the two extracted feature vectors respectively, the identification of greening types and defects of a certain type of greening becomes more targeted and improves the accuracy of identification. Attached Figure Description
[0055] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.
[0056] Figure 1 This is a flowchart illustrating the method steps provided in an embodiment of the present invention;
[0057] Figure 2 This is a system structure block diagram provided for an embodiment of the present invention. Detailed Implementation
[0058] To enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings.
[0059] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified. Furthermore, the terms "installed," "connected," and "linked" should be interpreted broadly; for example, they may refer to a fixed connection, a detachable connection, or an integral connection; they may refer to a mechanical connection or an electrical connection; they may refer to a direct connection or an indirect connection through an intermediate medium; and they may refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.
[0060] Exemplary embodiments will be described more fully below with reference to the accompanying drawings; however, these exemplary embodiments may be embodied in different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will enable those skilled in the art to fully understand the scope of this disclosure.
[0061] Where there is no conflict, the various embodiments of this disclosure and the features thereof in the embodiments may be combined with each other.
[0062] As used herein, the term “and / or” includes any and all combinations of one or more related enumerated entries.
[0063] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. As used herein, the singular forms “a” and “the” are also intended to include the plural forms, unless the context clearly indicates otherwise.
[0064] The embodiments described herein can be described with reference to plan views and / or cross-sectional views using the ideal schematic diagrams of this disclosure. Therefore, the example illustrations can be modified according to manufacturing techniques and / or tolerances. Therefore, the embodiments are not limited to those shown in the drawings, but include modifications to configurations formed based on manufacturing processes. Therefore, the areas illustrated in the drawings are schematic in nature, and the shapes of the areas shown in the figures illustrate specific shapes of areas of an element, but are not intended to be limiting.
[0065] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art.
[0066] Please see Figure 1 A computational analysis method for municipal greening maintenance based on image recognition includes the following steps:
[0067] S1. Acquire municipal images, preprocess and segment the municipal images to obtain multiple municipal image blocks, and perform first feature extraction on each municipal image block to generate a first municipal image block vector; perform second feature extraction on each municipal image block to generate a second municipal image block vector.
[0068] Preprocessing includes geometric correction, radiometric correction, and multi-source registration. Municipal images can be acquired using drones equipped with multispectral sensors (RGB+NIR+RedEdge), street view acquisition vehicles (binocular stereo camera + IMU), and IoT fixed-point cameras, ensuring a spatial resolution better than 5cm / pixel.
[0069] Segmenting municipal images includes the following steps:
[0070] The sliding window is set to slide on the municipal image with a set sliding step size. The sliding window extracts the internal image at each position and generates the corresponding municipal image block. The side length of the sliding window is greater than the sliding step size, so that adjacent municipal image blocks have partial overlap (such as 10%-60% overlap), thereby avoiding the critical target being cut off by the boundary.
[0071] In the first feature extraction, the extracted features include color features, texture features, and structural features. For color features, image patches can be converted to HSV and CIELAB spaces, and the mean and standard deviation of hue and saturation, as well as the green-red channel difference histogram (e.g., 16 bins), can be calculated according to the set superpixels (e.g., 32×32). Texture features can be obtained by calculating the contrast, entropy, and homogeneity of the gray-level co-occurrence matrix (GLCM), and the statistical histogram of local binary mode (LBP). Structural features can be extracted using a lightweight convolutional network (e.g., the first 7 layers of EfficientNet-B0, pre-trained on ImageNet), and further, the dimensionality can be compressed using principal component analysis. Then, these features are concatenated and normalized using the L2 norm to obtain the first municipal image patch vector.
[0072] In the second feature extraction, the extracted features include spectral information features, local anomaly features, and detailed textures. For spectral information features, these can be obtained by calculating the normalized vegetation index, normalized difference red edge index, and visible light atmospheric impedance index. Local anomaly features can be obtained by using Difference of Gaussian (DoG) bandpass filtering to highlight necrotic spots, insect holes, and discoloration, calculating the mean and extreme values of the filter response, or extracting edge density (such as the Canny edge ratio). Detailed textures can be obtained by extracting the gradient direction histogram to highlight local texture distortions such as leaf defects and lesions. Similarly, these features are concatenated and normalized using the L2 norm to obtain the second municipal image block vector.
[0073] S2. Collect all first municipal image block vectors and perform the first cluster analysis to generate multiple greening species vector clusters, including the following steps:
[0074] S2.1a. Based on the municipal greening GIS database, for each type of greening species (such as camphor, ginkgo, lawn, shrub, etc.), the first municipal image block vector with the number of seeds of each species is selected for labeling.
[0075] S2.2 Analyze the clustering effectiveness index (profile coefficient and Davies-Bouldin index) of all first municipal image block vectors as a function of the number of first clusters K, and determine the initial K value. K is greater than or equal to the number of categories with greening type labels. For example, take the minimum K value when the profile coefficient reaches a peak of 85%-95% as the initial K, and then fine-tune it according to the prior number of categories.
[0076] S2.3 For each type of greening label, calculate the mean of all labeled first municipal image block vectors as the first centroid of the greening type vector cluster corresponding to the greening type label. Then, randomly select zero to multiple first municipal image block vectors from the unlabeled first municipal image block vectors as the first centroids, so that the number of first centroids reaches K, that is, each greening type vector cluster has one first centroid.
[0077] S2.4. Assign all labeled first municipal image block vectors to the greening type vector clusters where the corresponding greening type labels are located; for all unlabeled first municipal image block vectors, calculate the cosine similarity to each first centroid, and assign the unlabeled first municipal image block vectors to the greening type vector clusters corresponding to the first centroids with the largest cosine similarity; after the assignment is completed, if there is a greening type vector cluster without a first municipal image block vector, then randomly select a first municipal image block vector from the unlabeled first municipal image block vectors to update the corresponding first centroid;
[0078] S2.5 Calculate the mean of all first municipal image block vectors within each greening type vector cluster to obtain the new first centroid of the corresponding greening type vector cluster;
[0079] S2.6 Determine whether the change in the average cosine distance between all new first centroids and all first centroids is less than the set first convergence threshold. If yes, output the new first centroids and the corresponding greening type vector clusters. If no, update the corresponding first centroids with the new first centroids and return to S2.4.
[0080] Furthermore, if the final output contains a greening type vector cluster with a number of first municipal image block vectors less than the set minimum vector number threshold for the first cluster, then the greening type vector cluster is deleted (except for the cluster with labeled first municipal image block vectors), and the unlabeled samples in it are reassigned to the second most similar cluster, and the mean of each cluster is recalculated as the centroid of the final output.
[0081] In addition, the first cluster analysis can also use the Mini-Batch K-Means algorithm, the BIRCH algorithm, or the Gaussian Mixture Model (GMM), etc.
[0082] S2 also includes: performing a second clustering analysis on the second municipal image block vector corresponding to the image block within each greening species vector cluster to obtain multiple species defect vector clusters;
[0083] The second cluster analysis can employ algorithms such as Hierarchical Agglomerative Clustering (HAC) and HDBSCAN clustering; or, through S2.1b, for each category, set a type defect label (such as leaf spot, nutrient deficiency, pests, wilt, etc.), select the second municipal image block vector with a set number of defective seeds for labeling, and then perform clustering in the same way as S2.2-S2.6.
[0084] Furthermore, the number of seed types and the number of defect seeds can be set to 5-10, that is, the number of first / second municipal image block vectors corresponding to each type of greening label and type defect label is 5-10.
[0085] S3. Associate each greening type vector cluster with the corresponding greening type label to generate a vector cluster type label lookup table; associate each type defect vector cluster with the corresponding type defect label to generate a vector cluster type defect label lookup table; when associating clusters with labels, appropriate labels can be selected and associated with corresponding vector clusters based on the experience of experts in relevant fields.
[0086] S4. Using the first municipal image block vector as input and the probability corresponding to the greening type label associated with each greening type vector cluster as output, the first model is trained by minimizing the sum of squared distances between each first municipal image block vector within the same greening type vector cluster and the first centroid of the corresponding cluster, and setting a first inter-cluster threshold as the first loss function to minimize the distance between the first centroids of corresponding clusters of different greening type vector clusters. Using the second municipal image block vector as input and the probability corresponding to the type defect label associated with each type defect vector cluster as output, the second model is trained by minimizing the sum of squared distances between each second municipal image block vector within the same type defect vector cluster and the second centroid of the corresponding cluster, and setting a second inter-cluster threshold as the second loss function to minimize the distance between the second centroids of corresponding clusters of different type defect vector clusters. The first / second model can be selected from models such as MLP, XGBoost, LightGBM, etc., as needed.
[0087] S5. Perform step S1 (preprocessing, segmentation, and first feature extraction) on the newly acquired municipal images to obtain a new first municipal image block vector. Input the new first municipal image block vector into the trained first model and output the probability corresponding to each greening type label to generate greening type probability data. Perform step S1 (preprocessing, segmentation, and second feature extraction) on the newly acquired municipal images to obtain a new second municipal image block vector. Input the new second municipal image block vector into the trained second model and output the probability corresponding to each type of defect label to generate type defect probability data.
[0088] S6. Select the highest probability and greening type label from the first greening type probability data to obtain the corresponding image block greening type data; select the highest probability and type defect label from the type defect probability data to obtain the corresponding image block type defect data, including the following steps:
[0089] S6.1 Select the highest probability and greening type label from the first greening type probability data, and determine whether the highest probability is greater than or equal to the set probability upper limit threshold.
[0090] S6.2 If so, then generate image patch greening type data based on the highest probability and greening type label;
[0091] S6.3 If not, determine whether the maximum probability is less than the set upper probability threshold and greater than or equal to the set lower probability threshold.
[0092] S6.4 If so, then generate image block greening type data to be reviewed based on the highest probability and greening type label. After the review is approved, generate image block greening type data.
[0093] S6.5 If not, the corresponding image block greening type data will show that the image block has no greening.
[0094] S6.6 Select the highest probability and type defect label from the type defect probability data, and compare it with the set upper probability threshold and the set lower probability threshold according to steps S6.1 and S6.3 to generate image block type defect data. If it is greater than or equal to the set upper probability threshold, the image block type defect data is generated based on the highest probability and type defect label; if it is less than the set lower probability threshold, the image block type defect data shows that the image block has no defects; otherwise, the image block type defect pending review data is generated based on the highest probability and type defect label. After the review is approved, the image block type defect data is generated.
[0095] Furthermore, defect judgment can be performed on image blocks without greenery.
[0096] S7: Based on the image patch greening type data, image patch type defect data, and corresponding image patch geographic coordinate data, generate a target area greening distribution and defect data and send it to municipal greening maintenance personnel. This includes the following steps:
[0097] S7.1 Collect the geographic coordinate data corresponding to each municipal image block and generate the geographic coordinate data of the image block;
[0098] S7.2 Collect image block greening type data, image block type defect data and image block geographic coordinate data for each municipal image block to form an image block three-dimensional data group;
[0099] S7.3 Collect the geographic coordinates of each target area, divide the image block geographic coordinate data into a target area three-dimensional data group, generate a target area greening distribution and defect data, and send it to the municipal greening maintenance personnel so that the municipal greening maintenance personnel can find out the greening problems in the target area based on the target area greening distribution and defect data and take action.
[0100] Please see Figure 2 A computational analysis system for municipal greening maintenance based on image recognition, comprising an image acquisition module, a storage module, an intelligent chip, and a data transmission module;
[0101] The image acquisition module is used to acquire municipal images and store them in the storage module;
[0102] The storage module is also used to store computer programs;
[0103] The intelligent chip is used to run computer programs and execute a calculation and analysis method for municipal greening maintenance based on image recognition;
[0104] The data transmission module is used to send the processing and analysis results of the smart chip to municipal greening maintenance personnel and system operation and maintenance personnel. Furthermore, system operation and maintenance personnel can also write data to the system through the data transmission module to achieve system maintenance, management and upgrades.
[0105] The foregoing has only described certain exemplary embodiments of the present invention by way of illustration. Undoubtedly, those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the foregoing drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.
Claims
1. A computational analysis method for municipal greening maintenance based on image recognition, characterized in that, Includes the following steps: S1. Acquire municipal images, preprocess and segment the municipal images to obtain multiple municipal image blocks, and extract the first feature for each municipal image block to generate the first municipal image block vector; S2. Collect all first municipal image block vectors and perform first cluster analysis to generate multiple greening species vector clusters; S3. Associate each greening type vector cluster with the corresponding greening type label to generate a vector cluster type label lookup table; S4. Take the first municipal image block vector as input, and the probability corresponding to the greening type label associated with each greening type vector cluster as output. Make the sum of squared distances between each first municipal image block vector in the same greening type vector cluster and the first centroid of the corresponding cluster the smallest. Set the first centroid distance between the corresponding clusters of different greening type vector clusters to be greater than the first inter-cluster threshold as the first loss function, and train the first model. S5. Perform step S1 on the newly acquired municipal images to obtain a new first municipal image block vector. Input the new first municipal image block vector into the trained first model and output the probability corresponding to each greening type label to generate greening type probability data. S6. Select the highest probability and greening type label from the first greening type probability data to obtain the corresponding image block greening type data.
2. The method for calculating and analyzing municipal greening maintenance based on image recognition according to claim 1, characterized in that, S1-S6 further includes the following steps: S1. Perform second feature extraction on each municipal image block to generate a second municipal image block vector; S2. Perform a second clustering analysis on the second municipal image block vector corresponding to the image block within each greening type vector cluster to obtain multiple type defect vector clusters; S3. Associate each type of defect vector cluster with the corresponding set type defect label to generate a vector cluster type defect label lookup table; S4. Take the second municipal image block vector as input, and the probability corresponding to the type defect label associated with each type defect vector cluster as output. Make the sum of squared distances between each second municipal image block vector in the same type defect vector cluster and the second centroid of the corresponding cluster the smallest. Set the second cluster threshold as the second loss function to make the distance between the second centroids of different types defect vector clusters greater than the second cluster threshold. Train the second model. S5. Perform step S1 on the newly acquired municipal images to obtain a new second municipal image block vector. Input the new second municipal image block vector into the trained second model and output the probability corresponding to various types of defect labels to generate type defect probability data. S6. Select the highest probability and the type defect label from the type defect probability data to obtain the corresponding image block type defect data.
3. The method for calculating and analyzing municipal greening maintenance based on image recognition according to claim 1, characterized in that, The segmentation of the municipal image in S1 includes the following steps: A sliding window is used to set a sliding step size to slide on the municipal image. The sliding window extracts the internal image at each position to generate a corresponding municipal image block. The side length of the sliding window is greater than the sliding step size.
4. The method for calculating and analyzing municipal greening maintenance based on image recognition according to claim 1, characterized in that, S2 includes the following steps: S2.1a. Based on the municipal greening GIS database, for each type of greening, select the first municipal image block vector with the number of seed types set for labeling the greening type; S2.2 Analyze the clustering effectiveness index of all first municipal image block vectors as a function of the number of first clusters K, and determine the initial K value. K is greater than or equal to the number of categories with greening type labels. S2.3 For each type of greening label, calculate the mean of all labeled first municipal image block vectors as the first centroid of the greening type vector cluster corresponding to the greening type label, and then randomly select zero to multiple first municipal image block vectors from the unlabeled first municipal image block vectors as the first centroids, so that the number of first centroids reaches K. S2.
4. Assign all labeled first municipal image block vectors to the greening type vector clusters where the corresponding greening type labels are located; for all unlabeled first municipal image block vectors, calculate the cosine similarity to each first centroid, and assign the unlabeled first municipal image block vectors to the greening type vector clusters corresponding to the first centroids with the largest cosine similarity; after the assignment is completed, if there is a greening type vector cluster without a first municipal image block vector, then randomly select a first municipal image block vector from the unlabeled first municipal image block vectors to update the corresponding first centroid; S2.5 Calculate the mean of all first municipal image block vectors within each greening type vector cluster to obtain the new first centroid of the corresponding greening type vector cluster; S2.6 Determine whether the change in the average cosine distance between all new first centroids and all first centroids is less than the set first convergence threshold. If yes, output the new first centroids and the corresponding greening type vector clusters. If no, update the corresponding first centroids with the new first centroids and return to S2.
4.
5. The method for calculating and analyzing municipal greening maintenance based on image recognition according to claim 2, characterized in that, S2 further includes the following steps: S2.1b: For each type of defect label setting, select the second municipal image block vector with the set defect seed number for annotation.
6. The method for calculating and analyzing municipal greening maintenance based on image recognition according to claim 2, characterized in that, S6 includes the following steps: S6.1 Select the highest probability and greening type label from the first greening type probability data, and determine whether the highest probability is greater than or equal to the set probability upper limit threshold. S6.2 If so, then generate image patch greening type data based on the highest probability and greening type label; S6.3 If not, determine whether the maximum probability is less than the set upper probability threshold and greater than or equal to the set lower probability threshold. S6.4 If so, then generate image block greening type data to be reviewed based on the highest probability and greening type label. After the review is approved, generate image block greening type data. S6.5 If not, the corresponding image block greening type data will show that the image block has no greening. S6.6 Select the highest probability and type defect label from the type defect probability data, and compare it with the set upper probability threshold and the set lower probability threshold according to steps S6.1 and S6.3 to generate image block type defect data.
7. The method for calculating and analyzing municipal greening maintenance based on image recognition according to claim 2, characterized in that, The method further includes the following steps: S7.1 Collect the geographic coordinate data corresponding to each municipal image block and generate the geographic coordinate data of the image block; S7.2 Collect image block greening type data, image block type defect data and image block geographic coordinate data for each municipal image block to form an image block three-dimensional data group; S7.3 Collect the geographic coordinates of each target area, divide the image block geographic coordinate data into a three-dimensional data group of image blocks that fall within the same target geographic coordinate area, generate a target area greening distribution and defect data, and send it to the municipal greening maintenance personnel.
8. A municipal greening maintenance calculation and analysis system based on image recognition, used to execute the municipal greening maintenance calculation and analysis method based on image recognition as described in any one of claims 1-7, characterized in that, It includes an image acquisition module, a storage module, a smart chip, and a data transmission module; The image acquisition module is used to acquire municipal images and store them in the storage module; The storage module is also used to store computer programs; The smart chip is used to run the computer program and execute the municipal greening maintenance calculation and analysis method based on image recognition as described in any one of claims 1-7; The data transmission module is used to send the intelligent chip's processing and analysis results to municipal greening maintenance personnel and system operation and maintenance personnel.