Method and system for determining green vision rate based on vehicle-mounted system

Abstract

The invention provides a green vision rate determination method and system based on a vehicle-mounted system, and the method employs multi-source data, improves the sensitivity of a model to vegetation through channel-space double attention and pyramid pooling in a prediction process of a two-dimensional green vision rate, and employs a scheme of combining intermediate loss and model loss. According to the method, convergence is accelerated, edge segmentation precision is improved, two-dimensional segmentation misjudgment is suppressed by using three-dimensional data, robustness is improved, lightweight design is carried out on a decoder, precision and efficiency are balanced, vehicle-mounted edge calculation is adapted, an energy function is used for further refining a segmentation result, and noise is reduced; according to the method, the collected multi-source data are preprocessed, mu s-level time synchronization and millimeter-level space registration can be realized, and the consistency of the multi-source data is ensured; according to the method, the laser radar point cloud is used for predicting the three-dimensional green vision rate, complementary optimization of the two-dimensional image and the three-dimensional point cloud is achieved through the dynamic weight model and semantic verification, and the prediction accuracy is effectively improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of urban greening, and particularly relates to a green view index determination method and system based on a vehicle-mounted system. BACKGROUND

[0002] Green view index (GVI) is a core index for measuring the level of urban greening, and directly reflects the visual perception of residents to green space. Studies have shown that a higher green view index can significantly improve the happiness of residents, relieve psychological stress, and reduce the incidence of cardiovascular diseases. Therefore, accurate and dynamic monitoring of the green view index is crucial for the construction of livable cities.

[0003] Current monitoring schemes for the green view index have defects such as being easily affected by light, shadow, and obstruction, large amount of data processing, and high requirements for the computing power of equipment. SUMMARY

[0004] The present disclosure at least provides a green view index determination method and system based on a vehicle-mounted system to solve at least one of the above technical problems.

[0005] According to an aspect of the present disclosure, a green view index determination method based on a vehicle-mounted system is provided, comprising:

[0006] Collecting multi-source data in a target area using a vehicle-mounted multi-source sensor, wherein the multi-source data includes two-dimensional images of the target area, laser radar point clouds, and pose information of the vehicle-mounted system; the vehicle-mounted system includes the vehicle-mounted multi-source sensor;

[0007] Preprocessing the multi-source data; wherein the preprocessing includes noise elimination and space-time alignment;

[0008] Processing the two-dimensional images after preprocessing using a pre-trained two-dimensional green view index model to obtain a two-dimensional green view index and a green view segmentation map; wherein the two-dimensional green view index model includes a standard U-Net model, a channel-space dual attention module inserted after each down-sampling stage of the encoder of the standard U-Net model, a multi-scale pyramid pooling layer set at the jump connection of the decoder of the standard U-Net model, and an auxiliary segmentation head added at each up-sampling stage of the decoder of the standard U-Net model to calculate an intermediate loss; wherein the two-dimensional green view index model uses a two-dimensional mask map of the vegetation segmentation result corresponding to the laser radar point cloud as prior knowledge in the processing process, and suppresses the misjudgment area in the two-dimensional image processing process through a gating mechanism; a hollow space pyramid pooling layer is set in the deep encoder of the standard U-Net model; the two-dimensional green view index model is a U-Net model enhanced by dual-channel attention; the encoder in the two-dimensional green view index model is a lightweight encoder;

[0009] The edge smoothness and region consistency of the green view segmentation graph are processed by using an energy function, to obtain a two-dimensional green view segmentation graph.

[0010] The ground is segmented from the preprocessed laser radar point cloud to obtain a non-ground point cloud; the non-ground point cloud is subjected to density clustering to obtain a vegetation point cloud; the vegetation point cloud is subjected to vegetation category segmentation based on the reflectivity, height and density of the vegetation point cloud; the vegetation volume ratio is determined based on the vegetation point cloud and the human eye perspective feature, and the vegetation volume ratio is taken as a three-dimensional green view rate; a three-dimensional green view rate heat map is generated based on the three-dimensional green view rate, and the proportion information of different categories of vegetation is generated according to the results of the vegetation category segmentation;

[0011] The two-dimensional green view rate corresponds to a two-dimensional weight according to the vegetation occlusion rate and the light normalization value; the three-dimensional green view rate corresponds to a three-dimensional weight according to the coverage rate; the fusion green view rate is determined based on the two-dimensional green view rate, the two-dimensional weight, the three-dimensional green view rate and the three-dimensional weight; and a fusion green view rate heat map is generated according to the fusion green view rate.

[0012] In a possible implementation, the intermediate loss corresponds to an intermediate loss function as follows:

[0013] ;

[0014] wherein P i is a prediction result of an i-th decoding stage, λ i is a weight coefficient, and DiceLoss represents a loss function for measuring the overlap degree of two samples, and the value of DiceLoss ranges from 0 to 1, and the smaller the value, the closer the prediction result is to the true label.

[0015] In a possible implementation, the loss function of the two-dimensional green view rate model includes a main loss function and a plurality of intermediate loss functions; and the weights of the main loss function and the intermediate loss functions are adjusted during the training of the two-dimensional green view rate model.

[0016] In a possible implementation, the encoder in the standard U-Net model is MobileNetV3-Small.

[0017] In a possible implementation, the determination of the vegetation volume ratio based on the vegetation point cloud and the human eye perspective feature includes:

[0018] The human eye emission ray is determined according to the human eye perspective feature.

[0019] The vegetation volume ratio is determined based on the human eye emission ray and the vegetation point cloud by using the following formula:

[0020] ;

[0021] wherein, GVI 3D is the three-dimensional green view fraction, representing the proportion of the vegetation volume in the visible area of the human eye view; V visible-green is the vegetation volume visible under the human eye view; V total_view is the volume visible under the human eye view, including vegetation and non-vegetation areas.

[0022] In a possible implementation, the vehicle-mounted system comprises a laser radar, a panoramic camera, a high-precision positioning device, a multi-layer PCB backboard, a bus, and a carbon fiber nested support; the laser radar and the panoramic camera are coaxially nested on the carbon fiber nested support;

[0023] The vehicle-mounted system further comprises a honeycomb silica gel shock-absorbing support; the laser radar is arranged on the honeycomb silica gel shock-absorbing support;

[0024] The vehicle-mounted system integrates a TEC semiconductor refrigeration sheet and an eddy current fan, which are attached to the bottom surface of the heat dissipation substrate of the laser radar, and are fixed in a manner of heat-conducting silica gel sheet + spring buckle combination;

[0025] The multi-layer PCB backboard integrates the high-precision positioning device, the laser radar, the panoramic camera, and the bus;

[0026] The vehicle-mounted system further comprises a shell and air-cooled cooling fins arranged on the shell.

[0027] In a possible implementation, the high-precision positioning device comprises a GNSS and an IMU;

[0028] The preprocessing of the multi-source data comprises:

[0029] Taking GNSS UTC time as a reference, the clocks of the laser radar, the panoramic camera, and the IMU are synchronized through a PTP protocol;

[0030] The angular velocity of the IMU and the position of the GNSS are fused by federated filtering to correct the time deviation of the multi-source data in real time;

[0031] The laser radar point cloud and the data of the GNSS are spatially registered through an affine transformation matrix;

[0032] The visual angle deviation of the multi-source data caused by vehicle vibration is dynamically corrected by using the attitude angle data of the IMU.

[0033] In a possible implementation, the method further comprises:

[0034] The green view ratio determination method is executed by using a distributed computing framework; wherein the distributed computing framework comprises a cloud and a plurality of computing nodes; the cloud divides data to be processed into a plurality of data blocks according to density, and generates data processing tasks and performs task scheduling; the data processing tasks comprise computationally intensive tasks and light tasks;

[0035] The task scheduling performed by the cloud comprises the following steps:

[0036] A load score of each computing node is determined;

[0037] The computationally intensive tasks are assigned to the computing nodes with a load score less than a first threshold value;

[0038] The light tasks are assigned to the computing nodes with a load score greater than or equal to the first threshold value and less than a second threshold value;

[0039] The computing nodes with a load score greater than or equal to the second threshold value are not assigned data processing tasks;

[0040] The load score of the computing node is determined in any one of the following manners:

[0041] According to the task timeout data of the computing node, a first weight corresponding to GPU utilization and a second weight corresponding to network bandwidth are determined; according to the CPU utilization, the GPU utilization, the network bandwidth, the video memory occupancy rate, the first weight, the second weight, a third weight corresponding to the CPU utilization, and a fourth weight corresponding to the video memory occupancy rate of the computing node, the load score of the computing node is determined;

[0042] Or, the CPU utilization, the GPU utilization, the network bandwidth, and the video memory occupancy rate of the computing node in a preset historical time period are input into a pre-trained LSTM model to obtain the load score of the computing node in a future preset time;

[0043] The task scheduling performed by the cloud further comprises the following steps:

[0044] The computing nodes with a load score less than a third threshold value send a task preemption request to the cloud; the third threshold value is less than the first threshold value;

[0045] The cloud allocates, based on the task preemption request, data processing tasks of the computing nodes with a load score greater than a fourth threshold value to the computing nodes with a load score less than the third threshold value; the fourth threshold value is greater than the first threshold value and less than the second threshold value;

[0046] The task scheduling performed by the cloud further comprises the following steps:

[0047] determining a scheduling priority of each data processing task;

[0048] storing each data processing task in a queue based on RabbitMQ, wherein RabbitMQ adjusts the position of the data processing task in the queue according to the scheduling priority.

[0049] In a possible implementation, the green view ratio determination method further includes:

[0050] The cloud receives task execution result data fed back by each computing node performing data processing tasks on data blocks, wherein the data blocks correspond to geographical ranges;

[0051] Converting the received task execution result data into a preset data format and performing coordinate alignment operation;

[0052] For task execution result data of data blocks without geographical range overlap, the task execution result data is spliced according to the geographical range;

[0053] For task execution result data of data blocks with geographical range overlap, the splicing weight of each data block is determined according to the confidence of each data block in the overlap area, and the task execution result data corresponding to each data block is spliced by using the splicing weight of each data block;

[0054] generating an initial fused green view ratio heat map according to the splicing result;

[0055] performing outlier correction, spatial interpolation, and pre-generating a multi-resolution fused green view ratio heat map using a pyramid model on the initial fused green view ratio heat map;

[0056] In the case where the task execution result data corresponding to the data block in the missing geographical range is determined according to the splicing result, the method further includes:

[0057] obtaining the data block in the missing geographical range, assigning a computing node to process, receiving the task execution result data fed back by the computing node, and returning the step of converting the received task execution result data into a preset data format and performing coordinate alignment operation.

[0058] According to another aspect of the present disclosure, a green view ratio determination system based on a vehicle-mounted system is provided, comprising:

[0059] The vehicle-mounted system in the green view ratio determination method based on a vehicle-mounted system according to any one of the above;

[0060] The cloud in the green view ratio determination method based on a vehicle-mounted system according to any one of the above;

[0061] The computing node in the green view ratio determination method based on a vehicle-mounted system according to any one of the above.

[0062] The vehicle-mounted system-based green view ratio determination method and system of the present disclosure first collects multi-source data in a target area using a vehicle-mounted multi-source sensor, and pre-processes the multi-source data. Then, a pre-trained two-dimensional green view ratio model is used to process the pre-processed two-dimensional image to obtain a two-dimensional green view ratio and a green view segmentation map. The two-dimensional green view ratio model includes a standard U-Net model, a channel-space dual attention module inserted after each down-sampling stage of the encoder of the standard U-Net model, a multi-scale pyramid pooling layer set at the skip connection of the decoder of the standard U-Net model, and an auxiliary segmentation head added at each up-sampling stage of the decoder of the standard U-Net model to calculate an intermediate loss. During processing, the two-dimensional mask map of the vegetation segmentation result corresponding to the three-dimensional point cloud is used as prior knowledge to suppress misjudgment areas in the two-dimensional image processing process through a gating mechanism. A hollow space pyramid pooling layer is set in the deep encoder of the standard U-Net model. Then, an energy function is used to process the green view segmentation map for edge smoothness and regional consistency to obtain a two-dimensional green view segmentation map. Then, ground segmentation is performed on the pre-processed laser radar point cloud to obtain non-ground point cloud. Density clustering is performed on the non-ground point cloud to obtain vegetation point cloud. Based on the reflectivity, height, and density of the vegetation point cloud, vegetation classification is performed. Based on the vegetation point cloud and human eye perspective features, the vegetation volume ratio is determined, and the vegetation volume ratio is used as the three-dimensional green view ratio. A three-dimensional green view ratio heat map is generated based on the three-dimensional green view ratio, and the proportion information of different types of vegetation is generated according to the results of the vegetation classification. Finally, the two-dimensional green view ratio corresponding two-dimensional weight is determined according to the vegetation occlusion ratio and the illumination normalization value. The three-dimensional green view ratio corresponding three-dimensional weight is determined according to the coverage rate. The fusion green view ratio is determined based on the two-dimensional green view ratio, two-dimensional weight, three-dimensional green view ratio, and three-dimensional weight. A fusion green view ratio heat map is generated based on the fusion green view ratio.The scheme of the present disclosure obtains multi-source data including two-dimensional data and three-dimensional data, enhances the sensitivity of the model to vegetation by channel-space double attention and pyramid pooling in the process of predicting two-dimensional green view rate by using the improved two-dimensional green view rate model, uses the scheme of combining intermediate loss and model itself loss to accelerate convergence and improve segmentation edge accuracy, uses three-dimensional data to suppress two-dimensional segmentation misjudgment and improve robustness, balances accuracy and efficiency by lightweight design of the decoder, adapts to vehicle-mounted edge computing, uses energy function to further refine the segmentation result and reduce noise; The scheme of the present disclosure pre-processes the collected multi-source data, can realize μs-level time synchronization and millimeter-level space registration, and ensures the consistency of multi-source data; The scheme of the present disclosure uses laser radar point cloud to predict three-dimensional green view rate, and realizes complementary optimization of two-dimensional image and three-dimensional point cloud through dynamic weight model and semantic verification, which effectively improves the accuracy of prediction; The combination of multi-source data in the scheme of the present disclosure can effectively overcome the defects that the monitoring result in the prior art is easily affected by light, shadow and shielding, and improve the accuracy of monitoring.

[0063] In summary, the present disclosure improves the prediction accuracy by using a multi-modal data fusion architecture, a double-channel attention enhanced U-Net model, three-dimensional point cloud assisted two-dimensional segmentation optimization, a lightweight decoder design, and a dynamic weight fusion algorithm, effectively overcomes the defects that the monitoring result in the prior art is easily affected by light, shadow and shielding, and improves the accuracy of monitoring.

[0064] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0065] The accompanying drawings are used to better understand the present scheme and do not constitute a limitation on the present disclosure. Among them:

[0066] Figure 1 is a flowchart of a green view rate determination method based on a vehicle-mounted system according to the present disclosure;

[0067] Figure 2 is a structural schematic diagram of a green view rate determination system based on a vehicle-mounted system according to the present disclosure. DETAILED DESCRIPTION

[0068] Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, which include various details of the embodiments of the present disclosure to help understanding, and should be considered only as exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Also, in order to be clear and concise, the description below omits the description of well-known functions and structures.

[0069] This disclosure addresses the shortcomings of current green view rate monitoring schemes, which suffer from drawbacks such as susceptibility to light, shadow, and occlusion, large data processing volume, high requirements for equipment computing power, low efficiency, high cost, and large monitoring equipment size. It provides a method and system for determining green view rate based on a vehicle-mounted system. This disclosure utilizes dynamic data acquisition, distributed computing, and multi-source data fusion technologies to achieve efficient and high-precision measurement and analysis of urban road green view rate, providing data support for urban planning, ecological assessment, and public health.

[0070] The technical solution of this disclosure will be described below through specific embodiments.

[0071] like Figure 1 The diagram shown is a flowchart of the green visibility rate determination method based on the vehicle-mounted system in this embodiment. The execution subject of this embodiment is a computing device, component, or system with data processing capabilities. Specifically, the method of this embodiment may include the following steps:

[0072] S110. Collect multi-source data within the target area using an on-board multi-source sensor, wherein the multi-source data includes a two-dimensional image of the target area, a lidar point cloud, and pose information of the on-board system; the on-board system includes the on-board multi-source sensor.

[0073] The vehicle-mounted system includes a LiDAR, a panoramic camera, a high-precision positioning device, a multi-layer PCB backplane, a bus, and a carbon fiber nested bracket; the LiDAR and panoramic camera are coaxially nested on the carbon fiber nested bracket. The vehicle-mounted system also includes a honeycomb silicone shock-absorbing bracket; the LiDAR is mounted on this bracket. The vehicle-mounted system integrates a TEC semiconductor cooling chip and a vortex fan. The multi-layer PCB backplane integrates the high-precision positioning device, LiDAR, panoramic camera, and bus. The vehicle-mounted system also includes a housing and air-cooled heat sinks mounted on the housing.

[0074] The high-precision positioning equipment includes GNSS and IMU.

[0075] S120. Preprocess the multi-source data; wherein the preprocessing includes noise removal and spatiotemporal alignment.

[0076] The above preprocessing may specifically include the following steps:

[0077] Taking GNSS UTC time as a reference, clocks of the laser radar, the panoramic camera and the IMU are synchronized through a PTP protocol; angular velocities of the IMU and positions of the GNSS are fused by federated filtering to correct time deviations of the multi-source data in real time; the laser radar point cloud and the GNSS data are spatially registered through an affine transformation matrix; and the multi-source data is dynamically corrected in terms of visual angle deviation caused by vehicle vibration by using attitude angle data of the IMU.

[0078] S130, the pre-processed two-dimensional image is processed by using a pre-trained two-dimensional green view ratio model to obtain a two-dimensional green view ratio and a green view segmentation map; wherein the two-dimensional green view ratio model comprises a standard U-Net model, a channel-space double attention module inserted after each down-sampling stage of the encoder of the standard U-Net model, a multi-scale pyramid pooling layer arranged at the jump connection of the decoder of the standard U-Net model, and an auxiliary segmentation head added at each up-sampling stage of the decoder of the standard U-Net model to calculate an intermediate loss; wherein the two-dimensional green view ratio model uses a two-dimensional mask map corresponding to the vegetation segmentation result of the laser radar point cloud as prior knowledge in the processing process, and suppresses the misjudgment area in the two-dimensional image processing process through a gating mechanism; and a hollow space pyramid pooling layer is arranged in a deep encoder in the standard U-Net model. The two-dimensional green view ratio model is a U-Net model enhanced by a double-channel attention; and the encoder in the two-dimensional green view ratio model is a lightweight encoder.

[0079] The intermediate loss function corresponding to the intermediate loss is:

[0080] ;

[0081] wherein P i is the prediction result of the i-th decoding stage, λ i is a weight coefficient, DiceLoss represents a loss function for measuring the overlap degree of two samples, the value range of DiceLoss is [0, 1], and the smaller the value is, the closer the prediction result is to the true label.

[0082] The loss function of the two-dimensional green view ratio model comprises a main loss function and a plurality of intermediate loss functions; and the weights of the main loss function and the intermediate loss functions are adjusted during the training process of the two-dimensional green view ratio model.

[0083] The encoder in the standard U-Net model is a lightweight MobileNetV3-Small.

[0084] S140, the green view segmentation map is processed in terms of edge smoothness and regional consistency by using an energy function to obtain a two-dimensional green view segmentation map.

[0085] S150, ground segmentation is performed on the laser radar point cloud after preprocessing to obtain a non-ground point cloud; density clustering is performed on the non-ground point cloud to obtain a vegetation point cloud; vegetation category segmentation is performed based on the reflectivity, height and density of the vegetation point cloud; the vegetation volume ratio is determined based on the vegetation point cloud and the human eye perspective feature, and the vegetation volume ratio is taken as the three-dimensional green view factor; a three-dimensional green view factor heat map is generated based on the three-dimensional green view factor, and the proportion information of different categories of vegetation is generated according to the results of the vegetation category segmentation.

[0086] The vegetation volume ratio based on the vegetation point cloud and the human eye perspective feature can specifically include:

[0087] The human eye emission ray is determined according to the human eye perspective feature; and the vegetation volume ratio is determined based on the human eye emission ray and the vegetation point cloud by using the following formula:

[0088]

[0089] Wherein, GVI 3D is the three-dimensional green view factor, indicating the proportion of the vegetation volume in the human eye perspective visible area; V visible-green is the vegetation volume visible under the human eye perspective; V total_view is the volume visible under the human eye perspective, including vegetation and non-vegetation areas.

[0090] S160, determining the two-dimensional weight corresponding to the two-dimensional green view factor according to the vegetation occlusion rate and the illumination normalization value; determining the three-dimensional weight corresponding to the three-dimensional green view factor according to the coverage rate; determining the fusion green view factor based on the two-dimensional green view factor, the two-dimensional weight, the three-dimensional green view factor and the three-dimensional weight; and generating a fusion green view factor heat map according to the fusion green view factor.

[0091] In some embodiments, the green view factor determination method is executed by using a distributed computing framework; wherein the distributed computing framework includes a cloud and a plurality of computing nodes; the cloud divides the data to be processed into a plurality of data blocks according to the density; and generates data processing tasks and performs task scheduling; the data processing tasks include computationally intensive tasks and lightweight tasks.

[0092] Wherein, the task scheduling step of the cloud is as follows:

[0093] Step one, determine the load score of each computing node.

[0094] Step two, assign computationally intensive tasks to computing nodes with a load score less than a first threshold.

[0095] Step three, assign lightweight tasks to computing nodes with a load score greater than or equal to a first threshold and less than a second threshold.

[0096] ​The computing node with the load score greater than or equal to the second threshold value does not allocate the data processing task.

[0097] The load score of the computing node is determined in any one of the following manners:

[0098] According to the task timeout data of the computing node, a first weight corresponding to GPU utilization and a second weight corresponding to network bandwidth are determined; according to the CPU utilization, the GPU utilization, the network bandwidth, the video memory occupancy rate, the first weight, the second weight, a third weight corresponding to the CPU utilization, and a fourth weight corresponding to the video memory occupancy rate of the computing node, a load score of the computing node is determined.

[0099] Alternatively,

[0100] The CPU utilization, the GPU utilization, the network bandwidth, and the video memory occupancy rate of the computing node within a preset historical time period are input into a pre-trained LSTM model to obtain a load score of the computing node within a future preset time.

[0101] The cloud end performs task scheduling in the following manner:

[0102] The computing node with the load score less than a third threshold value sends a task preemption request to the cloud end; the third threshold value is less than the first threshold value; the cloud end allocates, based on the task preemption request, a data processing task of a computing node with a load score greater than a fourth threshold value to the computing node with the load score less than the third threshold value; the fourth threshold value is greater than the first threshold value and less than the second threshold value.

[0103] The cloud end performs task scheduling in the following manner:

[0104] A scheduling priority of each data processing task is determined; each data processing task is stored in a queue based on RabbitMQ, wherein RabbitMQ adjusts a position of the data processing task in the queue according to the scheduling priority.

[0105] In some embodiments, the green view rate determination method further comprises:

[0106] The cloud receives task execution result data fed back by each computing node performing data processing tasks on the data blocks; wherein, the data blocks correspond to geographical ranges; the received task execution result data is converted into a preset data format and a coordinate alignment operation is performed; for task execution result data of data blocks without geographical range overlap, splicing is performed according to the geographical ranges; for task execution result data of data blocks with geographical range overlap, the splicing weights of each data block are respectively determined according to the confidence of each data block in the overlapping area, and the task execution result data corresponding to each data block is spliced by using the splicing weights of each data block; an initial fused green view rate heat map is generated according to the splicing result; the initial fused green view rate heat map is subjected to outlier correction, spatial interpolation and pre-generation of a multi-resolution fused green view rate heat map using a pyramid model.

[0107] In the case where the task execution result data corresponding to the data blocks in the missing geographical range is determined according to the splicing result, it further comprises:

[0108] The data blocks in the missing geographical range are obtained, computing nodes are allocated for processing, and task execution result data fed back by the computing nodes is received, and the step of converting the received task execution result data into a preset data format and performing a coordinate alignment operation is returned.

[0109] In some embodiments, a vehicle-mounted multi-source sensor integration design is adopted, in which the laser radar for collecting laser radar point cloud can adopt XT32M, and the performance parameters thereof are: ranging 300 meters, point frequency 64W points per second, wavelength 905nm, and supporting rain and fog penetration; and the anti-vibration design thereof is specifically: adopting a silica gel shock-absorbing support, and the vibration error suppression rate is ≥90%.

[0110] The panoramic camera for collecting two-dimensional images adopts Ladybug5+, and the optical parameters thereof are: 6-lens array, horizontal field of view 150°, vertical field of view 120°, and resolution 350dpi (single image 18MB). The trigger mechanism thereof is: DMI (Distance Measurement Instrument) distance trigger, and automatic shooting at 50-meter intervals.

[0111] The GNSS module is Beidou No. 3+RTK positioning, and the horizontal accuracy is ≤2 meters and the height accuracy is ≤3 meters. The IMU module is a 6-axis inertial measurement unit (3-axis accelerometer+3-axis gyroscope), the sampling frequency is 200Hz, and the attitude angle error is ≤0.1°.

[0112] The processor of the computing node is NVIDIA Jetson AGX Xavier, supporting CUDA parallel computing, and the power consumption is 30W. The storage module is 1TB NVMe SSD, supporting data caching and real-time uploading.

[0113] The laser radar (XT32M) and panoramic camera (Ladybug5+) adopt a carbon fiber nested bracket, internally integrated with a micro heat dissipation fin, with an overall size of Φ180mmx120mm and a weight reduction of 45%, and realize compact layout through coaxial nesting. The scheme utilizes honeycomb silica gel damping, and the vibration transfer function is attenuated by ≥25dB in the 1-100Hz frequency band (the actual measurement of the vibration suppression rate under the vehicle speed of 60km / h is 93%). The present disclosure realizes double-channel active heat dissipation, integrates a TEC semiconductor refrigeration sheet and an eddy current fan, and the working temperature range is expanded to -30°C to 70°C, and the temperature rise is ≤15°C in continuous operation for 4 hours.

[0114] The present disclosure designs a bus optimization scheme, and designs a 6-layer PCB backboard to integrate GNSS / IMU (CAN bus), laser radar (LVDS interface), and camera (GigE Vision), with a cable length reduction of 60% and a signal delay of ≤1μs. In the present disclosure, wide voltage power management is realized, supporting 9-36V input, with a peak power consumption of 120W, and equipped with a super capacitor buffer module (capacity 10F) to cope with vehicle power instantaneous fluctuation (±10V). In the present disclosure, heat dissipation is realized by using an aluminum alloy shell + air-cooled fin, and the working temperature range is -20°C to 60°C.

[0115] The technical scheme of the present disclosure realizes a multi-source space-time registration algorithm:

[0116] For time synchronization, based on IEEE 1588 PTP protocol, μs-level synchronization is realized through FPGA hardware timestamp, combined with federal Kalman filter (predicting IMU angular velocity and GNSS position), and the time residual is ≤0.3ms.

[0117] For space registration, an improved ICP (Iterative Closest Point) algorithm is adopted, introducing normal vector constraint and dynamic weight (point cloud curvature weight ratio 70%), and the registration accuracy is improved to 0.02 meters.

[0118] The scheme of the present disclosure is realized by using a distributed computing framework, wherein dynamic task scheduling can be realized, specifically: adaptive blocking, according to the point cloud density (5GB / block in sparse area, 3GB / block in dense area) and node load (GPU memory utilization>80% to suspend allocation), the response time is ≤5ms. Priority strategy, the task queue is sorted according to the "data freshness" (acquisition time difference), and the data collected within 10 minutes is processed preferentially, to ensure real-time performance. GPU acceleration optimization, point cloud denoising (statistical filtering) adopts CUDA parallel kernel function, and the single node processing speed is improved by 4 times (the actual measurement of 30GB data time consumption is reduced from 2170 seconds to 540 seconds). Data compression transmission, LASzip+entropy encoding is used, the compression rate is improved to 85%, and the bandwidth occupation is reduced by 75%.

[0119] The disclosure utilizes a two-dimensional greenness model to process a two-dimensional image, and realizes semantic segmentation. The two-dimensional greenness model can be an improved U-Net model structure, which will be described in detail below.

[0120] 1. Basic U-Net architecture The standard U-Net is composed of a symmetrical encoder (downsampling) and decoder (upsampling), and the shallow and deep features are fused through skip connection (SkipConnection). Its core advantage is to retain multi-scale spatial information, which is suitable for high-precision semantic segmentation tasks.

[0121] 2. Improvement points The following innovative designs are made on the basis of the standard U-Net to improve the accuracy and efficiency of greenness calculation: (1) Multi-scale attention fusion module (MSAF), used to enhance the model's attention ability to the vegetation area, while fusing multi-scale features.

[0122] Improvements to the encoder in the multi-scale attention fusion module: a channel-space dual attention module (CSDA) is inserted after each downsampling stage of the encoder.

[0123] Channel attention: dynamically adjust the channel weight through the SE (Squeeze-and-Excitation) module.

[0124] Spatial attention: generate a weight mask based on the spatial position of the feature map to highlight the vegetation area.

[0125] Improvements to the decoder in the multi-scale attention fusion module: introduce multi-scale pyramid pooling (MSPP) at the skip connection to fuse context information of different scales.

[0126] The pyramid pooling layer contains 4 different size pooling kernels (1x1, 3x3, 6x6, global average), and the output is spliced and fused with the decoder feature map.

[0127] (2) Deep supervision and dynamic loss function

[0128] Deep supervision is to add an auxiliary segmentation head at each upsampling stage of the decoder to calculate the intermediate loss.

[0129] Loss function of intermediate loss: ;

[0130] Where P i is the prediction result of the i-th decoding stage, and λ i is the weight coefficient (λ1=0.1, λ2=0.2, λ3=0.3, λ4=0.4).

[0131] The scheme of the present disclosure can realize dynamic loss adjustment, specifically, automatically adjusting the weights of the main loss function (cross-entropy loss) and the intermediate loss function according to the training stage, focusing on the auxiliary loss in the early stage to stabilize the training, and focusing on the main loss in the later stage to improve the accuracy.

[0132] (3) Cross-modal feature fusion (combined with point cloud data), used to optimize the segmentation result of the two-dimensional image by using three-dimensional point cloud data.

[0133] The implementation is specifically to project the point cloud classification result (vegetation / non-vegetation) into the two-dimensional image coordinate system to generate a mask graph. Cross-modal attention gate: in the skip connection, the point cloud mask graph is used as prior knowledge to suppress the misjudgment area (such as shadow, glass reflection) in the image through the gating mechanism.

[0134]

[0135] wherein σ is a Sigmoid function, and ⊕ represents a concatenation operation.

[0136] (4) Lightweight encoder design

[0137] Encoder replacement: replace the encoder (such as VGG16) of the standard U-Net with MobileNetV3-Small, which reduces the parameter amount (from 138M to 12M) while ensuring accuracy.

[0138] Dilated convolution replacement: use an all-space pyramid pooling (ASPP) in the deep encoder to expand the receptive field to capture a larger range of vegetation distribution.

[0139] (5) Post-processing optimization

[0140] Conditional random field (CRF), which performs fine processing on the segmentation result output by the model, and optimizes the edge smoothness and regional consistency through an energy function.

[0141] Energy function:

[0142] wherein ψ u is a Unary term (model prediction probability), and ψ p is a Pairwise term (similarity constraint based on color and spatial distance).

[0143] In this embodiment, the core data is a two-dimensional image, which is collected by a vehicle-mounted panoramic camera (such as Ladybug5+) in the target area. The result is: a green view segmentation graph (vegetation area mask; two-dimensional green view rate value (green pixel ratio).

[0144] Auxiliary data is the three-dimensional point cloud segmentation result, which comes from the point cloud data collected by the laser radar (such as XT32M). The result is: a three-dimensional model of vegetation point cloud; a vegetation category segmentation result (trees, shrubs, grassland); a two-dimensional mask map: project the three-dimensional vegetation point cloud into the two-dimensional image coordinate system to generate a mask of the vegetation area.

[0145] Other multi-source data is the pose information (GNSS / IMU): used for spatio-temporal alignment to ensure the consistency of two-dimensional images and three-dimensional point clouds in time and space (μs-level time synchronization + millimeter-level spatial registration); illumination normalization value: analyze the lighting conditions through two-dimensional images to dynamically adjust the green pixel judgment threshold (HSV space self-adaptation).

[0146] 3. Performance verification of two-dimensional green view rate model

[0147] Dataset: 100,000 labeled images (including urban roads, parks, buildings, and other scenes), with a division ratio of 8:1:1 (training: validation: testing).

[0148] Training configuration: Optimizer: AdamW (initial learning rate 3e-4, cosine annealing scheduling).

[0149] Data augmentation: random rotation (±30°), brightness jitter (±20%), shadow simulation.

[0150] Test results: mIoU: 92.1% (6.5% higher than the standard U-Net).

[0151] Inference speed: single frame processing time 23ms (1080p resolution, NVIDIA Jetson AGX Xavier).

[0152] Cross-scene generalization: mIoU reaches 89.3% on suburban road data not involved in training.

[0153] The scheme of the present application realizes multi-scale attention fusion: through channel-space dual attention and pyramid pooling, the sensitivity of the model to vegetation is enhanced. Deep supervision and dynamic loss are realized: to accelerate convergence and improve the accuracy of segmentation edges. Cross-modal fusion is realized: using point cloud data to suppress two-dimensional segmentation misjudgment and improve robustness. Lightweight design is realized: MobileNetV3+ASPP balances accuracy and efficiency, and adapts to vehicle-mounted edge computing. Post-processing optimization is also realized: CRF further refines the segmentation result and reduces noise.

[0154] Dynamic threshold optimization:

[0155] According to the intensity of light, the green pixel judgment threshold is adjusted adaptively (H∈[35, 85] in HSV space, S≥40, V≥20).

[0156] Formula: ;

[0157] In this disclosure, the point cloud data of the laser radar is mainly used in the following three aspects: three-dimensional modeling: constructing three-dimensional models of roads and vegetation through high-density point clouds. Vegetation classification: distinguishing trees, shrubs, and grasslands based on point cloud density, reflection intensity, and other features. Perspective simulation: simulating human eye perspective (horizontal 84° x vertical 62°, ground height 1.7 meters) to calculate the visible vegetation volume ratio.

[0158] The specific processing steps are as follows: 1) Denoising and preprocessing:

[0159] Use statistical filtering (Statistical Outlier Removal) to remove outliers (points with a distance greater than 2σ from the mean).

[0160] 2) Ground segmentation: separate ground and non-ground point clouds based on the RANSAC algorithm.

[0161] 3) Point cloud classification:

[0162] Density clustering (DBSCAN): separate vegetation and non-vegetation (such as buildings and vehicles).

[0163] Random forest classification: refine vegetation categories (trees, shrubs, grasslands) based on reflection intensity, height, and density features, with a classification accuracy F1-score ≥ 89%.

[0164] 4) Green view ratio calculation:

[0165] Ray casting algorithm: emit rays from the simulated human eye position to calculate the visible vegetation volume ratio.

[0166] Formula: ;

[0167] The processing result is: output three-dimensional green view ratio heat map, label vegetation vertical structure (such as tree layer accounting for 60%, shrub layer accounting for 25%).

[0168] For example, the output result is that the three-dimensional green view ratio of A section is 58.7%, and it is recommended to plant trees to improve the vertical coverage rate.

[0169] In this disclosure, the data of high-precision positioning equipment (GNSS / IMU) is mainly used in the following aspects: spatiotemporal reference: providing high-precision position (longitude, latitude, elevation) and attitude (pitch angle, roll angle, heading angle). Sensor synchronization: unify the timestamps and spatial coordinate systems of multiple sources.

[0170] The processing steps are as follows: 1) Time synchronization:

[0171] Synchronize LiDAR, camera, IMU clocks with PTP protocol, error < 0.5ms.

[0172] Federated filter fusion IMU angular velocity (200Hz) and GNSS position (1Hz), real-time correction of time deviation.

[0173] 2) Spatial registration:

[0174] Coordinate system alignment: register LiDAR point cloud (local coordinate system) and GNSS data (WGS84 coordinate system) through affine transformation matrix.

[0175] Attitude compensation: dynamically correct the viewing angle deviation caused by vehicle vibration using IMU attitude angle data, residual error < 0.03 meters.

[0176] The processing result is to generate a multi-source data set with accurate space-time label (such as point cloud coordinate error < 0.05 meters).

[0177] For example: after B section data registration, the space-time consistency error is reduced to 1.2 cm.

[0178] In this disclosure, panoramic camera image data is mainly used in the following aspects: two-dimensional green view rate calculation: extract the proportion of green pixels through semantic segmentation. Dynamic threshold optimization: adjust the green judgment range adaptively according to the lighting conditions.

[0179] The processing steps are as follows: 1) Image preprocessing:

[0180] Distortion correction: correct fisheye distortion based on camera calibration parameters (internal and external parameters).

[0181] Light equalization: use CLAHE algorithm to enhance the contrast of shadow area.

[0182] 2) Semantic segmentation:

[0183] Improved U-Net model: add channel attention mechanism, training set contains 100,000 labeled images, mIoU reaches 92%.

[0184] Dynamic threshold adjustment: in HSV space, the green pixel judgment range is adaptive (H∈35,8535,85, S≥40, V≥20).

[0185] 3) Green view rate calculation:

[0186] Formula: ;

[0187] The processing result is: output a two-dimensional green view rate distribution map, and mark the road sections with low green view rate (such as the section from K3+200 to K3+500 of Wenyu River, which has only 32.5%).

[0188] Example: The error between the 2D calculation result and the manual annotation is ≤3.5%.

[0189] After obtaining the two-dimensional and three-dimensional green view rates, they are fused. The steps are as follows:

[0190] 1. Data Fusion Process

[0191] Spatiotemporal alignment:

[0192] The pixel coordinates of the 2D image (via camera intrinsics) are mapped to the 3D point cloud coordinate system to establish the pixel-point cloud correspondence.

[0193] Dynamically compensate for viewpoint shifts caused by vehicle vibration using IMU attitude data.

[0194] Dynamic weight allocation, the specific calculation formula is as follows:

[0195] GVI Fusion = W2⋅GVI2D + W3⋅GVI3D

[0196] Wherein: GVI fusion is the fused green view rate, W2 is the two-dimensional weight, GVI2D is the two-dimensional green view rate, W3 is the three-dimensional weight, and GVI3D is the three-dimensional green view rate.

[0197] W2 = 0.6⋅e−0.1⋅Occlusion rate + 0.4⋅Normalized illumination value

[0198] W3 = 0.7⋅F1 - score + 0.3⋅coverage

[0199] Semantic consistency check:

[0200] Artifact removal: If the vegetation is identified in 2D but not in 3D, it is determined to be a light and shadow artifact (such as reflection between leaves), and the 2D weight W2 is set to zero.

[0201] Low-visibility vegetation supplementation: If the 3D classification is vegetation but the 2D classification is not identified, the 3D weight W3 is reduced by 50% by combining lidar reflectivity (>0.3).

[0202] 2. Validation of Fusion Results

[0203] Accuracy comparison: The combined green view rate MAE is 2.8%, which is 49% higher than the single two-dimensional method (MAE=5.5%).

[0204] Case verification: The green view rate of section C is 55.2%, which is closer to the human assessment value (54.5%) than the two-dimensional (58.7%) and three-dimensional (53.1%) values.

[0205] 3. Application output

[0206] Comprehensive report: Generate a PDF report containing the fused green view rate heat map, vegetation vertical structure analysis, and planting recommendations (e.g., supplementing Platanus orientalis).

[0207] Decision support: Based on three-dimensional point cloud analysis, recommend increasing the density of trees to 70%, which is expected to increase the green view rate to 65%.

[0208] The technical solution of the present disclosure realizes the deep fusion of multi-source data: through dynamic weight model and semantic verification, the complementary optimization of two-dimensional image and three-dimensional point cloud is realized. The technical solution of the present disclosure realizes high-precision space-time registration: μs-level time synchronization and millimeter-level space registration, ensuring the consistency of multi-source data. The technical solution of the present disclosure realizes compact hardware design: carbon fiber nested structure and double-channel cooling scheme, size ≤Φ180mm×120mm, vibration suppression rate ≥93%.

[0209] The present disclosure utilizes a distributed processing framework to perform the green view rate determination method, with the following architecture design:

[0210] Master Node (Master_Node): Deployed in the cloud, responsible for task scheduling, data block division (each block 5GB), and result merging.

[0211] Computing node (Slave_Node): Deployed on the edge server, single node configuration: Intel Xeon 16-core, 128GB memory, NVIDIA A100 GPU.

[0212] The task flow is as follows:

[0213] 1) Data block division: The master node divides 30GB of point cloud data into 6 blocks (each block 5GB).

[0214] 2) Dynamic scheduling of task implementation:

[0215] Existing dynamic scheduling schemes mostly rely on a single indicator (such as CPU utilization) or static threshold to allocate tasks, while this scheme first calculates the load score and then implements task scheduling based on the load score.

[0216] Multi-dimensional load indicator fusion, get load score. Among them, CPU utilization, GPU utilization, network bandwidth, video memory occupancy rate and other multi-dimensional indicators are considered at the same time, a normalized weighted scoring model is constructed to avoid single index deviation. And the dynamic weight adjustment is realized in the disclosure: according to the task type (such as compute-intensive, data-intensive), the weight of each index is dynamically adjusted (such as the GPU utilization weight in the GPU task is increased to 70%). In the scheduling process, the adaptive priority threshold is realized, specifically, fuzzy logic control: based on the historical load trend and task completion speed, the priority threshold is dynamically adjusted (such as priority when load <70%, but if the node response delay, the threshold is automatically lowered to 65%). At the same time, the elastic preemption mechanism is realized: allowing low-load nodes to preempt high-load nodes' unfinished tasks, ensuring task atomicity through distributed locks and reducing resource idling. In addition, the disclosure can also realize predictive scheduling optimization, specifically, using LSTM load prediction, that is, using long short-term memory network to predict the load trend of the node in the next 5 minutes, and allocating tasks in advance to avoid resource bottlenecks. In addition, task pre-migration is realized in the task scheduling process, that is, if it is predicted that a node will be overloaded, part of the tasks are migrated to a low-load node in advance to reduce real-time scheduling delay. Through multi-dimensional indicator fusion, adaptive threshold adjustment and predictive optimization, the scheme significantly improves the cluster resource utilization and task processing efficiency, and has substantial technical innovation.

[0217] Dynamic scheduling detailed steps

[0218] 21) Real-time monitoring and data collection of node state

[0219] The monitoring indicators are as follows:

[0220] CPU / GPU utilization: real-time collection by Prometheus+Node Exporter, sampling frequency 1 second.

[0221] Network bandwidth: use iftop to monitor the data transmission rate between nodes, distinguish uplink / downlink traffic.

[0222] Video memory occupancy rate: get GPU video memory usage through NVIDIA DCGM API.

[0223] Data normalization: normalize each indicator to the range of 0-1, formula:

[0224] Indicator i =(current value i -minimum value i )÷(maximum value i -minimum value i )

[0225] For example, the maximum value of GPU utilization is set to 100%, and the maximum value of network bandwidth is set to 10Gbps.

[0226] 22) Comprehensive load score calculation

[0227] Weighted score model:

[0228] Load score = w1•CPU utilization + w2•GPU utilization + w3•network bandwidth + w4•video memory occupancy

[0229] Initial weight configuration (can be dynamically adjusted):

[0230] w1=0.1, w2=0.6, w3=0.2, w4=0.1

[0231] Dynamic weight adjustment: if a computing node has a task timeout for three consecutive times, automatically increase the weight corresponding to GPU utilization to 0.7 and reduce the weight corresponding to network bandwidth to 0.1.

[0232] 23) Priority strategy and task allocation

[0233] Priority determination:

[0234] High priority (load score <0.7): preferentially allocate compute-intensive tasks (such as point cloud classification, semantic segmentation).

[0235] Medium priority (0.7≤load score <0.9): allocate light tasks (such as data denoising, format conversion).

[0236] Low priority (load score ≥0.9): suspend task allocation until the load decreases.

[0237] In addition, the task partitioning strategy in the present disclosure is as follows:

[0238] Density adaptive partitioning: when the point cloud density is >1000 points / ㎡, the partition size is 3GB; in sparse areas, the partition size is 5GB.

[0239] Task labeling: label the type of task (such as "vegetation classification" "heat map generation"), and match the node hardware characteristics (such as A100 GPU preferentially allocating deep learning tasks).

[0240] In addition, the predictive scheduling and elastic preemption steps in the present disclosure are as follows:

[0241] Load trend prediction:

[0242] LSTM model input: past 10 minutes of load sequence (CPU utilization, GPU utilization, network bandwidth, video memory occupancy).

[0243] Output: predicted value of load score in the next 5 minutes, if the predicted load score >0.8, trigger task pre-migration.

[0244] Elastic preemption mechanism:

[0245] Preemption condition: Low-load nodes (load score <0.6) can apply for preemption of high-load nodes (load score >0.8) tasks.

[0246] Atomicity guarantee: Ensure task state consistency through Redis distributed lock, and the preempted task re-joins the queue.

[0247] Real-time scheduling execution and fault tolerance are implemented in the present disclosure, and the specific implementation methods are as follows:

[0248] Task queue management: Multi-level priority queue is implemented using RabbitMQ, and high-priority tasks are inserted at the head of the queue. Task timeout (default 300 seconds) is automatically retried, and after a maximum of 3 times, it is marked as failed.

[0249] Node state synchronization: Real-time synchronization of node state through gRPC protocol, delay ≤10ms.

[0250] When a node is offline, its unfinished tasks are redistributed by the master node to other nodes.

[0251] Performance optimization and disaster recovery implementation in the present disclosure are as follows:

[0252] Data locality optimization: Preferentially allocate tasks to nodes that store related data, reducing network transmission overhead.

[0253] Disaster recovery backup: Critical tasks (such as green view rate calculation) are backed up in real time to at least 2 nodes to ensure quick recovery in the event of a single point failure.

[0254] Performance measurement and comparison of the scheme of the present disclosure are as follows:

[0255] Test environment:

[0256] Cluster size: 10 nodes (Intel Xeon 16-core + NVIDIA A100 GPU).

[0257] Dataset: 30GB point cloud + 2000 images.

[0258] Result comparison:

[0259] Speedup ratio: Dynamic scheduling strategy reduces task time from 2170 seconds (static partitioning) to 890 seconds, with a speedup ratio of 7.1 times.

[0260] Resource utilization: GPU average utilization rate increased from 65% to 85%, and network bandwidth utilization rate increased from 40% to 70%.

[0261] Fault tolerance: Task recovery time ≤5 seconds when a node fails, with zero data loss.

[0262] The technical solution of the present disclosure realizes multi-dimensional load fusion: breaking through the limitation of a single index, accurately evaluating node state. The technical solution of the present disclosure realizes adaptive threshold adjustment: dynamically optimizing priority strategy through fuzzy logic. The technical solution of the present disclosure realizes predictive scheduling: LSTM predicts load trend, avoiding resource bottleneck. The technical solution of the present disclosure realizes elastic preemption and atomicity guarantee: improving resource utilization while ensuring task consistency.

[0263] The present disclosure uses distributed computing nodes to perform parallel computing, and each computing node synchronously performs point cloud denoising (statistical filtering), ground classification (RANSAC), and green view ratio calculation. After obtaining the results, result merging is needed. Specifically, the following scheme can be used to realize result merging.

[0264] 1. Metadata management and data reception

[0265] Data block marking: the master node assigns a unique ID to each data block when it is divided, and records its geographical range (such as the latitude and longitude bounding box or projection coordinate range).

[0266] Result reception: after each computing node completes the task, it packs the results into the following format and sends them to the master node:

[0267] Packaging code

[0268] {

[0269] "block_id": "B001",

[0270] "geobbox": [116.301 ° E, 39.905 ° N, 116.305 ° E, 39.910 ° N], / / Geographical boundary

[0271] "gvi_data": "gvi_block_001.tif", / / Green view ratio raster file

[0272] "metadata": {"resolution": "0.5m", "timestamp": "2024-09-16T14:30:00Z"}

[0273] }

[0274] 2. Data preprocessing and format unification

[0275] Format conversion: if the data format returned by the node is inconsistent (such as some are GeoTIFF and some are NumPy arrays), the master node will convert it to a standard raster format (GeoTIFF).

[0276] Coordinate Alignment: Reproject raster data using GDAL library (e.g., unify to WGS84 coordinate system) to ensure consistent spatial reference.

[0277] 3. Spatial Mosaic and Data Fusion

[0278] Raster Mosaic Algorithm:

[0279] Non-overlapping Blocks: If data blocks have strictly continuous geographical ranges without overlap, directly mosaic by rows and columns.

[0280] Overlapping Blocks: Perform weighted averaging on overlapping areas, with weights assigned based on data confidence (e.g., point cloud density). Formula: ;

[0281] where wi is the confidence weight of the ith data block in the overlapping area.

[0282] Edge Processing: Smooth the edges of the global raster after mosaicking (e.g., using a Gaussian filter) to eliminate block gaps.

[0283] 4. Global Heatmap Generation

[0284] Color Mapping: Define a luminance color gradient (e.g., 0%-30% red, 30%-60% yellow, 60%-100% green).

[0285] Visualization Rendering: Use MapLibre GL or Python's Matplotlib library to render the merged raster data as a heatmap, overlaying road vector layers to enhance readability.

[0286] Interactive Functionality: Support zooming and clicking on a point to query its luminance value and display detailed vegetation classification information on the Web platform.

[0287] 5. Data Post-processing and Optimization

[0288] Outlier Correction: Replace pixels that deviate significantly from their neighbors (e.g., extreme values caused by sensor noise) with median filtering.

[0289] Spatial Interpolation: If there are missing data areas (e.g., due to node failure), use Kriging interpolation to estimate the missing values.

[0290] Performance Optimization: Pre-generate multi-resolution heatmaps using Pyramid Tiling to improve Web loading speed.

[0291] 6. Fault Tolerance and Dynamic Patching

[0292] Node failure handling: If a node times out (default 300 seconds), the master node re-adds the data block to the task queue and assigns it to other idle nodes.

[0293] Task status is recorded using Redis to avoid duplicate assignments.

[0294] Data integrity check: After merging, check if the global grid covers all geographic ranges. If there are missing parts, trigger the patching process.

[0295] 7. Actual case (D section)

[0296] Input data:

[0297] 6 nodes process 30GB point cloud data, returning 6 green view ratio grid blocks.

[0298] Merging process:

[0299] Splicing according to geographical boundaries, taking confidence weighted average in overlapping areas (weight based on point cloud density).

[0300] Gaussian filter (σ=1.5) to smooth edges.

[0301] Output results:

[0302] Global heat map shows an average green view ratio of 58.7%, with the lowest section (K3+200 to K3+500) only 32.5%.

[0303] Web platform supports interactive queries with response time <200ms.

[0304] The technical solution of the present disclosure realizes dynamic weighted splicing: confidence-based fusion in overlapping areas, improving result accuracy. The technical solution of the present disclosure realizes multi-resolution pyramid: optimizing large-scale data visualization performance. The technical solution of the present disclosure realizes fault tolerance and automatic patching: ensuring data integrity, adapting to the uncertainty of distributed environments.

[0305] The performance test of the present disclosure is as follows:

[0306] Test environment: 6-node cluster vs single machine (i7-11700K, RTX 3050).

[0307] Results: 30GB data processing time reduced from 6354 seconds to 2170 seconds, with a speedup of 2.93 times.

[0308] In the technical solution of the present disclosure, data management and application are as follows:

[0309] Metadata specification:

[0310] Field Definition: Latitude / Longitude (WGS84), Timestamp (ISO 8601), Image Number (UUID), Device ID.

[0311] Storage Format: Point Cloud (LAS 1.4), Image (GeoTIFF), Metadata (JSON).

[0312] Real-time Analysis Function:

[0313] Visualization Platform: Web-based support for green view rate heat map, vegetation vertical structure profile.

[0314] Report Generation: Automatically export PDF report, including green view rate ranking, noise correlation analysis, planting suggestions (such as planting suggestions for Platanus orientalis in the Wenyu River case).

[0315] Hardware Deployment Steps:

[0316] Sensor Installation: Laser radar and camera are fixed coaxially on the roof bracket, GNSS antenna is placed in the center of the roof, IMU is close to the vehicle's center of gravity. Use laser range finder to calibrate the relative position of each sensor (error ≤1cm).

[0317] Electrical Connection: 12V vehicle power supply, CAN bus transmission of IMU data, gigabit Ethernet connection to edge computing unit.

[0318] Software Configuration Process

[0319] System Initialization: Configure GNSS base station coordinates (such as Beijing Surveying and Mapping Bureau reference station), load pre-trained U-Net model (ONNX format).

[0320] Distributed Cluster Building: Use Kubernetes to manage computing nodes, set task queue (RabbitMQ) and data cache (Redis).

[0321] Data Collection and Processing Example (E Section)

[0322] Collection Parameters:

[0323] Mileage: 20km round trip, speed ≤60km / h, fixed distance trigger interval 50 meters.

[0324] Data Volume: Point Cloud 30GB, Image 2000 (about 36GB).

[0325] Processing Results:

[0326] Two-dimensional GVI: Average green view rate 58.7%, lowest section (K3+200 to K3+500) only 32.5% (need to plant).

[0327] Three-dimensional GVI: combined with point cloud analysis, it is suggested to increase the density of the arbor layer to improve the vertical green view rate.

[0328] Based on the same inventive concept, the disclosure provides a vehicle-mounted system-based green view rate determination system, the components of which perform the same or similar steps as the above-mentioned method, and therefore similar parts will not be described again. As shown in the figure, the vehicle-mounted system-based green view rate determination system of the embodiment includes: Figure 2

[0329] The vehicle-mounted system 201 in any of the above embodiments;

[0330] The cloud 202 in any of the above embodiments;

[0331] At least one computing node 203 in any of the above embodiments.

[0332] The various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuitry, integrated circuitry, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0333] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces the functions / operations specified in the flowcharts and / or the block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine as a stand-alone software package, and partially on a remote machine or entirely on a remote machine or server.

[0334] ​In the context of this disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0335] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0336] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0337] The computer system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0338] It should be understood that the various forms of flow shown above can be used to reorder, add, or remove steps. For example, the steps described in the present disclosure can be performed in parallel, in series, or in a different order, without limitation herein, so long as the desired results of the technology described in the present disclosure are achieved.

[0339] The specific implementation described above does not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for determining a green viewing ratio based on a vehicle-mounted system, characterized by, The application relates to a green view rate fusion method and device. Collecting multi-source data in a target area by using a vehicle-mounted multi-source sensor, wherein the multi-source data comprises a two-dimensional image of the target area, a laser radar point cloud and pose information of a vehicle-mounted system; the vehicle-mounted system comprises the vehicle-mounted multi-source sensor; Preprocessing the multi-source data, wherein the preprocessing comprises noise elimination and space-time alignment; Processing the two-dimensional image after preprocessing by using a pre-trained two-dimensional green view rate model to obtain a two-dimensional green view rate and a green view segmentation map; wherein the two-dimensional green view rate model comprises a standard U-Net model, a channel-space double attention module inserted after each down-sampling stage of an encoder of the standard U-Net model, a multi-scale pyramid pooling layer arranged at a jump connection of a decoder of the standard U-Net model, and an auxiliary segmentation head added to each up-sampling stage of the decoder of the standard U-Net model to calculate an intermediate loss; in the processing process of the two-dimensional green view rate model, a two-dimensional mask map of a vegetation segmentation result corresponding to the laser radar point cloud is used as prior knowledge to inhibit misjudgment areas in the processing of the two-dimensional image by using a gating mechanism; a hollow space pyramid pooling layer is arranged in a deep encoder of the standard U-Net model; the two-dimensional green view rate model is a U-Net model enhanced by double-channel attention; the encoder in the two-dimensional green view rate model is a lightweight encoder; Performing edge smoothness and region consistency processing on the green view segmentation map by using an energy function to obtain a two-dimensional green view segmentation map; Performing ground segmentation on the laser radar point cloud after preprocessing to obtain a non-ground point cloud; performing density clustering on the non-ground point cloud to obtain a vegetation point cloud; performing vegetation category segmentation based on the reflection intensity, height and density of the vegetation point cloud; determining a vegetation volume proportion based on the vegetation point cloud and human eye perspective features, and taking the vegetation volume proportion as a three-dimensional green view rate; generating a three-dimensional green view rate heat map based on the three-dimensional green view rate, and generating proportion information of different categories of vegetation according to the results of the vegetation category segmentation; Determining a two-dimensional weight corresponding to the two-dimensional green view rate according to a vegetation shielding rate and a light normalization value; determining a three-dimensional weight corresponding to the three-dimensional green view rate according to a coverage rate; determining a fusion green view rate based on the two-dimensional green view rate, the two-dimensional weight, the three-dimensional green view rate and the three-dimensional weight; and generating a fusion green view rate heat map according to the fusion green view rate.

2. The on-board system-based determination method of the green viewing ratio according to claim 1, characterized in that, The intermediate loss function corresponding to the intermediate loss is as follows: ; wherein P i is the prediction result of the i-th decoding stage, λ i is a weight coefficient, and DiceLoss represents a loss function for measuring the overlap degree of two samples, the value of DiceLoss ranges from 0 to 1, and the smaller the value, the closer the prediction result is to the true label.

3. The on-board system-based determination method of the green viewing ratio according to claim 2, characterized in that, The loss function of the two-dimensional green view rate model comprises a main loss function and a plurality of intermediate loss functions; the weights of the main loss function and the intermediate loss functions are adjusted in the training process of the two-dimensional green view rate model.

4. The on-board system-based determination method of green viewing ratio according to claim 1, characterized in that, The encoder in the standard U-Net model is a MobileNetV3-Small.

5. The vehicle-mounted system-based green viewing ratio determination method according to claim 1, characterized by, The determination of the vegetation volume proportion based on the vegetation point cloud and the human eye perspective features comprises: Determining a human eye emission ray according to the human eye perspective features; Determining the vegetation volume proportion based on the human eye emission ray and the vegetation point cloud by using the following formula: ; wherein GVI 3D is the three-dimensional green view index, representing the proportion of the vegetation volume in the visible area of the human eye view; V visible-green is the vegetation volume visible under the human eye view; V total_view is the volume visible under the human eye view, including vegetation and non-vegetation areas.

6. The vehicle-mounted system-based green viewing ratio determination method according to claim 1, characterized by, The vehicle-mounted system comprises a laser radar, a panoramic camera, a high-precision positioning device, a multi-layer PCB backboard, a bus and a carbon fiber nested support; the laser radar and the panoramic camera are coaxially nested on the carbon fiber nested support; The vehicle-mounted system further comprises a honeycomb silica gel shock-absorbing support; the laser radar is arranged on the honeycomb silica gel shock-absorbing support; The vehicle-mounted system integrates a TEC semiconductor refrigeration sheet and an eddy current fan, which are attached to the bottom surface of a heat dissipation substrate of the laser radar, and are fixed in a manner of heat-conducting silica gel sheet + spring buckle combination; The multi-layer PCB backboard integrates the high-precision positioning device, the laser radar, the panoramic camera and the bus; The vehicle-mounted system further comprises a shell and air-cooled cooling fins arranged on the shell.

7. The on-board system-based determination method of the green viewing ratio according to claim 6, characterized in that, The high-precision positioning device comprises a GNSS and an IMU; The preprocessing of the multi-source data comprises: Taking GNSS UTC time as a reference, the clocks of the laser radar, the panoramic camera and the IMU are synchronized through a PTP protocol; The angular velocity of the IMU and the position of the GNSS are fused by using federated filtering to correct the time deviation of the multi-source data in real time; The laser radar point cloud and the GNSS data are spatially registered through an affine transformation matrix; The viewing angle deviation of the multi-source data caused by vehicle vibration is dynamically corrected by using the attitude angle data of the IMU.

8. The vehicle-based system-based green viewing ratio determination method according to claim 1, characterized by, Further comprising: The green view rate determination method is executed by using a distributed computing framework; wherein the distributed computing framework comprises a cloud and a plurality of computing nodes; the cloud divides the data to be processed into a plurality of data blocks according to density, and generates data processing tasks and performs task scheduling; the data processing tasks comprise computationally intensive tasks and lightweight tasks; The task scheduling step of the cloud is as follows: Determine the load score of each computing node; Assign computationally intensive tasks to computing nodes with a load score less than a first threshold value; Assign lightweight tasks to computing nodes with a load score greater than or equal to a first threshold value and less than a second threshold value; Computing nodes with a load score greater than or equal to a second threshold value are not assigned data processing tasks; The load score of the computing node is determined in any of the following ways: According to the task timeout data of the computing node, determine a first weight corresponding to GPU utilization and a second weight corresponding to network bandwidth; according to the CPU utilization, GPU utilization, network bandwidth, video memory occupancy rate, first weight, second weight, third weight corresponding to CPU utilization, fourth weight corresponding to video memory occupancy rate of the computing node, determine the load score of the computing node; Or, Input the CPU utilization, GPU utilization, network bandwidth and video memory occupancy rate of the computing node within a preset historical time period into a pre-trained LSTM model to obtain the load score of the computing node within a future preset time; The task scheduling step of the cloud further comprises: Computing nodes with a load score less than a third threshold value send a task preemption request to the cloud; the third threshold value is less than the first threshold value; The cloud allocates the data processing task of the computing node with the load score greater than the fourth threshold to the computing node with the load score less than the third threshold based on the task preemption request; wherein the fourth threshold is greater than the first threshold and less than the second threshold; The cloud performs the task scheduling step further comprises: determining the scheduling priority of each data processing task; storing each data processing task using a RabbitMQ-based queue, wherein RabbitMQ adjusts the position of the data processing task in the queue according to the scheduling priority.

9. The in-vehicle system-based green viewing ratio determination method according to claim 8, characterized by, The green view rate determination method further comprises: The cloud receives the task execution result data fed back by each computing node for performing the data processing task on the data block; wherein the data block corresponds to a geographical range; convert the received task execution result data into a preset data format and perform coordinate alignment operation; for the task execution result data of the data block without geographical range overlap, according to the geographical range, the task execution result data of the data block without geographical range overlap is spliced according to the confidence of each data block in the overlapping area; for the task execution result data of the data block with geographical range overlap, respectively determine the splicing weight of each data block according to the confidence of each data block in the overlapping area, and splice the task execution result data corresponding to each data block using the splicing weight of each data block; generate an initial fusion green view rate heat map according to the splicing result; perform outlier correction, spatial interpolation and pre-generate multi-resolution fusion green view rate heat map using pyramid model on the initial fusion green view rate heat map; wherein, in the case of determining the task execution result data corresponding to the data block in the missing geographical range according to the splicing result, further comprising: obtain the data block in the missing geographical range, assign a computing node for processing, receive the task execution result data fed back by the computing node, and return the step of converting the received task execution result data into a preset data format and performing coordinate alignment operation.

10. A vehicle-mounted system-based green viewing ratio determination system, characterized by comprising: comprises: the vehicle-mounted system in the green view rate determination method based on the vehicle-mounted system according to any one of claims 1 to 9; the cloud in the green view rate determination method based on the vehicle-mounted system according to any one of claims 8 to 9; at least one computing node in the green view rate determination method based on the vehicle-mounted system according to any one of claims 8 to 9.

Images