Method and system for determining green view ratio based on vehicle-mounted system

Abstract

The present disclosure provides a kind of green view ratio determination method and system based on vehicle-mounted system, the present disclosure utilizes multi-source data, in the process of predicting two-dimensional green view ratio, through channel-space double attention and pyramid pooling, enhance the sensitivity of model to vegetation, use the scheme of intermediate loss and model itself loss combination, accelerate convergence and improve segmentation edge precision, using three-dimensional data to suppress two-dimensional segmentation misjudgment, improve robustness, the decoder is lightened design, to balance accuracy and efficiency, adapt to vehicle-mounted edge computing, using energy function further refines segmentation result, reduce noise;The present disclosure has carried out pre-processing to the multi-source data collected, can realize μs level time synchronization and millimeter level space registration, ensure the consistency of multi-source data;The present disclosure utilizes laser radar point cloud to carry out three-dimensional green view ratio prediction, and realizes the complementary optimization of two-dimensional image and three-dimensional point cloud by dynamic weight model and semantic verification, effectively improves the accuracy of prediction.

Description

Technical Field

[0001] This disclosure relates to the field of urban greening, and in particular to a method and system for determining green visibility based on a vehicle-mounted system. Background Technology

[0002] Green View Index (GVI) is a core indicator for measuring the level of urban greening, directly reflecting residents' visual perception of green spaces. Studies have shown that a higher GVI can significantly improve residents' sense of well-being, alleviate psychological stress, and reduce the incidence of cardiovascular diseases. Therefore, accurate and dynamic monitoring of the GVI is crucial for building livable cities.

[0003] Current green visibility monitoring schemes have drawbacks such as the monitoring results being easily affected by light, shadows, and obstruction, large data processing volume, and high requirements for the computing power of equipment. Summary of the Invention

[0004] This disclosure provides at least one method and system for determining green visibility rate based on an in-vehicle system, in order to solve at least one of the above-mentioned technical problems.

[0005] According to one aspect of this disclosure, a method for determining green visibility rate based on an in-vehicle system is provided, comprising:

[0006] Multi-source data within a target area is collected using an onboard multi-source sensor. The multi-source data includes a two-dimensional image of the target area, a lidar point cloud, and pose information from the onboard system. The onboard system includes the onboard multi-source sensor.

[0007] The multi-source data is preprocessed; wherein, the preprocessing includes noise removal and spatiotemporal alignment.

[0008] A pre-trained 2D green visibility model is used to process pre-processed 2D images to obtain 2D green visibility and green visibility segmentation maps. The 2D green visibility model includes a standard U-Net model, a channel-spatial dual attention module inserted after each downsampling stage of the encoder in the standard U-Net model, a multi-scale pyramid pooling layer at the skip connections of the decoder in the standard U-Net model, and an auxiliary segmentation head added to each upsampling stage of the decoder in the standard U-Net model to calculate intermediate loss. During processing, the 2D green visibility model uses a 2D mask map of vegetation segmentation results corresponding to the LiDAR point cloud as prior knowledge and uses a gating mechanism to suppress misclassified regions during 2D image processing. The deep encoder in the standard U-Net model includes a hollow spatial pyramid pooling layer. The 2D green visibility model is a dual-channel attention-enhanced U-Net model. The encoder in the 2D green visibility model is a lightweight encoder.

[0009] The green view segmentation map is processed for edge smoothness and region consistency using an energy function to obtain a two-dimensional green view segmentation map.

[0010] The preprocessed lidar point cloud is segmented to obtain a non-ground point cloud; density clustering is performed on the non-ground point cloud to obtain a vegetation point cloud; vegetation categories are segmented based on the reflection intensity, height, and density of the vegetation point cloud; the vegetation volume ratio is determined based on the vegetation point cloud and human eye view features, and the vegetation volume ratio is used as the three-dimensional green view rate; a three-dimensional green view rate heatmap is generated based on the three-dimensional green view rate, and the proportion information of different vegetation categories is generated according to the results of vegetation category segmentation.

[0011] The two-dimensional weights corresponding to the two-dimensional green view rate are determined based on the vegetation shading rate and the normalized value of light. The three-dimensional weights corresponding to the three-dimensional green view rate are determined based on the coverage rate. The fused green view rate is determined based on the two-dimensional green view rate, the two-dimensional weights, the three-dimensional green view rate, and the three-dimensional weights. A fused green view rate heatmap is generated based on the fused green view rate.

[0012] In one possible implementation, the intermediate loss function corresponding to the intermediate loss is:

[0013] ;

[0014] Among them, P i Let λ be the prediction result for the i-th decoding stage. i is the weight coefficient, and DiceLoss represents the loss function used to measure the overlap between two samples. The value of DiceLoss ranges from [0, 1]. The smaller the value, the closer the predicted result is to the true label.

[0015] In one possible implementation, the loss function of the two-dimensional green visibility model includes a main loss function and several intermediate loss functions; the weights of the main loss function and each intermediate loss function are adjusted during the training of the two-dimensional green visibility model.

[0016] In one possible implementation, the encoder in the standard U-Net model is MobileNetV3-Small.

[0017] In one possible implementation, determining the vegetation volume percentage based on vegetation point cloud and human eye view features includes:

[0018] Determine the rays emitted by the human eye based on the characteristics of the human eye's visual perspective;

[0019] Based on the emitted rays from the human eye and vegetation point clouds, the vegetation volume percentage is determined using the following formula:

[0020] ;

[0021] Among them, GVI 3D The three-dimensional green visibility rate represents the proportion of vegetation volume within the area visible to the human eye; V visible-green The volume of vegetation visible from a human eye's perspective; V total_view The volume visible from the human eye, including both vegetated and non-vegetated areas.

[0022] In one possible implementation, the vehicle-mounted system includes a lidar, a panoramic camera, a high-precision positioning device, a multi-layer PCB backplane, a bus, and a carbon fiber nested bracket; the lidar and panoramic camera are coaxially nested on the carbon fiber nested bracket.

[0023] The vehicle-mounted system also includes a honeycomb silicone shock-absorbing bracket; the lidar is mounted on the honeycomb silicone shock-absorbing bracket.

[0024] The vehicle-mounted system integrates a TEC semiconductor cooling chip and a vortex fan, which are attached to the bottom surface of the heat dissipation substrate of the lidar and fixed by a combination of thermally conductive silicone sheet and spring clips.

[0025] The multi-layer PCB backplane integrates the high-precision positioning device, LiDAR, panoramic camera, and bus.

[0026] The vehicle system also includes a housing and air-cooled heat sinks mounted on the housing.

[0027] In one possible implementation, the high-precision positioning device includes GNSS and IMU;

[0028] The preprocessing of the multi-source data includes:

[0029] Using GNSS UTC time as a reference, the clocks of the lidar, panoramic camera, and IMU are synchronized via the PTP protocol;

[0030] By using federated filtering to fuse the angular velocity of the IMU and the position of the GNSS, the time deviation of the multi-source data is corrected in real time.

[0031] Spatial registration of lidar point cloud and GNSS data is performed using an affine transformation matrix.

[0032] The attitude angle data from the IMU is used to dynamically correct the viewpoint shift of the multi-source data caused by vehicle vibration.

[0033] In one possible implementation, it also includes:

[0034] The green visibility rate determination method is executed using a distributed computing framework; wherein the distributed computing framework includes a cloud and multiple computing nodes; the cloud divides the data to be processed into multiple data blocks according to density; and generates data processing tasks and schedules them; the data processing tasks include computationally intensive tasks and lightweight tasks.

[0035] The steps for task scheduling in the cloud are as follows:

[0036] Determine the load score for each compute node;

[0037] Assign computationally intensive tasks to computing nodes whose load scores are less than the first threshold;

[0038] Lightweight tasks are assigned to computing nodes whose load scores are greater than or equal to the first threshold and less than the second threshold.

[0039] Computing nodes with a load score greater than or equal to the second threshold are not assigned data processing tasks;

[0040] The load score of the computing node is determined according to any of the following methods:

[0041] Based on the task timeout data of the computing node, determine the first weight corresponding to GPU utilization and the second weight corresponding to network bandwidth; based on the CPU utilization, GPU utilization, network bandwidth, video memory utilization, first weight, second weight, third weight corresponding to CPU utilization, and fourth weight corresponding to video memory utilization of the computing node, determine the load score of the computing node.

[0042] Alternatively, the CPU utilization, GPU utilization, network bandwidth, and video memory usage of the computing node within a preset historical time period can be input into a pre-trained LSTM model to obtain the load score of the computing node in the future preset time period.

[0043] The cloud-based task scheduling steps also include:

[0044] Computing nodes whose load scores are less than a third threshold send a task preemption request to the cloud; the third threshold is less than the first threshold.

[0045] Based on the task preemption request, the cloud allocates data processing tasks of computing nodes with load scores greater than a fourth threshold to computing nodes with load scores less than a third threshold; wherein the fourth threshold is greater than the first threshold and less than the second threshold.

[0046] The cloud-based task scheduling steps also include:

[0047] Determine the scheduling priority of each data processing task;

[0048] Each data processing task is stored in a RabbitMQ-based queue, where RabbitMQ adjusts the position of the data processing task in the queue according to scheduling priority.

[0049] In one possible implementation, the green visibility rate determination method further includes:

[0050] The cloud receives task execution result data from each computing node performing data processing tasks on the data block; wherein, the data block corresponds to a geographical range;

[0051] The received task execution result data is converted into a preset data format and then aligned to coordinates.

[0052] For task execution result data that do not have overlapping geographical ranges, the data is spliced ​​together according to the geographical range;

[0053] For task execution result data with overlapping geographical areas, the splicing weight of each data block is determined according to the confidence level of each data block in the overlapping area. The task execution result data corresponding to each data block is spliced ​​together using the splicing weight of each data block.

[0054] An initial fused green view rate heatmap is generated based on the splicing results;

[0055] The initial fused green view rate heatmap is subjected to outlier correction, spatial interpolation, and multi-resolution fused green view rate heatmaps are pre-generated using a pyramid model;

[0056] This includes, in cases where the splicing results determine that the task execution result data corresponding to the missing data blocks within a geographical range is missing, the following:

[0057] The steps include: acquiring data blocks that are missing from a geographical area, allocating computing nodes for processing, receiving task execution result data from the computing nodes, converting the received task execution result data into a preset data format, and performing coordinate alignment.

[0058] According to another aspect of this disclosure, a green visibility rate determination system based on an in-vehicle system is provided, comprising:

[0059] The vehicle system in the green visibility rate determination method based on the vehicle system described in any of the above-mentioned methods;

[0060] The cloud in any of the above-mentioned methods for determining green visibility based on vehicle systems;

[0061] The calculation node in the green visibility rate determination method based on vehicle-mounted systems described in any of the above-mentioned methods.

[0062] This disclosure discloses a method and system for determining green visibility rate based on an in-vehicle system. First, it utilizes an in-vehicle multi-source sensor to collect multi-source data within a target area and preprocesses the multi-source data. Then, it uses a pre-trained two-dimensional green visibility rate model to process the preprocessed two-dimensional image, obtaining the two-dimensional green visibility rate and a green visibility segmentation map. The two-dimensional green visibility rate model includes a standard U-Net model, inserting a channel-spatial dual attention module after each downsampling stage of the encoder in the standard U-Net model, setting a multi-scale pyramid pooling layer at the skip connections of the decoder in the standard U-Net model, and adding an auxiliary segmentation head at each upsampling stage of the decoder in the standard U-Net model to calculate intermediate losses. During processing, the two-dimensional green visibility rate model uses a two-dimensional mask map of the vegetation segmentation results corresponding to the three-dimensional point cloud as prior knowledge and uses a gating mechanism to suppress misjudged regions during the two-dimensional image processing. The standard U-Net model... A hollow space pyramid pooling layer is set in the deep encoder; then, the green view segmentation map is processed for edge smoothing and region consistency using an energy function to obtain a two-dimensional green view segmentation map; then, the preprocessed lidar point cloud is segmented into ground to obtain a non-ground point cloud; density clustering is performed on the non-ground point cloud to obtain a vegetation point cloud; vegetation category segmentation is performed based on the reflection intensity, height, and density of the vegetation point cloud; based on the vegetation point cloud and human eye perspective features, the vegetation volume ratio is determined and used as the three-dimensional green view rate; a three-dimensional green view rate heatmap is generated based on the three-dimensional green view rate, and the proportion information of different vegetation categories is generated according to the results of vegetation category segmentation; finally, the two-dimensional weights corresponding to the two-dimensional green view rate are determined according to the vegetation occlusion rate and the illumination normalization value; the three-dimensional weights corresponding to the three-dimensional green view rate are determined according to the coverage rate; the fused green view rate is determined based on the two-dimensional green view rate, two-dimensional weights, three-dimensional green view rate, and three-dimensional weights; and a fused green view rate heatmap is generated based on the fused green view rate.This disclosed solution acquires multi-source data, including 2D and 3D data. In predicting 2D green visibility rate using an improved 2D green visibility rate model, it enhances the model's sensitivity to vegetation through channel-spatial dual attention and pyramid pooling. A combination of intermediate loss and model-specific loss accelerates convergence and improves segmentation edge accuracy. 3D data is used to suppress 2D segmentation misjudgments and improve robustness. A lightweight decoder design balances accuracy and efficiency, adapting to vehicle-mounted edge computing. An energy function further refines the segmentation results and reduces noise. This disclosed solution preprocesses the acquired multi-source data, achieving μs-level time synchronization and millimeter-level spatial registration to ensure data consistency. This disclosed solution uses LiDAR point clouds for 3D green visibility rate prediction and achieves complementary optimization of 2D imagery and 3D point clouds through a dynamic weight model and semantic verification, effectively improving prediction accuracy. The combination of multi-source data in this disclosed solution effectively overcomes the shortcomings of existing technologies where monitoring results are easily affected by lighting, shadows, and occlusion, improving monitoring accuracy.

[0063] In summary, this disclosure improves prediction accuracy through a multimodal data fusion architecture, a dual-channel attention-enhanced U-Net model, 3D point cloud-assisted 2D segmentation optimization, lightweight decoder design, and dynamic weight fusion algorithm. It effectively overcomes the shortcomings of existing technologies where monitoring results are easily affected by lighting, shadows, and occlusion, thereby improving the accuracy of monitoring.

[0064] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0065] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0066] Figure 1 This is a flowchart of the method for determining green visibility rate based on an in-vehicle system according to this disclosure;

[0067] Figure 2 This is a schematic diagram of the structure of the system for determining the green visibility rate of the vehicle-mounted system based on this disclosure. Detailed Implementation

[0068] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0069] This disclosure addresses the shortcomings of current green view rate monitoring schemes, which suffer from drawbacks such as susceptibility to light, shadow, and occlusion, large data processing volume, high requirements for equipment computing power, low efficiency, high cost, and large monitoring equipment size. It provides a method and system for determining green view rate based on a vehicle-mounted system. This disclosure utilizes dynamic data acquisition, distributed computing, and multi-source data fusion technologies to achieve efficient and high-precision measurement and analysis of urban road green view rate, providing data support for urban planning, ecological assessment, and public health.

[0070] The technical solution of this disclosure will be described below through specific embodiments.

[0071] like Figure 1 The diagram shown is a flowchart of the green visibility rate determination method based on the vehicle-mounted system in this embodiment. The execution subject of this embodiment is a computing device, component, or system with data processing capabilities. Specifically, the method of this embodiment may include the following steps:

[0072] S110. Collect multi-source data within the target area using an on-board multi-source sensor, wherein the multi-source data includes a two-dimensional image of the target area, a lidar point cloud, and pose information of the on-board system; the on-board system includes the on-board multi-source sensor.

[0073] The vehicle-mounted system includes a LiDAR, a panoramic camera, a high-precision positioning device, a multi-layer PCB backplane, a bus, and a carbon fiber nested bracket; the LiDAR and panoramic camera are coaxially nested on the carbon fiber nested bracket. The vehicle-mounted system also includes a honeycomb silicone shock-absorbing bracket; the LiDAR is mounted on this bracket. The vehicle-mounted system integrates a TEC semiconductor cooling chip and a vortex fan. The multi-layer PCB backplane integrates the high-precision positioning device, LiDAR, panoramic camera, and bus. The vehicle-mounted system also includes a housing and air-cooled heat sinks mounted on the housing.

[0074] The high-precision positioning equipment includes GNSS and IMU.

[0075] S120. Preprocess the multi-source data; wherein the preprocessing includes noise removal and spatiotemporal alignment.

[0076] The above preprocessing may specifically include the following steps:

[0077] Using GNSS UTC time as a reference, the clocks of the lidar, panoramic camera, and IMU are synchronized via the PTP protocol; the angular velocity of the IMU and the position of the GNSS are fused using federated filtering to correct the time deviation of the multi-source data in real time; the lidar point cloud and GNSS data are spatially registered using an affine transformation matrix; and the viewpoint offset of the multi-source data caused by vehicle vibration is dynamically corrected using the attitude angle data of the IMU.

[0078] S130. The pre-trained two-dimensional green visibility model is used to process the pre-processed two-dimensional image to obtain the two-dimensional green visibility and green visibility segmentation map. The two-dimensional green visibility model includes a standard U-Net model, a channel-spatial dual attention module inserted after each downsampling stage of the encoder in the standard U-Net model, a multi-scale pyramid pooling layer set at the skip connections of the decoder in the standard U-Net model, and an auxiliary segmentation head added to each upsampling stage of the decoder in the standard U-Net model to calculate intermediate loss. During processing, the two-dimensional green visibility model uses a two-dimensional mask map of the vegetation segmentation results corresponding to the LiDAR point cloud as prior knowledge and uses a gating mechanism to suppress misjudged regions during the two-dimensional image processing. A hollow spatial pyramid pooling layer is set in the deep encoder of the standard U-Net model. The two-dimensional green visibility model is a dual-channel attention-enhanced U-Net model; the encoder in the two-dimensional green visibility model is a lightweight encoder.

[0079] The intermediate loss function corresponding to the intermediate loss is:

[0080] ;

[0081] Among them, P i Let λ be the prediction result for the i-th decoding stage. i is the weight coefficient, and DiceLoss represents the loss function used to measure the overlap between two samples. The value of DiceLoss ranges from [0, 1]. The smaller the value, the closer the predicted result is to the true label.

[0082] The loss function of the two-dimensional green visibility model includes a main loss function and several intermediate loss functions; the weights of the main loss function and each intermediate loss function are adjusted during the training of the two-dimensional green visibility model.

[0083] The encoder in the standard U-Net model is the lightweight MobileNetV3-Small.

[0084] S140. The green view segmentation map is processed by the energy function to improve edge smoothness and regional consistency, thereby obtaining a two-dimensional green view segmentation map.

[0085] S150. Perform ground segmentation on the preprocessed lidar point cloud to obtain non-ground point cloud; perform density clustering on the non-ground point cloud to obtain vegetation point cloud; perform vegetation category segmentation based on the reflection intensity, height, and density of the vegetation point cloud; determine the vegetation volume ratio based on the vegetation point cloud and human eye perspective features, and use the vegetation volume ratio as the three-dimensional green view rate; generate a three-dimensional green view rate heatmap based on the three-dimensional green view rate, and generate the proportion information of different vegetation categories according to the results of vegetation category segmentation.

[0086] The above determination of vegetation volume proportion based on vegetation point cloud and human eye view features can specifically include:

[0087] The emitted rays from the human eye are determined based on the characteristics of the human eye's visual field; the vegetation volume ratio is determined using the following formula based on the emitted rays from the human eye and the vegetation point cloud:

[0088] ;

[0089] Among them, GVI 3D The three-dimensional green visibility rate represents the proportion of vegetation volume within the area visible to the human eye; V visible-green The volume of vegetation visible from a human eye's perspective; V total_view The volume visible from the human eye, including both vegetated and non-vegetated areas.

[0090] S160. Determine the two-dimensional weights corresponding to the two-dimensional green view rate based on the vegetation shading rate and the normalized value of light; determine the three-dimensional weights corresponding to the three-dimensional green view rate based on the coverage rate; determine the fused green view rate based on the two-dimensional green view rate, the two-dimensional weights, the three-dimensional green view rate, and the three-dimensional weights; generate a fused green view rate heatmap based on the fused green view rate.

[0091] In some embodiments, the green visibility rate determination method is executed using a distributed computing framework; wherein the distributed computing framework includes a cloud and multiple computing nodes; the cloud divides the data to be processed into multiple data blocks according to density; and generates data processing tasks and schedules them; the data processing tasks include computationally intensive tasks and lightweight tasks.

[0092] The steps for task scheduling in the cloud are as follows:

[0093] Step 1: Determine the load score for each compute node.

[0094] Step 2: Assign computationally intensive tasks to computing nodes whose load scores are less than the first threshold.

[0095] Step 3: Assign lightweight tasks to computing nodes whose load scores are greater than or equal to the first threshold and less than the second threshold.

[0096] Computing nodes with a load score greater than or equal to the second threshold are not assigned data processing tasks.

[0097] The load score of the computing node is determined according to any of the following methods:

[0098] Based on the task timeout data of the computing node, determine the first weight corresponding to GPU utilization and the second weight corresponding to network bandwidth; based on the CPU utilization, GPU utilization, network bandwidth, video memory utilization, first weight, second weight, third weight corresponding to CPU utilization, and fourth weight corresponding to video memory utilization of the computing node, determine the load score of the computing node.

[0099] or,

[0100] Input the CPU utilization, GPU utilization, network bandwidth, and video memory usage of the computing node within a preset historical time period into a pre-trained LSTM model to obtain the load score of the computing node in the future preset time period.

[0101] The cloud-based task scheduling steps also include:

[0102] A computing node with a load score less than a third threshold sends a task preemption request to the cloud; the third threshold is less than the first threshold; based on the task preemption request, the cloud allocates data processing tasks of computing nodes with a load score greater than a fourth threshold to computing nodes with a load score less than the third threshold; wherein, the fourth threshold is greater than the first threshold and less than the second threshold.

[0103] The cloud-based task scheduling steps also include:

[0104] Determine the scheduling priority of each data processing task; use a RabbitMQ-based queue to store each data processing task, where RabbitMQ adjusts the position of the data processing task in the queue according to the scheduling priority.

[0105] In some embodiments, the green visibility rate determination method further includes:

[0106] The cloud receives task execution result data from each computing node performing data processing tasks on data blocks; wherein each data block corresponds to a geographical area; the received task execution result data is converted into a preset data format and coordinate alignment is performed; for task execution result data of data blocks without overlapping geographical areas, they are stitched together according to the geographical area; for task execution result data of data blocks with overlapping geographical areas, the stitching weight of each data block is determined according to the confidence level of each data block in the overlapping area, and the task execution result data corresponding to each data block are stitched together using the stitching weights of each data block; an initial fused green view rate heatmap is generated based on the stitching result; outlier correction, spatial interpolation, and multi-resolution fused green view rate heatmaps are pre-generated using a pyramid model on the initial fused green view rate heatmap.

[0107] This includes, in cases where the splicing results determine that the task execution result data corresponding to the missing data blocks within a geographical range is missing, the following:

[0108] The process involves acquiring data blocks that are missing from the geographic range, assigning computing nodes to process them, receiving task execution result data from the computing nodes, and returning the steps described above: converting the received task execution result data into a preset data format and performing coordinate alignment.

[0109] In some embodiments, the vehicle-mounted multi-source sensor integrated design includes a lidar for acquiring lidar point clouds, which can be an XT32M with the following performance parameters: ranging distance of 300 meters, point frequency of 64W points / second, wavelength of 905nm, and support for rain and fog penetration; its anti-vibration design specifically includes the use of a silicone shock-absorbing bracket with a vibration error suppression rate of ≥90%.

[0110] The panoramic camera used to acquire 2D images is the Ladybug5+, with the following optical parameters: 6-lens array, 150° horizontal field of view, 120° vertical field of view, and 350dpi resolution (18MB per image). Its triggering mechanism is DMI (Distance Measurement Instrument) distance triggering, with automatic shooting at 50-meter intervals.

[0111] The GNSS module uses BeiDou-3 + RTK positioning, with a horizontal accuracy of ≤2 meters and an elevation accuracy of ≤3 meters. The IMU module is a 6-axis inertial measurement unit (3-axis accelerometer + 3-axis gyroscope), with a sampling frequency of 200Hz and an attitude angle error of ≤0.1°.

[0112] The compute nodes are powered by NVIDIA Jetson AGX Xavier processors, supporting CUDA parallel computing with a power consumption of 30W. The storage modules are 1TB NVMe SSDs, supporting data caching and real-time uploads.

[0113] The LiDAR (XT32M) and panoramic camera (Ladybug5+) utilize a carbon fiber nested bracket with integrated micro-heat dissipation fins. The overall dimensions are Φ180mm×120mm, resulting in a 45% weight reduction. The coaxial nesting achieves a compact layout. This solution utilizes honeycomb silicone for vibration damping, achieving a vibration transfer function attenuation of ≥25dB ​​in the 1-100Hz frequency band (93% vibration suppression rate measured in a vehicle at 60km / h). This disclosure employs dual-channel active cooling, integrating a TEC semiconductor cooler and a vortex fan, extending the operating temperature range to -30°C to 70°C, with a temperature rise of ≤15°C after 4 hours of continuous operation.

[0114] This disclosure presents a bus optimization scheme, integrating GNSS / IMU (CAN bus), LiDAR (LVDS interface), and camera (GigE Vision) on a 6-layer PCB backplane, reducing cable length by 60% and signal delay ≤1μs. It implements wide-voltage power management, supporting 9-36V input with a peak power consumption of 120W, and is equipped with a supercapacitor buffer module (10F capacity) to handle instantaneous fluctuations in vehicle power (±10V). The disclosure utilizes an aluminum alloy casing and air-cooled heat sink for heat dissipation, achieving an operating temperature range of -20°C to 60°C.

[0115] The technical solution disclosed herein implements a multi-source spatiotemporal registration algorithm:

[0116] For time synchronization, based on the IEEE 1588 PTP protocol, μs-level synchronization is achieved through FPGA hardware timestamps. Combined with federated Kalman filtering (predicting IMU angular velocity and GNSS position), the time residual is ≤0.3ms.

[0117] For spatial registration, an improved ICP (Iterative Closest Point) algorithm is adopted, which introduces normal vector constraints and dynamic weights (point cloud curvature weight accounts for 70%), improving the registration accuracy to 0.02 meters.

[0118] This disclosed solution utilizes a distributed computing framework, enabling dynamic task scheduling. Specifically: adaptive block partitioning is implemented based on point cloud density (5GB / block in sparse areas, 3GB / block in dense areas) and node load (allocation is paused when GPU memory utilization is >80%), with a response time ≤5ms. A priority strategy sorts the task queue by "data freshness" (collection time difference), prioritizing data collected within the last 10 minutes to ensure real-time performance. GPU acceleration optimization uses CUDA parallel kernel functions for point cloud denoising (statistical filtering), increasing single-node processing speed by 4 times (tested processing time for 30GB of data decreased from 2170 seconds to 540 seconds). Data compression and transmission use LASzip+entropy encoding, increasing compression ratio to 85% and reducing bandwidth usage by 75%.

[0119] This disclosure utilizes a two-dimensional green visibility model to process two-dimensional images and achieve semantic segmentation. The two-dimensional green visibility model can be an improved U-Net model structure, which will be described in detail below.

[0120] 1. Basic U-Net Architecture The standard U-Net consists of a symmetrical encoder (downsampling) and decoder (upsampling), fusing shallow and deep features through skip connections. Its core advantage lies in preserving multi-scale spatial information, making it suitable for high-precision semantic segmentation tasks.

[0121] 2. Areas for improvement The following innovative designs were implemented based on the standard U-Net to improve the accuracy and efficiency of green visibility calculation: (1) Multi-Scale Attention Fusion (MSAF) module is used to enhance the model’s ability to pay attention to vegetation areas and fuse multi-scale features.

[0122] Improvements to the encoder in the multi-scale attention fusion module: Insert a channel-space dual attention module (CSDA) after each downsampling stage of the encoder.

[0123] Channel attention: Channel weights are dynamically adjusted via the SE (Squeeze-and-Excitation) module.

[0124] Spatial attention: Generate weight masks based on the spatial location of feature maps to highlight vegetated areas.

[0125] Improvements to the decoder in the multi-scale attention fusion module: Multi-scale pyramid pooling (MSPP) is introduced at the skip connections to fuse contextual information at different scales.

[0126] The pyramid pooling layer contains four pooling kernels of different sizes (1×1, 3×3, 6×6, and global average), and the outputs are concatenated and fused with the feature map of the decoder.

[0127] (2) Deep supervision and dynamic loss function

[0128] Deep supervision involves adding an auxiliary segmentation head at each upsampling stage of the decoder to calculate the intermediate loss.

[0129] Loss function for intermediate loss: ;

[0130] Among them, P i Let λ be the prediction result for the i-th decoding stage. i These are the weighting coefficients (λ1=0.1, λ2=0.2, λ3=0.3, λ4=0.4).

[0131] The scheme disclosed herein can achieve dynamic loss adjustment. Specifically, it automatically adjusts the weights of the main loss function (cross-entropy loss) and the intermediate loss function according to the training phase. In the early stage, it focuses on the auxiliary loss to stabilize training, and in the later stage, it focuses on the main loss to improve accuracy.

[0132] (3) Cross-modal feature fusion (combining point cloud data) is used to optimize the segmentation results of two-dimensional images using three-dimensional point cloud data.

[0133] The specific implementation involves projecting the point cloud classification results (vegetation / non-vegetation) onto a two-dimensional image coordinate system to generate a mask map. Cross-modal attention gating: In skip connections, the point cloud mask map serves as prior knowledge, and a gating mechanism is used to suppress misclassified regions in the image (such as shadows and glass reflections).

[0134]

[0135] Where σ is the Sigmoid function and ⊕ represents the concatenation operation.

[0136] (4) Lightweight encoder design

[0137] Encoder replacement: Replace the standard U-Net encoder (such as VGG16) with MobileNetV3-Small to reduce the number of parameters (from 138M to 12M) while maintaining accuracy.

[0138] Hollow convolutional alternative: Hollow spatial pyramid pooling (ASPP) is employed in deep encoders to expand the receptive field and capture a wider range of vegetation distribution.

[0139] (5) Post-processing optimization

[0140] Conditional Random Fields (CRFs) refine the segmentation results output by the model by optimizing edge smoothness and region consistency through an energy function.

[0141] Energy function:

[0142] Where, ψ u For the Unary term (model prediction probability), ψ p For Pairwise terms (based on similarity constraints between color and spatial distance).

[0143] In this embodiment, the core data is a two-dimensional image, sourced from a vehicle-mounted panoramic camera (such as Ladybug5+) capturing images of the target area. The result is: a green view segmentation map (vegetation area mask; two-dimensional green view ratio value (percentage of green pixels).

[0144] The auxiliary data consists of 3D point cloud segmentation results, sourced from point cloud data collected by LiDAR (such as XT32M). The results are: 3D model of vegetation point cloud; vegetation category segmentation results (trees, shrubs, grasslands); 2D mask map: the 3D vegetation point cloud is projected onto a 2D image coordinate system to generate a mask of the vegetation area.

[0145] Other multi-source data includes pose information (GNSS / IMU): used for spatiotemporal alignment to ensure consistency between 2D images and 3D point clouds in time and space (μs-level time synchronization + millimeter-level spatial registration); illumination normalization value: through analysis of illumination conditions in 2D images, the threshold for determining green pixels is dynamically adjusted (HSV spatial adaptation).

[0146] 3. Performance Validation of the Two-Dimensional Green Visibility Model

[0147] Dataset: 100,000 labeled images (including various scenes such as urban roads, parks, and buildings), divided in an 8:1:1 ratio (training:validation:test).

[0148] Training configuration: Optimizer: AdamW (initial learning rate 3e-4, cosine annealing scheduling).

[0149] Data augmentation: random rotation (±30°), brightness jitter (±20%), shadow simulation.

[0150] Test results: mIoU: 92.1% (6.5% improvement over standard U-Net).

[0151] Inference speed: 23ms per frame (1080p resolution, NVIDIA Jetson AGX Xavier).

[0152] Cross-scenario generalization: mIoU reaches 89.3% on suburban road data that was not used in the training.

[0153] Our solution achieves multi-scale attention fusion: enhancing the model's sensitivity to vegetation through channel-spatial dual attention and pyramid pooling. It implements deep supervision and dynamic loss: accelerating convergence and improving segmentation edge accuracy. It achieves cross-modal fusion: utilizing point cloud data to suppress 2D segmentation misjudgments and improve robustness. It achieves lightweight design: MobileNetV3 + ASPP balances accuracy and efficiency, adapting to vehicle-mounted edge computing. Furthermore, it implements post-processing optimization: CRF further refines the segmentation results and reduces noise.

[0154] Dynamic threshold optimization:

[0155] The threshold for determining green pixels is adaptively adjusted based on the light intensity (H∈[35,85], S≥40, V≥20 in HSV space).

[0156] formula: ;

[0157] In this disclosure, the point cloud data from LiDAR is mainly used in the following three aspects: 3D modeling: Constructing 3D models of roads and vegetation using high-density point clouds. Vegetation classification: Differentiating trees, shrubs, and grasslands based on features such as point cloud density and reflection intensity. Viewpoint simulation: Calculating the percentage of visible vegetation volume by simulating the human eye's viewpoint (84° horizontal × 62° vertical, 1.7 meters above the ground).

[0158] The specific processing steps are as follows: 1) Noise reduction and preprocessing:

[0159] Use statistical filtering (Statistical Outlier Removal) to remove outliers (points with a distance ≥ 2σ from the mean).

[0160] 2) Ground segmentation: Separate ground and non-ground point clouds based on the RANSAC algorithm.

[0161] 3) Point cloud classification:

[0162] Density Clustering (DBSCAN): Separates vegetation from non-vegetation (such as buildings and vehicles).

[0163] Random forest classification: vegetation categories (trees, shrubs, grasslands) are refined based on reflectance intensity, height, and density features, with a classification accuracy of F1-score ≥ 89%.

[0164] 4) Calculation of green view ratio:

[0165] Ray casting algorithm: Emit rays from the simulated human eye position to calculate the proportion of visible vegetation volume.

[0166] formula: ;

[0167] The processing result is: outputting a three-dimensional green view rate heat map, and marking the vertical structure of vegetation (e.g., the tree layer accounts for 60% and the shrub layer accounts for 25%).

[0168] For example, if the output result shows that the three-dimensional green view rate of section A is 58.7%, it is recommended to replant trees to improve the vertical coverage.

[0169] In this disclosure, the data from high-precision positioning equipment (GNSS / IMU) is mainly used for the following aspects: Spatiotemporal reference: providing high-precision position (longitude, latitude, elevation) and attitude (pitch angle, roll angle, heading angle). Sensor synchronization: unifying the timestamps and spatial coordinate systems of multi-source data.

[0170] The processing steps are as follows: 1) Time synchronization:

[0171] Using GNSS UTC time as a reference, the LiDAR, camera, and IMU clocks are synchronized via PTP protocol with a synchronization error of ≤0.5ms.

[0172] Federal filtering fuses IMU angular velocity (200Hz) and GNSS position (1Hz) to correct time deviation in real time.

[0173] 2) Spatial registration:

[0174] Coordinate system alignment: The lidar point cloud (local coordinate system) is registered with the GNSS data (WGS84 coordinate system) through an affine transformation matrix.

[0175] Attitude compensation: Dynamically correct the viewing angle shift caused by vehicle vibration using IMU attitude angle data, with a residual of ≤0.03 meters.

[0176] The processing result is a multi-source dataset with accurate spatiotemporal labels (e.g., point cloud coordinate error ≤ 0.05 meters).

[0177] For example, after data registration for section B, the spatiotemporal consistency error was reduced to 1.2cm.

[0178] In this disclosure, panoramic camera image data is mainly used for the following aspects: Two-dimensional green visibility calculation: extracting the proportion of green pixels through semantic segmentation. Dynamic threshold optimization: adaptively adjusting the green detection range according to lighting conditions.

[0179] The processing steps are as follows: 1) Image preprocessing:

[0180] Distortion correction: Correct fisheye distortion based on camera calibration parameters (intrinsic and extrinsic parameters).

[0181] Light equalization: The CLAHE algorithm is used to enhance the contrast of shadow areas.

[0182] 2) Semantic segmentation:

[0183] Improved U-Net model: Incorporates channel attention mechanism, training set contains 100,000 labeled images, mIoU reaches 92%.

[0184] Dynamic threshold adjustment: In HSV space, the green pixel determination range is adaptive (H∈35,85, S≥40, V≥20).

[0185] 3) Calculation of green view ratio:

[0186] formula: ;

[0187] The processing result is: output a two-dimensional green view rate distribution map, and mark the road sections with low green view rate (such as the section from K3+200 to K3+500 of Wenyu River, which has only 32.5%).

[0188] Example: The error between the 2D calculation result and the manual annotation is ≤3.5%.

[0189] After obtaining the two-dimensional and three-dimensional green view rates, they are fused. The steps are as follows:

[0190] 1. Data Fusion Process

[0191] Spatiotemporal alignment:

[0192] The pixel coordinates of the 2D image (via camera intrinsics) are mapped to the 3D point cloud coordinate system to establish the pixel-point cloud correspondence.

[0193] Dynamically compensate for viewpoint shifts caused by vehicle vibration using IMU attitude data.

[0194] Dynamic weight allocation, the specific calculation formula is as follows:

[0195] GVI Fusion = W2⋅GVI2D + W3⋅GVI3D

[0196] Wherein: GVI fusion is the fused green view rate, W2 is the two-dimensional weight, GVI2D is the two-dimensional green view rate, W3 is the three-dimensional weight, and GVI3D is the three-dimensional green view rate.

[0197] W2 = 0.6⋅e−0.1⋅Occlusion rate + 0.4⋅Normalized illumination value

[0198] W3 = 0.7⋅F1 - score + 0.3⋅coverage

[0199] Semantic consistency check:

[0200] Artifact removal: If the vegetation is identified in 2D but not in 3D, it is determined to be a light and shadow artifact (such as reflection between leaves), and the 2D weight W2 is set to zero.

[0201] Low-visibility vegetation supplementation: If the 3D classification is vegetation but the 2D classification is not identified, the 3D weight W3 is reduced by 50% by combining lidar reflectivity (>0.3).

[0202] 2. Validation of Fusion Results

[0203] Accuracy comparison: The combined green view rate MAE is 2.8%, which is 49% higher than the single two-dimensional method (MAE=5.5%).

[0204] Case verification: The green view rate of section C is 55.2%, which is closer to the human assessment value (54.5%) than the two-dimensional (58.7%) and three-dimensional (53.1%) values.

[0205] 3. Application Output

[0206] Comprehensive Report: Generates a PDF report that includes a heatmap of integrated green view rate, analysis of vegetation vertical structure, and planting recommendations (such as replanting sycamore trees).

[0207] Decision support: Based on 3D point cloud analysis, it is recommended to increase the tree density to 70%, and the green view rate is expected to increase to 65%.

[0208] The technical solution disclosed herein achieves deep fusion of multi-source data: through a dynamic weighting model and semantic verification, complementary optimization of 2D images and 3D point clouds is realized. The technical solution disclosed herein achieves high-precision spatiotemporal registration: μs-level time synchronization and millimeter-level spatial registration ensure the consistency of multi-source data. The technical solution disclosed herein achieves a compact hardware design: a carbon fiber nested structure and a dual-channel heat dissipation solution, with dimensions ≤Φ180mm×120mm and a vibration suppression rate ≥93%.

[0209] This disclosure utilizes a distributed processing framework to execute the method for determining the green visibility rate, with the following architecture design:

[0210] Master Node: Deployed in the cloud, responsible for task scheduling, data partitioning (5GB per block), and result merging.

[0211] Compute Node (Slave_Node): Deployed on the edge server, single node configuration: Intel Xeon 16 cores, 128GB memory, NVIDIA A100 GPU.

[0212] The task flow is as follows:

[0213] 1) Data partitioning: The master node divides the 30GB point cloud data into 6 blocks (5GB each).

[0214] 2) Dynamic scheduling of tasks:

[0215] Existing dynamic scheduling schemes often rely on a single metric (such as CPU utilization) or static thresholds to allocate tasks, while this scheme first calculates a load score and then schedules tasks based on the load score.

[0216] A load score is obtained by fusing multiple load metrics. This includes considering CPU utilization, GPU utilization, network bandwidth, and memory usage, among others, to construct a normalized weighted scoring model, avoiding bias from a single metric. Furthermore, this disclosure implements dynamic weight adjustment: the weights of each metric are dynamically adjusted based on task type (e.g., compute-intensive, data-intensive tasks) (e.g., increasing GPU utilization weight to 70% in GPU tasks). Adaptive priority thresholds are implemented during scheduling, specifically using fuzzy logic control: the priority threshold is dynamically adjusted based on historical load trends and task completion speed (e.g., priority is given when load is <70%, but the threshold is automatically lowered to 65% if node response is delayed). An elastic preemption mechanism is also implemented: allowing low-load nodes to preempt unfinished tasks from high-load nodes, ensuring task atomicity through distributed locks, and reducing resource idleness. Additionally, this disclosure can also implement predictive scheduling optimization, specifically using LSTM load prediction, i.e., using a Long Short-Term Memory network to predict the load trend of nodes in the next 5 minutes, and pre-allocating tasks to avoid resource bottlenecks. Furthermore, task pre-migration is implemented during task scheduling; that is, if an overloaded node is predicted, some tasks are migrated to low-load nodes in advance, reducing real-time scheduling latency. This disclosure, through multi-dimensional indicator fusion, adaptive threshold adjustment, and predictive optimization, significantly improves cluster resource utilization and task processing efficiency, demonstrating substantial technological innovation.

[0217] Detailed steps of dynamic scheduling

[0218] 21) Real-time monitoring and data acquisition of compute node status

[0219] The monitoring indicators are as follows:

[0220] CPU / GPU utilization: Real-time data collected via Prometheus+Node Exporter, with a sampling frequency of 1 second.

[0221] Network bandwidth: Use iftop to monitor the data transmission rate between nodes and distinguish between uplink and downlink traffic.

[0222] GPU memory usage: Obtain GPU memory usage via the NVIDIA DCGM API.

[0223] Data normalization: Normalize each indicator to the range of 0-1. Formula:

[0224] index i =(current value) i -Minimum value i )÷(maximum value) i -Minimum value i )

[0225] For example, the maximum GPU utilization is set to 100%, and the maximum network bandwidth is set to 10Gbps.

[0226] 22) Comprehensive load score calculation

[0227] Weighted scoring model:

[0228] Load score = w1⋅CPU utilization + w2⋅GPU utilization + w3⋅Network bandwidth + w4⋅VRAM usage

[0229] Initial weight configuration (can be dynamically adjusted):

[0230] w1=0.1, w2=0.6, w3=0.2, w4=0.1

[0231] Dynamic weight adjustment: If a computing node experiences three consecutive task timeouts, the weight corresponding to GPU utilization will be automatically increased to 0.7, and the weight corresponding to network bandwidth will be decreased to 0.1.

[0232] 23) Prioritization strategy and task allocation

[0233] Priority determination:

[0234] High priority (load score < 0.7): Prioritize the allocation of computationally intensive tasks (such as point cloud classification and semantic segmentation).

[0235] Medium priority (0.7 ≤ load score < 0.9): Assign lightweight tasks (such as data denoising and format conversion).

[0236] Low priority (load score ≥ 0.9): Pause task assignment until the load decreases.

[0237] In addition, the task partitioning strategy in this disclosure is as follows:

[0238] Density-adaptive block partitioning: When the point cloud density is >1000 points / ㎡, the block size is 3GB; the sparse region block size is 5GB.

[0239] Task labeling: Label the task type (such as "vegetation classification" or "heat map generation") and match it with the node hardware characteristics (such as prioritizing deep learning tasks for A100 GPUs).

[0240] In addition, the predictive scheduling and elastic preemption steps in this disclosure are as follows:

[0241] Load trend forecasting:

[0242] LSTM model input: the load sequence of the past 10 minutes (CPU utilization, GPU utilization, network bandwidth, and video memory usage).

[0243] Output: Predicted load score for the next 5 minutes. If the predicted load score is >0.8, trigger task pre-migration.

[0244] Flexible preemption mechanism:

[0245] Preemption conditions: Low-load nodes (load score < 0.6) can apply to preempt tasks from high-load nodes (load score > 0.8).

[0246] Atomicity guarantee: Redis distributed locks ensure the consistency of task states, and preempted tasks are re-added to the queue.

[0247] This disclosure implements real-time scheduling and fault tolerance, as detailed below:

[0248] Task queue management uses RabbitMQ to implement a multi-level priority queue, with high-priority tasks inserted at the head of the queue. Tasks that time out (default 300 seconds) will be automatically retried, and will be marked as failed after a maximum of 3 attempts.

[0249] Node status synchronization is achieved through the gRPC protocol, with a latency of ≤10ms.

[0250] When a node goes offline, its unfinished tasks are redistributed to other nodes by the master node.

[0251] The performance optimization and disaster recovery implementation methods in this disclosure are as follows:

[0252] Data locality optimization prioritizes assigning tasks to nodes that store relevant data, reducing network transmission overhead.

[0253] Disaster recovery backup: Critical tasks (such as green visibility calculation) are backed up in real time to at least two nodes to ensure rapid recovery in the event of a single point of failure.

[0254] The actual performance test and comparison results of the disclosed solution are as follows:

[0255] Test environment:

[0256] Cluster size: 10 nodes (Intel Xeon 16 cores + NVIDIA A100 GPU).

[0257] Dataset: 30GB point cloud + 2000 images.

[0258] Result comparison:

[0259] Speedup: The dynamic scheduling strategy reduced task time from 2170 seconds (static chunking) to 890 seconds, a speedup of 7.1 times.

[0260] Resource utilization: Average GPU utilization increased from 65% to 85%, and network bandwidth utilization increased from 40% to 70%.

[0261] Fault tolerance: Task recovery time is ≤5 seconds in case of node failure, with zero data loss.

[0262] The technical solutions disclosed herein achieve multi-dimensional load fusion: breaking through the limitations of single indicators and accurately assessing node status. The technical solutions disclosed herein achieve adaptive threshold adjustment: dynamically optimizing priority strategies through fuzzy logic. The technical solutions disclosed herein achieve predictive scheduling: LSTM predicts load trends, avoiding resource bottlenecks. The technical solutions disclosed herein achieve elastic preemption and atomicity guarantees: improving resource utilization while ensuring task consistency.

[0263] This disclosure utilizes distributed computing nodes for parallel computation, with each node synchronously performing point cloud denoising (statistical filtering), ground classification (RANSAC), and green view rate calculation. After obtaining the results, they need to be merged. Specifically, the following scheme can be used to achieve result merging.

[0264] 1. Metadata Management and Data Reception

[0265] Data block labeling: When dividing data into blocks, the master node assigns a unique ID to each data block and records its geographical range (such as latitude and longitude bounding box or projected coordinate range).

[0266] Result reception: After each compute node completes its task, it packages the results into the following format and sends them to the master node:

[0267] Package code

[0268] {

[0269] "block_id": "B001",

[0270] "geobbox": [116.301°E, 39.905°N, 116.305°E, 39.910°N], / / Geographic boundary

[0271] "gvi_data": "gvi_block_001.tif", / / Green visibility raster file

[0272] "metadata": {"resolution": "0.5m", "timestamp": "2024-09-16T14:30:00Z"}

[0273] }

[0274] 2. Data preprocessing and format standardization

[0275] Format conversion: If the data returned by the node is in an inconsistent format (e.g., some are GeoTIFF and some are NumPy arrays), the master node will convert it to the standard raster format (GeoTIFF).

[0276] Coordinate alignment: Use the GDAL library to reproject the raster data (e.g., to unify it to the WGS84 coordinate system) to ensure consistent spatial reference.

[0277] 3. Spatial stitching and data fusion

[0278] The grid stitching algorithm is as follows:

[0279] Non-overlapping blocks: If the geographical range of data blocks is strictly continuous and non-overlapping, they are directly concatenated by rows and columns.

[0280] Overlapping regions are segmented: a weighted average is applied to the overlapping regions, with weights allocated based on data confidence (e.g., point cloud density). Formula: ;

[0281] Where wi is the confidence weight of the i-th data block in the overlapping region.

[0282] Edge processing: Smooth the edges of the stitched global grid (e.g., using a Gaussian filter) to eliminate gaps between blocks.

[0283] 4. Global Heatmap Generation

[0284] Color mapping: Define the green visibility color gradient (e.g., 0%-30% is red, 30%-60% is yellow, and 60%-100% is green).

[0285] Visualization rendering: Using MapLibre GL or Python's Matplotlib library, the merged raster data is rendered into a heatmap, and road vector layers are overlaid to enhance readability.

[0286] Interactive features: The web platform supports zooming and clicking to query the green view rate value of a point and displays detailed vegetation classification information.

[0287] 5. Data Post-processing and Optimization

[0288] Outlier correction: For pixels that deviate significantly from their neighborhood values ​​(such as extreme values ​​caused by sensor noise), median filtering is used to replace them.

[0289] Spatial interpolation: If there are missing data regions (such as node failures that do not return results), Kriging interpolation is used to estimate the missing values.

[0290] Performance optimization: Pyramid Tiling is used to pre-generate multi-resolution heatmaps to improve web loading speed.

[0291] 6. Fault tolerance and dynamic patching

[0292] Node failure handling: If a node times out (default 300 seconds), the master node will add the data block back to the task queue and distribute it to other idle nodes.

[0293] Use Redis to record task status and avoid duplicate assignment.

[0294] Data integrity verification: After the merge is completed, check whether the global raster covers all geographic areas. If there are any missing areas, trigger the patching process.

[0295] 7. Case Study (Section D)

[0296] Input data:

[0297] Six nodes process 30GB of point cloud data and return six green visibility raster blocks.

[0298] Merging process:

[0299] The points are stitched together according to geographical boundaries, and a confidence-weighted average is taken for overlapping areas (the weights are based on point cloud density).

[0300] Gaussian filtering (σ=1.5) smooths edges.

[0301] Output result:

[0302] The overall heat map shows an average green visibility rate of 58.7%, with the lowest rate (K3+200 to K3+500) at only 32.5%.

[0303] The web platform supports interactive queries with a response time of less than 200ms.

[0304] The technical solution disclosed herein implements dynamic weighted splicing: merging overlapping regions based on confidence level improves result accuracy. The technical solution disclosed herein implements a multi-resolution pyramid: optimizing the performance of large-scale data visualization. The technical solution disclosed herein implements fault tolerance and automatic patching: ensuring data integrity and adapting to the uncertainties of distributed environments.

[0305] The performance test results of this disclosure are as follows:

[0306] Test environment: 6-node cluster vs. single machine (i7-11700K, RTX 3050).

[0307] Result: The processing time for 30GB of data was reduced from 6354 seconds on a single machine to 2170 seconds, a speedup of 2.93 times.

[0308] The data management and application in the technical solution disclosed herein are as follows:

[0309] Metadata specifications:

[0310] Field definitions: latitude and longitude (WGS84), timestamp (ISO 8601), image number (UUID), device ID.

[0311] Storage formats: point cloud (LAS 1.4), image (GeoTIFF), metadata (JSON).

[0312] Real-time analysis function:

[0313] Visualization platform: The web version supports heat maps of green visibility and vertical structure profiles of vegetation.

[0314] Report generation: Automatically export PDF reports, including green view rate ranking, noise correlation analysis, and planting recommendations (such as the recommendation to replant sycamore trees in the Wenyu River case).

[0315] The hardware deployment steps are as follows:

[0316] Sensor installation: The lidar and camera are coaxially fixed to the roof bracket, the GNSS antenna is placed in the center of the roof, and the IMU is close to the vehicle's center of gravity. A laser rangefinder is used to calibrate the relative positions of each sensor (error ≤ 1cm).

[0317] Electrical connections: Powered by 12V vehicle power supply, CAN bus for IMU data transmission, and Gigabit Ethernet for edge computing unit connection.

[0318] Software configuration process

[0319] System initialization: Configure GNSS base station coordinates (such as Beijing Municipal Bureau of Surveying and Mapping benchmark station), and load the pre-trained U-Net model (ONNX format).

[0320] Distributed cluster setup: Use Kubernetes to manage compute nodes and set up task queues (RabbitMQ) and data caches (Redis).

[0321] Data collection and processing example (Section E)

[0322] Data collection parameters:

[0323] Mileage: 20km round trip, vehicle speed ≤60km / h, fixed distance trigger interval 50 meters.

[0324] Data volume: 30GB point cloud, 2000 images (approximately 36GB).

[0325] Processing result:

[0326] Two-dimensional GVI: The average green view rate is 58.7%, and the lowest section (K3+200 to K3+500) is only 32.5% (requiring replanting).

[0327] 3D GVI: Based on point cloud analysis, it is recommended to increase the density of the tree layer to improve the vertical green visibility.

[0328] Based on the same inventive concept, this disclosure provides a green visibility rate determination system based on an in-vehicle system. The steps performed by the components of this system are the same as or similar to those described above, therefore, similar details will not be repeated. Figure 2 As shown, the green visibility rate determination system based on the vehicle-mounted system in this embodiment includes:

[0329] The vehicle system 201 in any of the above embodiments;

[0330] Cloud 202 in any of the above embodiments;

[0331] At least one computing node 203 in any of the above embodiments.

[0332] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0333] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0334] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0335] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0336] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0337] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0338] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0339] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for determining green visibility rate based on a vehicle-mounted system, characterized in that, include: Multi-source data within a target area is collected using an onboard multi-source sensor. The multi-source data includes a two-dimensional image of the target area, a lidar point cloud, and pose information from the onboard system. The onboard system includes the onboard multi-source sensor. The multi-source data is preprocessed; wherein, the preprocessing includes noise removal and spatiotemporal alignment. A pre-trained 2D green visibility model is used to process pre-processed 2D images to obtain 2D green visibility and green visibility segmentation maps. The 2D green visibility model includes a standard U-Net model, a channel-spatial dual attention module inserted after each downsampling stage of the encoder in the standard U-Net model, a multi-scale pyramid pooling layer at the skip connections of the decoder in the standard U-Net model, and an auxiliary segmentation head added to each upsampling stage of the decoder in the standard U-Net model to calculate intermediate loss. During processing, the 2D green visibility model uses a 2D mask map of vegetation segmentation results corresponding to the LiDAR point cloud as prior knowledge and uses a gating mechanism to suppress misclassified regions during 2D image processing. The deep encoder in the standard U-Net model includes a hollow spatial pyramid pooling layer. The 2D green visibility model is a dual-channel attention-enhanced U-Net model. The encoder in the 2D green visibility model is a lightweight encoder. The green view segmentation map is processed for edge smoothness and region consistency using an energy function to obtain a two-dimensional green view segmentation map. The preprocessed lidar point cloud is segmented to obtain a non-ground point cloud; density clustering is performed on the non-ground point cloud to obtain a vegetation point cloud; vegetation categories are segmented based on the reflection intensity, height, and density of the vegetation point cloud; the vegetation volume ratio is determined based on the vegetation point cloud and human eye view features, and the vegetation volume ratio is used as the three-dimensional green view rate; a three-dimensional green view rate heatmap is generated based on the three-dimensional green view rate, and the proportion information of different vegetation categories is generated according to the results of vegetation category segmentation. The two-dimensional weights corresponding to the two-dimensional green view rate are determined based on the vegetation shading rate and the normalized value of light. The three-dimensional weights corresponding to the three-dimensional green view rate are determined based on the coverage rate. The fused green view rate is determined based on the two-dimensional green view rate, the two-dimensional weights, the three-dimensional green view rate, and the three-dimensional weights. A fused green view rate heatmap is generated based on the fused green view rate.

2. The method for determining green visibility rate based on a vehicle-mounted system according to claim 1, characterized in that, The intermediate loss function corresponding to the intermediate loss is: Among them, P i Let λ be the prediction result for the i-th decoding stage. i is the weight coefficient, and DiceLoss represents the loss function used to measure the overlap between two samples. The value of DiceLoss ranges from [0, 1]. The smaller the value, the closer the predicted result is to the true label.

3. The method for determining green visibility rate based on a vehicle-mounted system according to claim 2, characterized in that, The loss function of the two-dimensional green visibility model includes a main loss function and multiple intermediate loss functions; the weights of the main loss function and each intermediate loss function are adjusted during the training of the two-dimensional green visibility model.

4. The method for determining green visibility rate based on a vehicle-mounted system according to claim 1, characterized in that, The encoder in the standard U-Net model is MobileNetV3-Small.

5. The method for determining green visibility rate based on a vehicle-mounted system according to claim 1, characterized in that, The determination of vegetation volume proportion based on vegetation point cloud and human eye view features includes: Determine the rays emitted by the human eye based on the characteristics of the human eye's visual perspective; Based on the emitted rays from the human eye and vegetation point clouds, the vegetation volume percentage is determined using the following formula: Among them, GVI 3D The three-dimensional green visibility rate represents the proportion of vegetation volume within the area visible to the human eye; V visible-green The volume of vegetation visible from a human eye's perspective; V total_view The volume visible from the human eye, including both vegetated and non-vegetated areas.

6. The method for determining green visibility rate based on a vehicle-mounted system according to claim 1, characterized in that, The vehicle-mounted system includes a lidar, a panoramic camera, a high-precision positioning device, a multi-layer PCB backplane, a bus, and a carbon fiber nested bracket; the lidar and panoramic camera are coaxially nested on the carbon fiber nested bracket. The vehicle-mounted system also includes a honeycomb silicone shock-absorbing bracket; the lidar is mounted on the honeycomb silicone shock-absorbing bracket. The vehicle-mounted system integrates a TEC semiconductor cooling chip and a vortex fan, which are attached to the bottom surface of the heat dissipation substrate of the lidar and fixed by a combination of thermally conductive silicone sheet and spring clips. The multi-layer PCB backplane integrates the high-precision positioning device, LiDAR, panoramic camera, and bus. The vehicle system also includes a housing and air-cooled heat sinks mounted on the housing.

7. The method for determining green visibility rate based on a vehicle-mounted system according to claim 6, characterized in that, The high-precision positioning equipment includes GNSS and IMU; The preprocessing of the multi-source data includes: Using GNSS UTC time as a reference, the clocks of the lidar, panoramic camera, and IMU are synchronized via the PTP protocol; By using federated filtering to fuse the angular velocity of the IMU and the position of the GNSS, the time deviation of the multi-source data is corrected in real time. Spatial registration of lidar point cloud and GNSS data is performed using an affine transformation matrix. The attitude angle data from the IMU is used to dynamically correct the viewpoint shift of the multi-source data caused by vehicle vibration.

8. The method for determining green visibility rate based on a vehicle-mounted system according to claim 1, characterized in that, Also includes: The green visibility rate determination method is executed using a distributed computing framework; wherein the distributed computing framework includes a cloud and multiple computing nodes; the cloud divides the data to be processed into multiple data blocks according to density; and generates data processing tasks and schedules them; the data processing tasks include computationally intensive tasks and lightweight tasks. The steps for task scheduling in the cloud are as follows: Determine the load score for each compute node; Assign computationally intensive tasks to computing nodes whose load scores are less than the first threshold; Lightweight tasks are assigned to computing nodes whose load scores are greater than or equal to the first threshold and less than the second threshold. Computing nodes with a load score greater than or equal to the second threshold are not assigned data processing tasks; The load score of the computing node is determined according to any of the following methods: Based on the task timeout data of the computing node, determine the first weight corresponding to GPU utilization and the second weight corresponding to network bandwidth; based on the CPU utilization, GPU utilization, network bandwidth, video memory utilization, first weight, second weight, third weight corresponding to CPU utilization, and fourth weight corresponding to video memory utilization of the computing node, determine the load score of the computing node. or, Input the CPU utilization, GPU utilization, network bandwidth, and video memory usage of the computing node within a preset historical time period into a pre-trained LSTM model to obtain the load score of the computing node in the future preset time period. The cloud-based task scheduling steps also include: Computing nodes whose load scores are less than a third threshold send a task preemption request to the cloud; the third threshold is less than the first threshold. Based on the task preemption request, the cloud allocates data processing tasks of computing nodes with load scores greater than a fourth threshold to computing nodes with load scores less than a third threshold; wherein the fourth threshold is greater than the first threshold and less than the second threshold. The cloud-based task scheduling steps also include: Determine the scheduling priority of each data processing task; Each data processing task is stored in a RabbitMQ-based queue, where RabbitMQ adjusts the position of the data processing task in the queue according to scheduling priority.

9. The method for determining green visibility rate based on a vehicle-mounted system according to claim 8, characterized in that, The method for determining green visibility ratio also includes: The cloud receives task execution result data from each computing node performing data processing tasks on the data block; wherein, the data block corresponds to a geographical range; The received task execution result data is converted into a preset data format and then aligned to coordinates. For task execution result data that do not have overlapping geographical ranges, the data is spliced ​​together according to the geographical range; For task execution result data with overlapping geographical areas, the splicing weight of each data block is determined according to the confidence level of each data block in the overlapping area. The task execution result data corresponding to each data block is spliced ​​together using the splicing weight of each data block. An initial fused green view rate heatmap is generated based on the splicing results; The initial fused green view rate heatmap is subjected to outlier correction, spatial interpolation, and multi-resolution fused green view rate heatmaps are pre-generated using a pyramid model; This includes, in cases where the splicing results determine that the task execution result data corresponding to the missing data blocks within a geographical range is missing, the following: The steps include: acquiring data blocks that are missing from a geographical area, allocating computing nodes for processing, receiving task execution result data from the computing nodes, converting the received task execution result data into a preset data format, and performing coordinate alignment.

10. A green visibility rate determination system based on a vehicle-mounted system, characterized in that, include: The vehicle-mounted system in the green visibility rate determination method based on the vehicle-mounted system according to any one of claims 1 to 9; The cloud in the green visibility rate determination method based on vehicle-mounted systems according to any one of claims 8 to 9; At least one computing node in the green visibility rate determination method based on a vehicle-mounted system as described in any one of claims 8 to 9.

Images