Road surface cleaning decision-making method and device based on image recognition, equipment and medium

By employing an image recognition-based road cleaning decision-making method, combined with multimodal sensor data fusion and neural network analysis, refined processing of road surface types and defects has been achieved. This solves the problems of resource waste and low efficiency in existing technologies, enabling safe and efficient road cleaning.

CN120853127AInactive Publication Date: 2025-10-28XINHONGCHANG TECH (SHENZHEN) CO LTD
View PDF 0 Cites 3 Cited by

Patent Information

Application Number
CN202510924385.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-07-04
Publication Date
2025-10-28
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Existing road cleaning systems lack precise identification and targeted response to different road environments, resulting in wasted cleaning resources and low efficiency.

Method used

A road cleaning decision-making method based on image recognition is adopted. Through multimodal sensor data fusion, lightweight neural network analysis, improved U-Net network segmentation, multi-objective optimization solution and adaptive linkage control, real-time cleaning parameter instructions and collision-free path planning are generated to achieve refined processing of road surface type and defects.

Benefits of technology

It enables precise cleaning of different road surface environments, reduces resource waste, improves cleaning efficiency, and ensures the safe and efficient operation of cleaning equipment.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120853127A_ABST
    Figure CN120853127A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of image recognition, and provides a road surface cleaning decision-making method and device based on image recognition, equipment and a medium. The environment perception matrix is subjected to feature analysis through a lightweight neural network to obtain a pavement type probability distribution vector, and the binocular vision image is subjected to pavement defect segmentation through an improved U-Net network to obtain a pavement damage space distribution map, so that a resource constraint condition is combined to carry out multi-target optimization solution to obtain an optimal strategy set. And performing entropy weight-superior and inferior solution distance dynamic decision on the optimal strategy set to generate a real-time cleaning parameter instruction, performing obstacle identification and path planning according to the environment perception matrix to generate a collision-free path point sequence, and performing linkage control according to the real-time cleaning parameter instruction and the collision-free path point sequence to obtain a cleaning equipment driving signal. Through the technical means of multi-mode, multi-scale, dynamic feedback and the like, the method effectively adapts to diversified road conditions, and the resource utilization rate of road surface cleaning is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of image recognition technology, and in particular to a method, apparatus, device, and medium for road cleaning decision-making based on image recognition. Background Technology

[0002] Road cleaning plays a vital role in urban management and public safety. On the one hand, a clean road environment helps improve the city's image, air quality, and residents' living experience; on the other hand, timely removal of accumulated dust, garbage, oil stains, and other debris can effectively prevent safety and environmental problems such as slippery roads, water accumulation and blockages, and dust pollution. Especially in areas with heavy traffic, construction sites, or in adverse weather conditions, road cleaning is directly related to the safety of vehicle traffic and the comfort of pedestrians.

[0003] Currently, street cleaning primarily relies on manually driven sweepers or semi-automatic cleaning equipment. Basic cleaning functions are achieved through fixed work paths, preset cleaning modes, and manual supervision. This depends on simple timed scheduling systems or GPS-based path planning technology, combined with physical cleaning methods such as pressure spraying and roller brush suction. Some more advanced cleaning systems incorporate basic cameras or radar for obstacle avoidance or simple area recognition, but overall, cleaning remains largely extensive, lacking precise identification and targeted response to real-time road conditions. This often results in resource waste, low efficiency, and uncontrollable effects, making it difficult to adapt to the complex and ever-changing urban road environment. Summary of the Invention

[0004] In view of this, this application provides a road cleaning decision-making method, apparatus, equipment and medium based on image recognition to solve the problem of low precision in road cleaning and waste of cleaning resources due to lack of perception of different road environments.

[0005] The first aspect of this application provides a road cleaning decision-making method based on image recognition, the method comprising: Spatiotemporal alignment and fusion processing is performed on real-time acquired multimodal sensor data to obtain an environmental perception matrix; The environmental perception matrix is ​​subjected to multi-scale feature analysis using a preset lightweight neural network to obtain the road surface type probability distribution vector. The road surface defects are segmented using a pre-set improved U-Net network to obtain a spatial distribution map of road surface damage from real-time acquired binocular vision images. Based on the preset resource constraints, the probability distribution vector of the road surface type, and the spatial distribution map of road surface damage, a multi-objective optimization solution is performed to obtain the optimal strategy set; The optimal strategy set is subjected to entropy weight-distance dynamic decision-making processing to generate real-time cleaning parameter instructions; Obstacle identification and repulsive force potential field path planning are performed based on the dynamically updated environmental perception matrix to generate a collision-free path point sequence. Adaptive linkage control processing is performed based on the real-time cleaning parameter instructions and the collision-free path point sequence to obtain the cleaning equipment drive signal.

[0006] In an optional implementation, the lightweight neural network includes a MobileNetV3 model and a SENet model. The step of performing multi-scale feature analysis on the environmental perception matrix using a preset lightweight neural network to obtain a road surface type probability distribution vector includes: The MobileNetV3 model is used to extract edge and texture features from the environment perception matrix to obtain the original road surface feature map. The original road surface feature map is processed by channel attention weighting using the SENet model to obtain an enhanced feature map. The enhanced feature map is processed by a preset Softmax function to make classification decisions, thereby obtaining the road surface type probability distribution vector.

[0007] In an optional implementation, the step of segmenting road surface defects in real-time acquired binocular vision images using a preset improved U-Net network to obtain a spatial distribution map of road surface damage includes: The defect segmentation region features are obtained by performing multi-level downsampling feature extraction and upsampling decoding and localization on the real-time acquired binocular vision images through a pre-set improved U-Net network. Based on the binocular disparity information of the binocular vision image, the features of the defect segmentation region are reconstructed in three dimensions to obtain a spatial distribution map of road damage.

[0008] In an optional implementation, the step of performing multi-objective optimization based on preset resource constraints, the road surface type probability distribution vector, and the road surface damage spatial distribution map to obtain the optimal strategy set includes: Based on a preset objective function, a model is constructed for the probability distribution vector of the road surface type and the spatial distribution map of the road surface damage to obtain a multi-objective optimization model. The multi-objective optimization model is constrained and solved according to preset resource constraints to obtain a set of candidate strategies; The candidate strategy set is guided by reference points and non-dominated by a preset NSGA-III algorithm to obtain the optimal strategy set.

[0009] In an optional implementation, the entropy-weighted-difference dynamic decision-making process on the optimal strategy set to generate real-time cleaning parameter instructions includes: Normalize and calculate the information entropy weights of each target index in the optimal strategy set to obtain a multi-target weight vector; The distance score between the superior and inferior solutions of the multi-objective weight vector is calculated according to the preset positive and negative ideal types to obtain the proximity score corresponding to each strategy in the optimal strategy set. The optimal strategy set is sorted and filtered according to the proximity score to obtain the strategy with the highest proximity score. The strategy with the highest proximity score is then processed for instruction mapping to generate real-time cleaning parameter instructions.

[0010] In an optional implementation, the step of performing obstacle identification and repulsive force potential field path planning based on a dynamically updated environmental perception matrix to generate a collision-free path point sequence includes: Dynamic obstacle distance detection is performed on the dynamically updated environmental perception matrix to obtain obstacle spatial distribution data; A potential field model is constructed based on a preset repulsive field function and the spatial distribution data of the obstacles to obtain a potential energy distribution map. The target-oriented path is iteratively optimized using a preset gradient descent algorithm based on the potential energy distribution map to generate a collision-free path point sequence.

[0011] In an optional implementation, the adaptive linkage control processing based on the real-time cleaning parameter command and the collision-free path point sequence to obtain the cleaning equipment drive signal includes: The real-time cleaning parameter command is parsed and processed to obtain the target control parameters; The collision-free path point sequence is subjected to motion trajectory interpolation to obtain the device pose sequence; Adaptive stability control processing is performed based on the target control parameters and the device pose sequence to obtain the cleaning device drive signal.

[0012] A second aspect of this application provides a road cleaning decision-making device based on image recognition, the device comprising: The perception fusion module is used to perform spatiotemporal alignment and fusion processing on real-time acquired multimodal sensor data to obtain an environmental perception matrix; The road surface analysis module is used to perform multi-scale feature analysis on the environmental perception matrix through a preset lightweight neural network to obtain the road surface type probability distribution vector. The defect identification module is used to segment road surface defects from real-time acquired binocular vision images through a preset improved U-Net network to obtain a spatial distribution map of road surface damage. The cleaning strategy module is used to perform multi-objective optimization based on preset resource constraints, the probability distribution vector of the road surface type, and the spatial distribution map of road surface damage, so as to obtain the optimal strategy set. The cleaning instruction module is used to perform entropy weight-distance dynamic decision processing on the optimal strategy set to generate real-time cleaning parameter instructions. The path planning module is used to identify obstacles and plan paths based on the dynamically updated environmental perception matrix to generate a sequence of collision-free path points. The linkage control module is used to perform adaptive linkage control processing based on the real-time cleaning parameter instructions and the collision-free path point sequence to obtain the cleaning equipment drive signal.

[0013] A third aspect of this application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the image recognition-based road cleaning decision method as described above.

[0014] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the image recognition-based road cleaning decision method as described above.

[0015] In summary, this application includes at least the following beneficial technical effects: 1. By using lightweight networks to perform multi-scale feature analysis on the sensed environmental information, the cleaning intensity and cleaning mode can be automatically adjusted for different road surfaces (e.g., asphalt, concrete, bricks, etc.), thereby reducing energy consumption and resource waste during the cleaning process.

[0016] 2. By combining the improved U-Net network with binocular vision to segment road defects, it can accurately locate damaged or contaminated areas such as potholes, cracks, and stains, ensuring that the cleaning equipment only operates on areas that truly need cleaning.

[0017] 3. Through adaptive linkage control, real-time cleaning parameter commands and collision-free path point sequences are seamlessly integrated and transformed into driving signals for cleaning equipment, realizing a closed-loop integration from perception to decision-making to planning to execution. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart of a road surface cleaning decision-making method based on image recognition provided in an embodiment of this application; Figure 2 This is a functional block diagram of a road cleaning decision-making device based on image recognition provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] The image recognition-based road cleaning decision-making method provided in this application is executed by an electronic device, and correspondingly, the image recognition-based road cleaning decision-making device operates within the electronic device. The electronic device in this application is a fully automated cleaning unmanned vehicle integrating environmental perception sensors (e.g., LiDAR, binocular cameras, and / or inertial measurement units). In addition to automatic cleaning functions, this fully automated cleaning unmanned vehicle also possesses adaptive cruise and obstacle avoidance functions. The electronic device in this application is used for complex patchwork road surfaces, such as the cleaning of public areas in residential communities with separate pedestrian and vehicle access. In these communities, underground parking lots use epoxy flooring, surface vehicle access surfaces use asphalt, and pedestrian walkways use paving bricks and cement. The image recognition-based road cleaning decision-making method provided in this application will be described below from the perspective of a fully automated cleaning unmanned vehicle, combined with the process of cleaning public areas in a residential community.

[0022] like Figure 1 The diagram shows a flowchart of a road surface cleaning decision-making method based on image recognition provided in an embodiment of this application. The road surface cleaning decision-making method based on image recognition provided in this embodiment includes the following steps.

[0023] Step S1: Perform spatiotemporal alignment and fusion processing on the real-time acquired multimodal sensor data to obtain the environmental perception matrix.

[0024] It should be understood that the fully automated cleaning vehicle in this embodiment integrates three heterogeneous sensors: LiDAR, binocular camera, and inertial measurement unit (IMU). During the cleaning of public areas in the community, the fully automated cleaning vehicle (hereinafter referred to as the "vehicle") uses these three heterogeneous sensors to collect information about the surrounding environment in real time, thereby achieving environmental perception. When obtaining the multimodal sensor data collected in real time by these three heterogeneous sensors, spatiotemporal alignment and fusion processing is required to generate a unified environmental perception matrix representing the geometric features and visual information of the scene. The multimodal sensor data includes, but is not limited to, synchronously acquired data from LiDAR, binocular camera, and inertial measurement unit. The LiDAR emits 200,000 laser beams per second to generate point clouds, enabling it to capture the surface reflection features of objects in three-dimensional space. The binocular camera acquires two color images (left and right) at a resolution of 1920×1080 pixels, recording road texture and color information. The IMU records the vehicle's tilt angle and linear acceleration to correct distortions caused by sensor attitude and motion. These multi-source information complement each other, enabling accurate modeling of complex paved surfaces (e.g., epoxy, asphalt, brick, and cement) under lighting and motion conditions.

[0025] First, during the debugging phase, spatial calibration of the LiDAR and binocular camera is necessary to eliminate viewpoint differences. A specially designed checkerboard calibration board is placed in front of the autonomous vehicle's path. While stationary, the LiDAR scans the corner points of the board, while the camera simultaneously images the checkerboard. Based on the 3D corner coordinates measured by the LiDAR and the corresponding pixel coordinates in the camera image, the rotation matrix and translation vector are solved using a least-squares fitting method, thus establishing the mapping relationship between the LiDAR coordinate system and the camera coordinate system. Spatial calibration of the LiDAR and binocular camera ensures that the point cloud and the image can be mapped to the same world coordinate system.

[0026] Secondly, distortion correction is performed on the real-time collected point cloud data to address motion distortion during driving. During cruise and obstacle avoidance, the autonomous vehicle inevitably undergoes translation and rotation. The tilt angle and linear acceleration information recorded by the IMU are used to estimate the vehicle's trajectory over a short period, thereby performing reverse compensation on each laser point according to the vehicle's attitude at the time of acquisition. For example, when the autonomous vehicle moves 0.2 meters in 0.1 seconds with a slight turn, the original point cloud is corrected using reverse rotation and linear translation operations based on the angular velocity and acceleration data output by the IMU, ensuring that all point cloud data correspond to the same vehicle attitude at the same reference time.

[0027] Subsequently, disparity calculation and depth estimation are performed on the binocular camera images to obtain pixel-level 3D information from the left and right views. A classic stereo matching algorithm is employed, using a sliding window to search for identical feature points between the left and right images, calculating pixel offsets, and then combining this with the known camera focal length and lens baseline distance to convert the disparity into depth values ​​for the corresponding points in the world coordinate system, generating a dense depth map. This depth map, combined with point cloud data, can fill the measurement blind spots of LiDAR at long distances or on low-reflectivity material surfaces (such as moist epoxy flooring), providing rich geometric supplementation to the environmental perception matrix.

[0028] Subsequently, the data from each sensor is mapped onto a unified 3D mesh to form a voxel representation. The inspection area is divided into multiple layers of cubic units (i.e., voxels) with centimeter-level precision in both the horizontal and vertical directions. Within each voxel, the corrected point cloud reflectance intensity information, the RGB values ​​of the corresponding pixels in the depth map, and point cloud density statistics are aggregated. Specifically, for each voxel, the average point cloud reflectance intensity, the average color channel value of the corresponding pixel, and the density value of the number of points per unit volume are calculated, representing the object's material characteristics, visual characteristics, and geometric density characteristics, respectively.

[0029] Finally, after obtaining the voxelized structure with multimodal information spatial alignment, it is unfolded into an environment perception matrix. This matrix is ​​used for subsequent multi-scale feature analysis by the neural network. The voxel features are arranged sequentially along the vertical and horizontal depth dimensions, enabling the neural network to simultaneously perceive reflection intensity, color, and density information based on a single tensor input, achieving accurate differentiation of various materials such as epoxy, asphalt, paving bricks, and cement. Furthermore, the voxelized representation is compatible with dynamic obstacle position updates, allowing direct access to dynamic obstacle coordinates in subsequent potential field path planning.

[0030] Step S2: Perform multi-scale feature analysis on the environmental perception matrix using a preset lightweight neural network to obtain the road surface type probability distribution vector.

[0031] It should be understood that after constructing the environmental perception matrix, the fully automated cleaning vehicle immediately enters into multi-scale feature decoupling processing of the perception matrix to achieve accurate classification of the materials of the patchwork pavement in the community's public areas. The environmental perception matrix includes three parts of information: the average reflection intensity of the three-dimensional point cloud generated by the front-end LiDAR, the average RGB color values ​​of the binocular camera images, and the point cloud density. It is a three-dimensional tensor expanded after being divided into centimeter-level voxels, and it contains the comprehensive features of the pavement materials and geometric structures at various locations in the environment. The environmental perception matrix combines the high-precision geometric measurement capabilities of LiDAR with the sensitivity of binocular vision to material texture and color, allowing the apparent differences of different materials on complex patchwork pavements to be fully preserved.

[0032] When performing multi-scale feature analysis on the environmental perception matrix, the matrix is ​​first input into a pre-defined lightweight neural network. In this embodiment, the lightweight neural network consists of a cascaded MobileNetV3 model and a SENet model, balancing computational efficiency and feature representation capability. The MobileNetV3 model is a convolutional neural network designed specifically for resource-constrained devices. It introduces depthwise separable convolutions and inverse residual structures, and uses lightweight attention modules at network bottlenecks, thereby maintaining low computational and parameter counts while still extracting basic features such as edges and textures. By applying the primary convolutional layers and inverse residual units from the MobileNetV3 module to the environmental perception matrix, subtle differences in reflectance intensity and color channels on the surfaces of materials such as soil, asphalt, epoxy flooring, cement, and paving bricks can be quickly detected. For example, asphalt surfaces have low reflectance and a darker color, while epoxy flooring exhibits high reflectance and uniform color.

[0033] Next, the original road surface feature map extracted by the MobileNetV3 model is fed into the SENet model for channel attention weighting. The SENet model performs global average pooling and fully connected layer transformation on each channel in the original road surface feature map, and then generates channel-level attention weights through the Sigmoid activation function. This amplifies the channel signals that are more sensitive to road surface classification and suppresses redundant or weak response channels. In the complex road conditions of public areas in residential communities, different voxels may have high similarity in shape and color. It is difficult to distinguish the subtle differences between paved bricks and cement, or dry asphalt and wet asphalt, simply relying on the primary features extracted by the MobileNetV3 model. However, the attention weights generated by the SENet model can automatically adjust the contribution ratio of each channel in the final decision according to the global feature distribution. For example, it assigns higher weights to high-contrast feature channels around paving gaps, thereby making the regional key textures in the enhanced feature map set stand out significantly.

[0034] Finally, the multi-channel feature tensor output from the enhanced feature map set of the SENet model is processed using the Softmax function for classification decision processing. When the original output values ​​of multiple candidate material categories (i.e., the logits obtained by each fully connected unit) are input into the Softmax function, the Softmax function first exponentializes them and then normalizes them, thus transforming them into a probability distribution that sums to one and falls within [0,1], representing the confidence of different road surface types under the current autonomous vehicle's field of vision. For example, after the weighted feature map of the SENet model is input into the classification layer, it may produce a score of 2.5 for asphalt, 1.2 for epoxy, 0.8 for paving, and 0.6 for cement. After transformation by the Softmax function, the probability distributions of asphalt (58%), epoxy (20%), paving (13%), and cement (9%) are obtained, respectively. This probability distribution is the road surface type probability distribution vector, which can provide a material weight reference for subsequent multi-objective optimization, thereby guiding the priority allocation of cleaning parameters and path planning decisions.

[0035] Step S3: Use a pre-set improved U-Net network to segment road surface defects in the real-time acquired binocular vision images to obtain a spatial distribution map of road surface damage.

[0036] It should be understood that after completing the construction of the environmental perception matrix, the autonomous vehicle immediately performs detailed detection of pavement defects to support the accurate distribution of cleaning parameters such as water pressure and brush speed. Specifically, the binocular vision images refer to a sequence of images simultaneously captured by two 1920×1080 pixel cameras at the front of the vehicle. This image sequence contains both road texture and color information, and relies on the parallax generated by a fixed baseline between the cameras, thus enabling pixel-level depth calculation.

[0037] First, the binocular vision images are input into an improved U-Net network for multi-level downsampling and upsampling processing to achieve pixel-level segmentation of road surface defects (i.e., road surface defects, including but not limited to cracks, potholes, and spalling). The improved U-Net network retains the encoder-decoder structure and skip connections of the classic U-Net network, while introducing depthwise separable convolutions and batch normalization modules between each encoding and decoding layer to reduce computational overhead and accelerate convergence. Furthermore, multi-scale dilated convolutions are added to the skip channels, enabling the network to consider both global context and local details when extracting fine crack edges, pothole morphology, and spalling interfaces. For example, when an autonomous vehicle is driving in an underground parking garage, the crack width on the epoxy floor surface is typically only a few millimeters. Simple standard convolutions are insufficient to handle both small-scale defects and a large background; dilated convolutions can enhance the detection capability of fine cracks while maintaining the receptive field. In actual operation, the left and right binocular vision images are first downsampled through four layers to extract multi-scale features. Then, the feature maps are dilated convolution and skip fusion at the highest semantic layer. Finally, they are upsampled through four layers to gradually restore the original pixel resolution and output a binarized defect segmentation map (i.e., defect segmentation region features). In this map, the foreground pixels correspond to the defect region and the background pixels correspond to the normal road surface.

[0038] Next, based on the binocular disparity information from the left and right binocular visual images, the segmented defect region features are reconstructed in 3D to obtain a spatial distribution map of road damage. Here, binocular disparity information refers to the lateral offset of the same road defect element in the left and right images, calculated using classic block matching or semi-global matching algorithms. In road cleaning scenarios, the accuracy of the disparity map can be improved by matching edge feature points, thereby converting pixels located at the center of potholes or at the bends of cracks into depth values. After masking and fusing the segmented contour and the disparity map, the 3D coordinates of the road defect in the world coordinate system can be reconstructed from the 2D pixel coordinates and corresponding depth values. For example, when a crack has an x-coordinate of 500 and a y-coordinate of 300 in the left image, and the corresponding point has an x-coordinate of 470 in the right image, the disparity between the two is 30 pixels. Combining this with the known focal length of the binocular camera and the baseline distance, the absolute depth of that point from the camera can be calculated. In this way, the depth of every pixel on the entire crack curve can be measured, ultimately forming a continuous 3D defect curve.

[0039] In one optional implementation, after completing the 3D coordinate reconstruction, spatially adjacent defect pixels are aggregated into independent defect block units through connected component analysis, and their voxelized bounding boxes and geometric parameters, such as length, width, and depth range, are calculated. This allows the spatial distribution map of pavement damage to not only include the 3D location of each defect but also incorporate additional attributes such as point cloud density differences and crack depth estimation. This supports adaptive adjustment of water pressure and brush speed parameters required during cleaning for pavements made of different materials. For example, if chewing gum residue appears on a paved sidewalk, its depth may only be a few millimeters but its width may be small. The spatial distribution map will mark the corresponding 3D window position, and the cleaning system can instantly activate the high-pressure spray mode when it reaches that position, without having to use the same intensity of water flow on a large area of ​​asphalt pavement, thus avoiding resource waste and damage to defect-free areas.

[0040] Step S4: Perform multi-objective optimization based on preset resource constraints, the road surface type probability distribution vector, and the road surface damage spatial distribution map to obtain the optimal strategy set.

[0041] After obtaining the probability distribution vector of road surface types and the spatial distribution map of road damage, the unmanned vehicle immediately enters a multi-objective optimization solution to obtain the optimal cleaning strategy under limited water, electricity, and time budgets. This application's embodiment employs a pre-defined objective function to quantify the balance between three indicators: cleaning effect, water and electricity consumption, and operation time. This function combines the cleaning coefficients of each road surface type in the probability distribution vector with the cleaning difficulty of each defective area in the spatial distribution map to form a comprehensive evaluation function (i.e., a multi-objective optimization model). Specifically, the objective function first maps the cleaning efficiency corresponding to different materials in the road surface type probability distribution vector to numerical indicators. For example, in the epoxy flooring area of ​​an underground parking garage, high efficiency is prioritized, while in the paved passage area, water pressure priority is reduced due to the fragility of the material. Simultaneously, using the crack length or pothole area information marked in the spatial distribution map of road damage, the cleaning priority of each damaged area is synchronously converted into a numerical penalty term to ensure that more resources are allocated to severely damaged areas. The resulting multi-objective optimization model can be considered a mathematical description of the three objectives: cleaning effect, water and electricity consumption, and operation time.

[0042] After the model is built, the multi-objective optimization model is constrained and solved using preset resource constraints. These constraints include the remaining water tank volume, battery life, and the maximum operating speed of the equipment, all of which are monitored in real time by the autonomous vehicle and fed back to the optimization module during execution. The significance of constraint solving is to eliminate strategies that exceed the current physical resource capacity, preventing situations where insufficient water leads to missed tracks or power depletion causes mid-journey shutdowns. For example, when the spatial distribution map of road damage shows several cracks concentrated on a section of asphalt road ahead, the autonomous vehicle's system automatically assesses the water and current consumption required for that section of the path and compares it with the current remaining water tank capacity of 50 liters and battery life of 30 minutes. If the predicted total consumption exceeds the resource threshold, such high-intensity cleaning strategies will not be included in the subsequent ranking process. The resulting candidate strategy set satisfies the cleaning needs of different materials and damage characteristics on complex patchwork roads while achieving scientific resource allocation within the water, electricity, and time budgets.

[0043] Subsequently, the NSGA-III algorithm is used to guide the candidate strategy set with reference points and perform non-dominated sorting to select the truly optimal strategy set from the multidimensional optimization space. The NSGA-III algorithm can uniformly distribute reference points in the high-dimensional target space and, based on fast non-dominated sorting, improves the diversity and balance of the solution set through reference point guidance. Specifically, the NSGA-III algorithm first labels each candidate strategy with a non-dominated level based on cleaning effect, water and electricity consumption, and operation time, placing all solutions that cannot be surpassed by any other strategy in all three metrics into the first level. Then, using pre-configured uniformly distributed reference points in the three-dimensional target space, it calculates the congestion degree and measures the distance to reference points for the solutions in the first level, prioritizing those strategies that are more evenly distributed in the target space and closer to the reference points. This ensures that the final strategy set includes solutions with high cleaning effect, as well as solutions with low consumption and short operation time, covering different trade-off combinations. For example, in a cleaning task involving a mixture of epoxy, asphalt, and paved surfaces, the NSGA-III algorithm does not simply return a single water-saving or shortest-time solution, but instead outputs a set of Pareto solutions. These Pareto solutions include both fast solutions with slightly lower water pressure but extremely short processing times, and balanced solutions with higher water pressure but moderate overall water and electricity consumption.

[0044] Step S5: Perform entropy weight-superior solution distance dynamic decision processing on the optimal strategy set to generate real-time cleaning parameter instructions.

[0045] After obtaining the optimal strategy set, the autonomous vehicle needs to comprehensively evaluate different cleaning schemes in real-time resource and multi-material road scenarios, so as to select the cleaning parameters that best meet the actual needs under the constraints of current water tank capacity, battery range and material distribution in the cleaning area, and generate instruction vectors (i.e., real-time cleaning parameter instructions) that can be directly issued to the drive system.

[0046] First, the target indicators of each scheme in the optimal strategy set are vectorized. The cleaning effect indicator is mapped to [0,1] using a vector length standardization method. Water and electricity consumption and operation time are processed similarly, maintaining consistent numerical proportions for each strategy in the same dimension after mapping. The optimal strategy set is then normalized to map the original values ​​of each strategy in the three dimensions of cleaning effect, water and electricity consumption, and operation time to a unified dimensional range, eliminating comparison biases caused by differences in dimensions and numerical scales. Next, the information entropy value is calculated based on each column of the normalization matrix (i.e., the distribution of the same indicator across different strategies): when the normalized value of an indicator tends to be concentrated across all schemes (i.e., low dispersion), its information entropy is high, indicating a smaller contribution to distinguishing different schemes; conversely, indicators with low information entropy indicate significant differences in their values ​​across different strategies and should be given higher weight in decision-making. Finally, by subtracting the normalized information entropy from 1 and normalizing again, the entropy weight of each indicator is obtained, thus forming a multi-objective weight vector. The multi-objective weight vector enables the relative importance of cleaning effect, water and electricity consumption and duration to be adaptively adjusted according to the actual differences in the solution set in the current scenario.

[0047] Along with the generation of the multi-objective weight vector, the pre-defined positive and negative ideal types can be used to score the distance between the best and worst solutions. The positive ideal type refers to the idealized vector in which each indicator simultaneously reaches its optimal value among the candidate solutions after normalization and incorporating entropy weights, while the negative ideal type is the vector in which each indicator simultaneously reaches its worst value. The positive and negative ideal types are equivalent to two benchmark points in the decision space: the "optimal state" and the "worst state." In the three-dimensional objective space, they correspond to the theoretical cross-combination of the highest cleaning effect, the lowest water and electricity consumption, and the shortest operation time, as well as their opposites. Using a weighted Euclidean distance calculation method, the distance between the weighted normalized vector of each strategy and the positive ideal type is considered a negative evaluation quantity, and the distance with the negative ideal type is considered a positive evaluation quantity; the ratio of the two is the proximity score. The greater the proximity score, the closer the solution is to the positive ideal and the further away from the negative ideal, resulting in the best overall performance. Taking the paved area of ​​the pedestrian passage in the community as an example, when a certain strategy performs well in terms of cleaning effect and energy saving time, but is slightly high in water consumption, its weighted distance from the positive ideal type in the first and third dimensions is relatively small, while its distance from the negative ideal type in these two dimensions is relatively large. Although it is somewhat closer to the negative ideal in the second dimension of water consumption, the overall closeness score can still reflect the balanced advantage of the strategy of "high efficiency-short time-medium water consumption".

[0048] Finally, the optimal strategy set is sorted according to the proximity score of each strategy, and the one or more strategies with the highest proximity are selected and mapped to real-time cleaning parameter instructions. The instruction mapping process directly organizes the parameters such as water pressure, brush speed, and water volume corresponding to the strategy with the highest proximity into a numerical vector that can be called by the drive system, and combines it with the current motion state of the unmanned vehicle (such as current position, direction of travel, and real-time speed) to generate a complete sequence of motion and cleaning actions. For example, in the epoxy flooring area of ​​an underground parking garage, when the optimal strategy points to a water pressure of 7.0MPa, a brush speed of 320RPM, and a water volume of 10L, the instruction mapping unit will bind this numerical group with the current path point sequence to form a continuous control signal of "spraying and brushing with 7.0MPa, 320RPM, and 10L of water in the next 5-meter path segment," and then send it to the water pump controller and motor driver in real time to achieve closed-loop linkage control.

[0049] Step S6: Based on the dynamically updated environmental perception matrix, perform obstacle identification and repulsive force potential field path planning to generate a collision-free path point sequence.

[0050] After constructing the real-time environmental perception matrix, the autonomous vehicle must dynamically avoid moving obstacles such as pedestrians and vehicles to ensure both precise cleaning and safe passage on complex, spliced ​​road surfaces in the community's public areas. It should be understood that the environmental perception matrix obtained in step S1 is continuously refreshed at a centimeter-level frequency during the driving process, ensuring it constantly reflects the geometric positions and visual characteristics of surrounding obstacles. The environmental perception matrix not only carries information on road surface material and damage but also incorporates voxel representations of dynamic targets such as pedestrians and temporarily parked vehicles in real time.

[0051] First, within each control cycle, the autonomous vehicle calculates the environmental perception matrix at the current moment and performs Euclidean distance detection on newly added or moved non-ground voxels in the matrix. Euclidean distance detection involves calculating the three-dimensional spatial distance from the center point of the autonomous vehicle to each voxel and selecting voxel clusters with distances less than a safety threshold (e.g., 1.5 meters from the pedestrian walkway) as obstacle spatial distribution data. This is equivalent to drawing a "safety monitoring sphere" within the environmental perception matrix, and uniformly marking non-ground objects falling into the sphere as potential collision targets. For example, in the central plaza of a residential area, when a pedestrian steps out from under the shade of a tree and enters the cleaning path, the autonomous vehicle can quickly locate the pedestrian's head and shoulder contours in the perception matrix using point cloud and depth images, and collect the three-dimensional coordinates of the voxels containing them to form obstacle spatial distribution data.

[0052] Next, the autonomous vehicle uses a pre-defined repulsive potential field function as a blueprint to construct a comprehensive potential energy distribution map based on obstacle spatial distribution data. Specifically, the repulsive potential field function treats obstacles as "repulsive force sources" and generates repulsive forces of varying magnitudes based on the distance between the obstacle and the autonomous vehicle's current position. The resulting potential energy distribution map generates a continuous potential energy distribution surface across the entire driving space, with the highest potential energy and strongest repulsive force around obstacles, and the lowest potential energy in the safe area far from obstacles. During the construction of the potential energy distribution map, the potential energy value is first calculated based on distance modulation parameters, centered on each obstacle element. This can be described as follows: the closer the obstacle, the stronger the repulsive force; the farther the obstacle, the weaker the repulsive force; the repulsive force gradually returns to zero outside the safe distance range. Subsequently, the repulsive potential energy of all obstacle voxels is superimposed and merged with the gravitational potential energy pointing towards the target cleaning area. The gravitational potential energy can be analogized to a "centripetal force field," enabling the autonomous vehicle to move towards the cleaning target while moving away from obstacles. This results in a comprehensive potential energy distribution map that includes both obstacle avoidance repulsion and target attraction. For example, in a paved sidewalk in a residential area, when a temporarily parked electric vehicle appears ahead, its voxel forms a towering repulsive energy peak on the potential energy map, while the paved area to be cleaned in the distance forms a gravitational depression on the map. The autonomous vehicle will travel between these two areas along the direction of the fastest decrease in potential energy.

[0053] Subsequently, based on the constructed potential energy distribution map, a preset gradient descent algorithm is used to iteratively optimize the target guidance path. The gradient descent algorithm refers to moving the autonomous vehicle (RV) from its current position within a continuous potential energy field, advancing along the negative direction of the potential energy gradient at each step. This ensures that each displacement minimizes the potential energy value, thus avoiding obstacles and approaching the clean target with the least potential energy path. This process can be figuratively compared to walking on uneven terrain, always choosing to land on the downhill side to reach the valley floor with minimal effort. During path iteration, the RV calculates the local potential energy gradient between adjacent grid points or continuous coordinates, and updates its position according to a preset step size (which can be dynamically adjusted based on current speed and response delay) until the target area is reached or a sufficiently low potential energy threshold is reached before stopping the iteration. For example, at the corner where paved and asphalt roads meet, the gradient descent algorithm first generates a repulsive force at the edge of the brick joint to increase potential energy, and then finds an arc-shaped path at the sloping junction of the asphalt and brick joints. This path ensures that the RV maintains a minimum safe distance from obstacles while minimizing the travel distance.

[0054] After multiple rounds of gradient iteration, the recorded sequence of continuous location points is output as a collision-free path point sequence, with current attitude and velocity planning information appended to each point. The collision-free path point sequence not only defines the autonomous vehicle's trajectory on the plane, but also, in conjunction with the real-time updated environmental perception matrix, recalibrates for short-term sudden obstacles between points. If a pedestrian or small animal suddenly appears, the potential field and gradient iteration can be restarted at the next moment to dynamically generate a new point sequence, achieving a "seamless relay" continuous obstacle avoidance effect.

[0055] Step S7: Perform adaptive linkage control processing based on the real-time cleaning parameter instructions and the collision-free path point sequence to obtain the cleaning equipment drive signal.

[0056] After obtaining the collision-free pathpoint sequence and real-time cleaning parameter instructions, the autonomous vehicle must coordinate the two major actuators—the drive system and the cleaning system—to achieve closed-loop linkage control that ensures both safe travel along a predetermined trajectory and appropriate cleaning intensity at different road surface materials and damage levels. The real-time cleaning parameter instructions output specific numerical vectors such as water pressure, brush speed, and water volume, directly quantifying the optimal cleaning conditions for the autonomous vehicle on the current road segment for different materials (e.g., epoxy flooring, asphalt pavement, paved roads, or cement walkways) and different damage types (e.g., cracks, potholes). The collision-free pathpoint sequence refers to a series of three-dimensional coordinate points calculated in a dynamic obstacle environment based on potential field planning and gradient descent algorithms. This sequence includes not only the planar driving trajectory but also implicitly contains the vehicle's attitude, heading, and speed constraints at corresponding moments, used to drive the chassis control module for precise tracking. Adaptive linkage control fuses these two types of data, dynamically adjusting the behavior of the cleaning and drive actuators to ensure the autonomous vehicle maintains optimal coordination between cleaning parameters and the driving trajectory throughout its movement.

[0057] First, the real-time cleaning parameter commands need to be parsed to obtain target control parameters that can be directly invoked by the actuators. Parsing involves decoding the encoded values ​​in the command vector to correspond one-to-one with the actuator mapping: water pressure commands are converted into the opening percentage of the electromagnetic proportional valve, supplied to the pump motor speed control unit; brush speed commands correspond to the target torque setting of the brush head motor, enabling the drive motor controller to precisely maintain the set speed through a closed-loop speed loop; water volume commands are linked with the flow meter's electrical signal acquisition module, automatically setting the upper and lower limits of the flow rate through the valve flow regulation mechanism to ensure that the total sprayed water volume within the entire cleaning segment conforms to the optimal strategy. Real-time status feedback also needs to be considered during the parsing process. For example, in an underground parking garage with epoxy flooring, a sudden increase in resistance at a pipe interface may occur, causing the actual water pressure to fall below the set value. The control unit will combine the target electromagnetic valve opening with the current current deviation to recalculate a new valve opening to compensate for pressure loss, thereby ensuring a continuous output of constant water pressure during the cleaning process. After parsing, the closed-loop control module will generate the pump drive current, brush head motor PWM duty cycle, and valve stepper motor pulse sequence with millimeter-level precision, thus constituting the target control parameters for this stage.

[0058] Simultaneously, motion trajectory interpolation processing is performed on the collision-free pathpoint sequence to obtain a high-frequency, continuous, and smooth device pose sequence. The pathpoint sequence typically consists of a finite number of discrete points, each recording the autonomous vehicle's position (x, y, z), heading angle, and initial planned speed in the global coordinate system. To ensure the chassis control module can accurately track the predetermined trajectory without stuttering during execution, interpolation needs to be performed between adjacent pathpoints at appropriate time intervals: cubic spline interpolation or fifth-order polynomial interpolation is used for position coordinates to generate smooth curves, ensuring continuous curvature changes and that the radius at turns is not less than the vehicle's minimum turning radius; linear or cosine function interpolation is used for heading angle and vehicle speed to ensure a smooth transition between yaw rate and longitudinal acceleration during acceleration, deceleration, and steering transitions, avoiding cleaning brush head detachment from the road surface or uneven cleaning effect due to rapid acceleration or sharp turns. The interpolation processing also needs to be combined with the current attitude and acceleration feedback provided by the inertial measurement unit to perform short-term corrections on the calculated pose sequence to eliminate minor deviations caused by vehicle vibration or road surface unevenness. The device pose sequence obtained after interpolation is a series of high-precision timestamp-linked triplets of position, attitude, and velocity.

[0059] Finally, the target control parameters and the equipment pose sequence are input into the Lyapunov adaptive stability control algorithm to obtain the final cleaning equipment drive signal. Lyapunov adaptive control is a nonlinear closed-loop control method based on energy function construction. Its core lies in defining a positive definite energy function for the controlled object. When the system tracking error changes linearly or nonlinearly, an appropriate control law is constructed to ensure that the energy function monotonically decreases with time, thereby achieving global stability and accurate tracking. For the cleaning actuator, the error typically includes the difference between the water pressure setpoint and the actual pressure, and the difference between the target speed and the real-time speed of the brush head motor; for the drive actuator, the error includes the difference between the current position and the interpolated trajectory position, heading deviation, and speed deviation. In the control law design, a comprehensive error energy function is first constructed, which weights and sums the water pressure error, speed error, position error, and heading error according to their priority to form the total energy index. Then, based on the Lyapunov stability condition, the control gain is designed as a combination of the inverse proportional and integral terms of the error function. This ensures that when any error increases, the corresponding drive signals (such as water pump motor current, brush head drive voltage, and wheel drive torque) are adjusted in a timely manner to reduce the error as quickly as possible. The dynamic adaptive characteristic is reflected in the fact that when the road surface changes from smooth asphalt to rough paving bricks, the inertial sensor and speed feedback will detect the drift between the vehicle's actual speed and the set speed. The controller will automatically increase the wheel drive torque according to the deviation and slightly increase the water pump speed at appropriate times to overcome the additional resistance caused by the paving brick gaps, ensuring the dual stability of trajectory tracking and cleaning effect.

[0060] This application is applied to the field of image recognition technology. It obtains an environmental perception matrix by spatiotemporally aligning and fusing multimodal sensor data. A lightweight neural network is then used for multi-scale feature analysis to obtain a road surface type probability distribution vector. An improved U-Net network is used to segment road defects in binocular vision images to obtain a spatial distribution map of road damage. Based on resource constraints, the road surface type probability distribution vector, and the spatial distribution map of road damage, a multi-objective optimization solution is performed to obtain an optimal strategy set. This optimal strategy set is then used for entropy-weighted, superior-inferiority distance dynamic decision-making to generate real-time cleaning parameter commands. Simultaneously, obstacle recognition and path planning are performed based on the environmental perception matrix to generate a collision-free path point sequence. Adaptive linkage control is then performed based on the real-time cleaning parameter commands and the collision-free path point sequence to obtain the driving signal for the cleaning equipment. This application organically integrates modules such as multimodal sensing, lightweight network analysis, segmentation algorithms, optimization decision-making, and path planning. This not only effectively adapts to diverse road conditions but also significantly reduces manual intervention, maintenance costs, and resource waste.

[0061] like Figure 2 The diagram shown is a functional block diagram of a road cleaning decision device based on image recognition provided in an embodiment of this application.

[0062] In some embodiments, the image recognition-based road cleaning decision-making device 2 may include multiple functional modules composed of computer program segments. The computer programs for each program segment in the image recognition-based road cleaning decision-making device 2 may be stored in the server's memory and executed by at least one processor to perform (see details). Figure 1 (Description) Functionality of the image recognition-based road cleaning decision-making method.

[0063] In this embodiment, the image recognition-based road cleaning decision-making device 2 can be divided into multiple functional modules according to its functions. These functional modules may include: a perception fusion module 21, a road surface analysis module 22, a defect identification module 23, a cleaning strategy module 24, a cleaning instruction module 25, a path planning module 26, and a linkage control module 27. The module referred to in this invention is a series of computer program segments that can be executed by at least one processor and perform a fixed function, stored in memory. In this embodiment, the functions of each module will be detailed in subsequent embodiments.

[0064] The perception fusion module 21 is used to perform spatiotemporal alignment and fusion processing on real-time acquired multimodal sensor data to obtain an environmental perception matrix.

[0065] The road surface analysis module 22 is used to perform multi-scale feature analysis on the environmental perception matrix through a preset lightweight neural network to obtain the road surface type probability distribution vector.

[0066] In an optional implementation, the road surface analysis module 22 is specifically used for: The MobileNetV3 model is used to extract edge and texture features from the environment perception matrix to obtain the original road surface feature map. The original road surface feature map is processed by channel attention weighting using the SENet model to obtain an enhanced feature map. The enhanced feature map is processed by a preset Softmax function to make classification decisions, thereby obtaining the road surface type probability distribution vector.

[0067] The defect identification module 23 is used to segment road surface defects from real-time acquired binocular vision images through a preset improved U-Net network to obtain a spatial distribution map of road surface damage.

[0068] In an optional implementation, the defect identification module 23 is specifically used for: The defect segmentation region features are obtained by performing multi-level downsampling feature extraction and upsampling decoding and localization on the real-time acquired binocular vision images through a pre-set improved U-Net network. Based on the binocular disparity information of the binocular vision image, the features of the defect segmentation region are reconstructed in three dimensions to obtain a spatial distribution map of road damage.

[0069] The cleaning strategy module 24 is used to perform multi-objective optimization based on preset resource constraints, the probability distribution vector of the road surface type, and the spatial distribution map of road surface damage, so as to obtain the optimal strategy set.

[0070] In an optional implementation, the cleaning strategy module 24 is specifically used for: Based on a preset objective function, a model is constructed for the probability distribution vector of the road surface type and the spatial distribution map of the road surface damage to obtain a multi-objective optimization model. The multi-objective optimization model is constrained and solved according to preset resource constraints to obtain a set of candidate strategies; The candidate strategy set is guided by reference points and non-dominated by a preset NSGA-III algorithm to obtain the optimal strategy set.

[0071] The cleaning instruction module 25 is used to perform entropy weight-superiority solution distance dynamic decision processing on the optimal strategy set to generate real-time cleaning parameter instructions.

[0072] In an optional implementation, the cleaning instruction module 25 is specifically used for: Normalize and calculate the information entropy weights of each target index in the optimal strategy set to obtain a multi-target weight vector; The distance score between the superior and inferior solutions of the multi-objective weight vector is calculated according to the preset positive and negative ideal types to obtain the proximity score corresponding to each strategy in the optimal strategy set. The optimal strategy set is sorted and filtered according to the proximity score to obtain the strategy with the highest proximity score. The strategy with the highest proximity score is then processed for instruction mapping to generate real-time cleaning parameter instructions.

[0073] The path planning module 26 is used to perform obstacle recognition and repulsive force potential field path planning based on the dynamically updated environmental perception matrix to generate a collision-free path point sequence.

[0074] In an optional implementation, the path planning module 26 is specifically used for: Dynamic obstacle distance detection is performed on the dynamically updated environmental perception matrix to obtain obstacle spatial distribution data; A potential field model is constructed based on a preset repulsive field function and the spatial distribution data of the obstacles to obtain a potential energy distribution map. The target-oriented path is iteratively optimized using a preset gradient descent algorithm based on the potential energy distribution map to generate a collision-free path point sequence.

[0075] The linkage control module 27 is used to perform adaptive linkage control processing based on the real-time cleaning parameter instructions and the collision-free path point sequence to obtain the cleaning equipment drive signal.

[0076] In an optional implementation, the linkage control module 27 is specifically used for: The real-time cleaning parameter command is parsed and processed to obtain the target control parameters; The collision-free path point sequence is subjected to motion trajectory interpolation to obtain the device pose sequence; Adaptive stability control processing is performed based on the target control parameters and the device pose sequence to obtain the cleaning device drive signal.

[0077] It should be understood that the various variations and specific embodiments of the methods provided in the above embodiments are also applicable to the image recognition-based road cleaning decision-making device of this embodiment. Through the foregoing detailed description of the image recognition-based road cleaning decision-making method, those skilled in the art can clearly understand the implementation method of the image recognition-based road cleaning decision-making device of this embodiment. For the sake of brevity, it will not be described in detail here.

[0078] like Figure 3 The diagram shown is a structural schematic of an electronic device provided in an embodiment of this application.

[0079] In a preferred embodiment of the present invention, the electronic device 3 may include, but is not limited to, a memory 31, at least one processor 32, and at least one communication bus 33.

[0080] Those skilled in the art should understand that Figure 3 The structure of the electronic device 3 shown does not constitute a limitation of the embodiments of the present invention. The electronic device 3 may also include more or fewer other hardware or software than shown, or different component arrangements.

[0081] In some embodiments, the electronic device 3 is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits, programmable gate arrays, digital processors, and embedded devices.

[0082] It should be noted that the electronic device 3 is merely an example. Other existing or future electronic products that are suitable for this application should also be included within the scope of protection of this application and are incorporated herein by reference.

[0083] In some embodiments, the memory 31 stores a computer program that, when executed by the at least one processor 32, implements all or part of the steps in the image recognition-based road cleaning decision-making method described above. The memory 31 includes a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data. Further, the computer-readable storage medium may primarily include a program storage area and a data storage area, wherein the program storage area may store an operating system, at least one application program required for a function, etc.

[0084] In some embodiments, the at least one processor 32 is the control unit of the electronic device 3, connecting various components of the electronic device 3 via various interfaces and lines. It executes programs or modules stored in the memory 31 and calls data stored in the memory 31 to perform various functions and process data. For example, when the at least one processor 32 executes the computer program stored in the memory 31, it implements all or part of the steps of the image recognition-based road cleaning decision-making method described in this application embodiment; or it implements all or part of the functions of the image recognition-based road cleaning decision-making device. The at least one processor 32 may be composed of integrated circuits, such as a single-packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips.

[0085] In some embodiments, the at least one communication bus 33 is configured to enable communication between the memory 31 and the at least one processor 32, etc. Although not shown, the electronic device 3 may also include a power supply (e.g., a battery) to power the various components. Preferably, the power supply can be logically connected to the at least one processor 32 via a power management device, thereby enabling functions such as charging, discharging, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device 3 may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0086] The integrated unit implemented as a software functional module described above can be stored in a computer-readable storage medium. This software functional module, stored in a storage medium, includes several instructions to cause an electronic device (which may be a personal computer, electronic device, or network device, etc.) or processor to execute portions of the methods described in the various embodiments of this application.

[0087] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0088] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0089] The above are all preferred embodiments of the present application, and are not intended to limit the scope of protection of the present application. Therefore, any equivalent changes made based on the structure, shape, and principle of the present application should be included in the scope of protection of the present application.

Claims

1. A road surface cleaning decision-making method based on image recognition, characterized in that, The method comprises: Spatiotemporal alignment and fusion processing is performed on real-time acquired multimodal sensor data to obtain an environmental perception matrix; The environmental perception matrix is ​​subjected to multi-scale feature analysis using a preset lightweight neural network to obtain the road surface type probability distribution vector. The road surface defects are segmented using a pre-set improved U-Net network to obtain a spatial distribution map of road surface damage from real-time acquired binocular vision images. Based on the preset resource constraints, the probability distribution vector of the road surface type, and the spatial distribution map of road damage, a multi-objective optimization solution is performed to obtain the optimal strategy set; The optimal strategy set is subjected to entropy weight-distance dynamic decision-making processing to generate real-time cleaning parameter instructions; Obstacle identification and repulsive force potential field path planning are performed based on the dynamically updated environmental perception matrix to generate a collision-free path point sequence. Adaptive linkage control processing is performed based on the real-time cleaning parameter instructions and the collision-free path point sequence to obtain the cleaning equipment drive signal.

2. The road surface cleaning decision-making method based on image recognition according to claim 1, characterized in that, The lightweight neural network includes the MobileNetV3 model and the SENet model. The step of performing multi-scale feature analysis on the environmental perception matrix using the preset lightweight neural network to obtain the road surface type probability distribution vector includes: The MobileNetV3 model is used to extract edge and texture features from the environment perception matrix to obtain the original road surface feature map. The original road surface feature map is processed by channel attention weighting using the SENet model to obtain an enhanced feature map. The enhanced feature map is processed by a preset Softmax function to make classification decisions, thereby obtaining the road surface type probability distribution vector.

3. The road cleaning decision-making method based on image recognition according to claim 1, characterized in that, The step of segmenting road surface defects in real-time acquired binocular vision images using a preset improved U-Net network to obtain a spatial distribution map of road surface damage includes: The defect segmentation region features are obtained by performing multi-level downsampling feature extraction and upsampling decoding and localization on the real-time acquired binocular vision images through a pre-set improved U-Net network. Based on the binocular disparity information of the binocular vision image, the features of the defect segmentation region are reconstructed in three dimensions to obtain a spatial distribution map of road damage.

4. The road surface cleaning decision-making method based on image recognition according to claim 1, characterized in that, The step of performing multi-objective optimization based on preset resource constraints, the road surface type probability distribution vector, and the road surface damage spatial distribution map to obtain the optimal strategy set includes: Based on a preset objective function, a model is constructed for the probability distribution vector of the road surface type and the spatial distribution map of the road surface damage to obtain a multi-objective optimization model. The multi-objective optimization model is constrained and solved according to preset resource constraints to obtain a set of candidate strategies; The candidate strategy set is guided by reference points and non-dominated by a preset NSGA-III algorithm to obtain the optimal strategy set.

5. The road surface cleaning decision-making method based on image recognition according to claim 1, characterized in that, The step of performing entropy weight-distance dynamic decision-making on the optimal strategy set to generate real-time cleaning parameter instructions includes: Normalize and calculate the information entropy weights of each target index in the optimal strategy set to obtain a multi-target weight vector; The distance score between the superior and inferior solutions of the multi-objective weight vector is calculated according to the preset positive and negative ideal types to obtain the proximity score corresponding to each strategy in the optimal strategy set. The optimal strategy set is sorted and filtered according to the proximity score to obtain the strategy with the highest proximity score. The strategy with the highest proximity score is then processed for instruction mapping to generate real-time cleaning parameter instructions.

6. The road surface cleaning decision-making method based on image recognition according to claim 1, characterized in that, The step of performing obstacle identification and repulsive force potential field path planning based on a dynamically updated environmental perception matrix to generate a collision-free path point sequence includes: Dynamic obstacle distance detection is performed on the dynamically updated environmental perception matrix to obtain obstacle spatial distribution data; A potential field model is constructed based on a preset repulsive field function and the spatial distribution data of the obstacles to obtain a potential energy distribution map. The target-oriented path is iteratively optimized using a preset gradient descent algorithm based on the potential energy distribution map to generate a collision-free path point sequence.

7. The road surface cleaning decision-making method based on image recognition according to claim 1, characterized in that, The adaptive linkage control process based on the real-time cleaning parameter command and the collision-free path point sequence to obtain the cleaning equipment drive signal includes: The real-time cleaning parameter command is parsed and processed to obtain the target control parameters; The collision-free path point sequence is subjected to motion trajectory interpolation to obtain the device pose sequence; Adaptive stability control processing is performed based on the target control parameters and the device pose sequence to obtain the cleaning device drive signal.

8. A road cleaning decision-making device based on image recognition, characterized in that, The device comprises: The perception fusion module is used to perform spatiotemporal alignment and fusion processing on real-time acquired multimodal sensor data to obtain an environmental perception matrix; The road surface analysis module is used to perform multi-scale feature analysis on the environmental perception matrix through a preset lightweight neural network to obtain the road surface type probability distribution vector. The defect identification module is used to segment road surface defects from real-time acquired binocular vision images through a preset improved U-Net network to obtain a spatial distribution map of road surface damage. The cleaning strategy module is used to perform multi-objective optimization based on preset resource constraints, the probability distribution vector of the road surface type, and the spatial distribution map of road surface damage, so as to obtain the optimal strategy set. The cleaning instruction module is used to perform entropy weight-distance dynamic decision processing on the optimal strategy set to generate real-time cleaning parameter instructions. The path planning module is used to identify obstacles and plan paths based on the dynamically updated environmental perception matrix to generate a sequence of collision-free path points. The linkage control module is used to perform adaptive linkage control processing based on the real-time cleaning parameter instructions and the collision-free path point sequence to obtain the cleaning equipment drive signal.

9. An electronic device, characterized in that, The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the image recognition-based road cleaning decision method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the road cleaning decision-making method based on image recognition according to any one of claims 1 to 7.

Citation Information

Cited By

  • Autonomous cleaning method and device for ground stains and storage medium

    CN121392503A

  • Pavement crack detection method and system

    CN121937878A

  • Intelligent cooperative control method and system for multiple actuators of purging, patrolling and gathering environmental sanitation operation

    CN122331312A