Vehicle cleaning control methods, devices, equipment, storage media and program products

By fusing the matching degree of point cloud data and depth images and combining it with color image feature recognition processing, the problem of missing data in areas such as car windows and water mist in dual-arm car wash robots has been solved, generating more complete and reliable vehicle target point cloud data, and improving the cleaning control accuracy and stability of car wash equipment.

CN122573722BActive Publication Date: 2026-11-10上海云骥智行智能科技有限公司
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202611032774.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2026-07-13
Publication Date
2026-11-10
Estimated Expiration
2046-07-13

AI Technical Summary

Technical Problem

In existing technologies, when dual-arm car wash robots use fixed-parameter fusion of LiDAR and RGB-D camera data, data loss is prone to occur in areas such as car windows and water mist, resulting in poor accuracy of point cloud data and affecting the cleaning operation of the car wash equipment.

Method used

By acquiring point cloud data, depth images, and color images of the vehicle's outer surface, the data is fused based on the matching degree. The area to be filled is determined by combining the color image feature recognition processing, and the area is filled using an appropriate filling strategy to generate target point cloud data for controlling the cleaning of the car wash equipment.

Benefits of technology

This improves the accuracy and robustness of vehicle surface point cloud data, ensuring that car wash equipment can complete cleaning operations stably and accurately.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN122573722B_ABST
    Figure CN122573722B_ABST
Patent Text Reader

Abstract

This application provides a vehicle cleaning control method, apparatus, device, storage medium, and program product, relating to the field of vehicle cleaning control. The method includes: acquiring point cloud data, depth images, and color images of the vehicle's outer surface; determining the matching degree between each point cloud data point and depth data at the same location in the depth image based on the point cloud data and depth images; fusing the point cloud data and depth images based on the matching degree to obtain fused point cloud data; performing feature recognition processing on the color image to obtain target features corresponding to the areas to be filled in the fused point cloud data; filling the areas to be filled using a filling strategy corresponding to the target features to obtain target point cloud data of the vehicle; and controlling a car wash device to clean the vehicle based on the target point cloud data. This improves the accuracy of the final obtained vehicle surface point cloud data, thereby enhancing the reliability of the car wash device in cleaning the vehicle surface.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of vehicle cleaning control, and more particularly to a vehicle cleaning control method, apparatus, equipment, storage medium, and program product. Background Technology

[0002] Dual-arm car wash robots typically acquire data from LiDAR and Red Green Blue-D (RGB-D) cameras, respectively. They then fuse the two types of data using fixed fusion parameters to obtain point cloud data of the vehicle surface, which is then used for vehicle cleaning.

[0003] However, the above scheme uses fixed parameters to fuse two different types of data. For interference areas such as car windows and water mist, which are prone to data loss, fusing with fixed parameters may result in the inability to distinguish between different interference areas, leading to poor accuracy of the point cloud data of the vehicle surface, which in turn affects the cleaning operation of the subsequent car wash equipment. Summary of the Invention

[0004] This application provides a vehicle cleaning control method, apparatus, device, storage medium, and program product. It can acquire complete and reliable vehicle body point clouds, improving the accuracy of the acquired vehicle surface point cloud data, thereby enabling car wash equipment to complete cleaning accurately and stably.

[0005] In a first aspect, this application provides a vehicle cleaning control method, the method comprising: acquiring point cloud data, depth images, and color images of the vehicle's outer surface; determining the matching degree between each point cloud data and the depth data at the same location in the depth image based on the point cloud data and the depth image; performing fusion processing on the point cloud data and the depth image based on the matching degree to obtain fused point cloud data; performing feature recognition processing on the color image to obtain target features corresponding to the areas to be filled in the fused point cloud data; using a filling strategy corresponding to the target features to fill the areas to be filled to obtain target point cloud data of the vehicle; and controlling a car wash device to clean the vehicle based on the target point cloud data.

[0006] In one possible embodiment, point cloud data and depth images are fused according to the matching degree to obtain fused point cloud data, including: acquiring point cloud data corresponding to each spatial location on the vehicle surface, and coordinate data determined based on the depth image; if there is no corresponding point cloud data at a spatial location, the coordinate data corresponding to the spatial location is determined as fused point cloud data; if there is no corresponding coordinate data at a spatial location, the point cloud data corresponding to the spatial location is determined as fused point cloud data; if there are corresponding point cloud data and coordinate data at a spatial location, the point cloud data and coordinate data are fused according to the matching degree corresponding to the spatial location to obtain fused point cloud data corresponding to the spatial location.

[0007] In one possible embodiment, point cloud data and coordinate data are fused based on the matching degree corresponding to the spatial location to obtain fused point cloud data corresponding to the spatial location. This includes: determining a first confidence level corresponding to the point cloud data and a second confidence level corresponding to the coordinate data based on the matching degree corresponding to the spatial location; the second confidence level is determined based on the depth image and the color image; and performing a weighted fusion calculation on the point cloud data and coordinate data based on the first confidence level and the second confidence level to obtain fused point cloud data.

[0008] In one possible embodiment, determining the matching degree between each point cloud data and the depth data at the same location in the depth image, based on point cloud data and a depth image, includes: mapping the point cloud data to the camera coordinate system based on the camera intrinsic parameter matrix to obtain mapped point cloud data; mapping the depth data back to the camera coordinate system to obtain coordinate data based on the depth image; calculating the depth difference and normal deviation between the mapped point cloud data and the coordinate data; the depth difference is the Euclidean space distance deviation between the mapped point cloud data and the coordinate data; the normal deviation is the angular deviation between the normal vector corresponding to the mapped point cloud data and the normal vector corresponding to the coordinate data; and determining the matching degree based on the depth difference and normal deviation.

[0009] In one possible embodiment, feature recognition processing is performed on the color image to obtain target features corresponding to the region to be filled in the fused point cloud data, including: voxel rasterization processing of the fused point cloud data to obtain multiple voxel regions; determining the region to be filled that meets preset conditions among the multiple voxel regions; performing feature recognition processing on the color image at the location of the region to be filled to obtain target features; the preset conditions include at least one of the following: the number of voxel regions containing point cloud data is less than a preset number; the voxel regions whose distance from the reference region is greater than a preset distance threshold; the reference region is the region in the fused point cloud data whose number of point cloud data contained in the voxel regions is greater than or equal to a preset number; the corresponding voxel regions in the color image whose texture contrast is less than a preset contrast or whose texture variance is higher than a preset texture variance.

[0010] In one possible embodiment, a filling strategy corresponding to the target feature is used to fill the region to be filled to obtain the target point cloud data of the vehicle. This includes: if the target feature is a low-reflectivity surface feature, the region to be filled is filled using a surface fitting method with image edge constraints to obtain the target point cloud data; if the target feature is a water mist interference feature and / or a specular interference feature, the region to be filled is filled based on at least one set of point cloud data acquired before the current time and the point cloud data corresponding to the region to be filled to obtain the target point cloud data; if the target feature is a small defect feature, the region to be filled is filled using a radial basis function interpolation method to obtain the target point cloud data.

[0011] In one possible embodiment, the target point cloud data includes point cloud data at each spatial location, confidence levels at each spatial location, and target features at each spatial location; the method further includes: generating a confidence mask based on the confidence levels of the target point cloud data; and controlling a car wash device to clean the vehicle based on the confidence mask.

[0012] Secondly, this application provides a vehicle cleaning control device, comprising: an acquisition module for acquiring point cloud data, depth images, and color images of the vehicle's outer surface; a calculation module for determining the matching degree between the point cloud data and the depth data at the same location in the depth image based on the point cloud data and the depth image; a fusion module for fusing the point cloud data and the depth image based on the matching degree to obtain fused point cloud data; an extraction module for performing feature recognition processing on the color image to obtain target features corresponding to the areas to be filled in the fused point cloud data; a completion module for filling the areas to be filled using a filling strategy corresponding to the target features to obtain target point cloud data of the vehicle; and a control module for controlling a car wash device to clean the vehicle based on the target point cloud data.

[0013] Thirdly, this application provides a vehicle cleaning control system, including a control device and a car wash device, wherein the control device is communicatively connected to the car wash device; the control device is used to send control commands to the car wash device; the car wash device or the control device is used to execute the method provided above, determine the target point cloud data of the vehicle, and generate control commands based on the target point cloud data to control the cleaning of the vehicle.

[0014] Fourthly, this application provides a car wash device, including a body, at least one cleaning arm, a memory, and a processor; the body is connected to the cleaning arm; the cleaning arm is used to clean the surface of a vehicle; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory, causing the processor to perform the method provided above.

[0015] Fifthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method provided above.

[0016] Sixthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the method provided above.

[0017] The vehicle cleaning control method, apparatus, equipment, storage medium, and program products provided in this application acquire point cloud data, depth images, and color images of the vehicle's outer surface. They then fuse the point cloud data and depth images based on the matching degree between depth data at the same location in the point cloud data and depth images. This improves the consistency of multi-source 3D data and yields fused point cloud data. Furthermore, by performing feature recognition processing on the color images, target features corresponding to the areas to be filled in the fused point cloud data are obtained. A filling strategy corresponding to the target features is then used to fill the areas to be filled. This improves the accuracy, robustness, and real-time performance of point cloud hole filling on the vehicle surface, resulting in more complete and reliable vehicle target point cloud data. This supports more stable and accurate cleaning control of vehicles by car wash equipment. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0019] Figure 1 A flowchart illustrating the vehicle cleaning control method provided in this application;

[0020] Figure 2 This is a flowchart illustrating the point cloud completion method provided in this application.

[0021] Figure 3 A schematic diagram of the vehicle cleaning control device provided in this application;

[0022] Figure 4 This is a structural diagram of the car wash equipment provided in this application.

[0023] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0024] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application.

[0025] Vehicle surface 3D perception and cleaning control technology is commonly used in car wash equipment (such as dual-arm car wash robots). Car wash equipment is generally equipped with LiDAR, depth cameras, and color imaging units to collect spatial and image information of the vehicle's outer surface, and uses the perception results for cleaning path layout, etc.

[0026] In existing vehicle cleaning control methods, point cloud data and depth images of the vehicle's outer surface are usually acquired first, and then a three-dimensional model of the vehicle surface is generated by direct overlay, multi-frame accumulation or fixed weighting. Some methods may also combine color images to guide the completion of missing point cloud areas. The essence of the above methods is to use the ranging results of different sensors to complement each other in order to improve the completeness of surface modeling.

[0027] However, in real-world car wash environments, areas such as vehicle windows, sunroofs, and door edges are prone to low reflectivity, obstruction, or scattering interference. Depth results obtained from different sensors at the same location often show significant discrepancies. The aforementioned methods lack effective judgment on the reliability of point cloud data and corresponding depth image data, leading to calculations based on conflicting data. Furthermore, if the area to be completed is processed uniformly, it's difficult to distinguish between glass areas, water mist areas, or other surface areas, resulting in poor accuracy of the final point cloud data and potential completion distortions. This negatively impacts subsequent cleaning operations by the car wash equipment.

[0028] In view of this, this application provides a vehicle cleaning control method. After acquiring point cloud data, depth image and color image of the vehicle's outer surface, the matching degree is first determined based on the depth data at the same position in the point cloud data and the depth image. Then, the point cloud data and depth image are fused according to the matching degree, and feature recognition processing is performed on the color image to obtain the target features corresponding to the area to be filled. Subsequently, the area to be filled is filled using a filling strategy corresponding to the target features to obtain the target point cloud data of the vehicle. Based on the target point cloud data, the car wash equipment is controlled to clean the vehicle.

[0029] In this way, the method simultaneously collects vehicle body point clouds, depth images, and color images. First, it calculates the matching degree through 3D data at the same location to achieve multi-sensor point cloud fusion. Then, it extracts regional features based on color images to classify hole types and matches differentiated filling strategies to complete the vehicle body hole filling, generating a complete and reliable vehicle body target point cloud. Vehicle contour reconstruction does not need to rely on single sensor data and can specifically repair data loss caused by interference from windows, water mist, etc., reducing the distortion problem caused by a single data source, completely restoring the 3D shape of the vehicle body, and effectively improving the vehicle body recognition accuracy and cleaning control stability of intelligent car wash equipment.

[0030] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0031] Figure 1 A flowchart illustrating the vehicle cleaning control method provided in this application. The method includes the following steps:

[0032] S101. Acquire point cloud data, depth images, and color images of the vehicle's outer surface.

[0033] Point cloud data refers to spatial point data used to represent the geometric shape of the vehicle's outer surface, with each point including at least three-dimensional coordinate information.

[0034] Depth images are image data representing the distance distribution on a measured surface. Color images are image data used to represent the color and texture information of a vehicle's exterior surface.

[0035] Point cloud data, depth images, and color images can be acquired by multiple sensors, such as point cloud data acquired by LiDAR and depth and color images acquired by an RGB-D camera.

[0036] In practice, acquisition commands are sent to the LiDAR and RGB-D camera. The LiDAR acquires point cloud data, and the RGB-D camera acquires depth and color images, thus completing the acquisition of multi-source sensing data.

[0037] S102. Based on the point cloud data and depth image, determine the matching degree between each point cloud data and the depth data at the same location in the depth image.

[0038] Among them, the matching degree is a quantitative indicator used to represent the degree of consistency between point cloud data and depth image at the same location on the vehicle surface. The larger the value, the higher the consistency, and the smaller the value, the higher the possibility of deviation between the two.

[0039] The same position refers to the pixel position in the depth image corresponding to a spatial point in the point cloud data after coordinate transformation and projection.

[0040] In one possible implementation, the matching degree between each point cloud data and the depth data at the same location in the depth image is determined based on the point cloud data and the depth image. This includes: mapping the point cloud data to the camera coordinate system based on the camera intrinsic parameter matrix to obtain mapped point cloud data; mapping the depth data back to the camera coordinate system to obtain coordinate data based on the depth image; calculating the depth difference and normal deviation between the mapped point cloud data and the coordinate data; the depth difference is the Euclidean space distance deviation between the mapped point cloud data and the coordinate data; the normal deviation is the angular deviation between the normal vector corresponding to the mapped point cloud data and the normal vector corresponding to the coordinate data; and determining the matching degree based on the depth difference and normal deviation.

[0041] Specifically, spatial deviation and orientation deviation are calculated for point cloud data and depth image data at the same location, and a matching degree value is generated based on a preset threshold or weighting relationship. When the spatial deviation and normal deviation are small, the matching degree is high, indicating that the two types of data are consistent in geometric position and local surface morphology. If there is reflection, occlusion, or noise interference at a certain location, the corresponding depth difference or normal deviation increases, and the matching degree decreases accordingly. This matching degree is used to represent the reliability of the corresponding data.

[0042] For example, the process of determining the matching degree can be as follows:

[0043] It is understandable that point cloud data contains the three-dimensional coordinates of each individual point in the lidar coordinate system. Based on rotation matrix Translation vector The point cloud data is transformed to obtain the three-dimensional coordinates of each point in the camera coordinate system. Corresponding mapping point cloud coordinates The calculation formula is as follows:

[0044] ;

[0045] in, This is the vector dot product operator.

[0046] Then, using the camera intrinsic parameter matrix, the mapped point cloud coordinates are projected onto the image plane to obtain the corresponding pixel coordinates, as shown in the following formula:

[0047] ;

[0048] in, This is the camera intrinsic parameter matrix; , These are the horizontal and vertical coordinates of the pixels corresponding to the mapped point cloud coordinates projected onto the image plane.

[0049] It is understandable that mapped point cloud data is based on the three-dimensional coordinates of each individual point. Corresponding mapping point cloud coordinates Determined, i.e., the coordinates of the mapped point cloud. The data is aggregated to form a mapped point cloud.

[0050] It is understandable that this is based on pixel coordinates ( , It can extract depth data at corresponding locations in a depth image. .

[0051] Based on the inverse matrix of the camera intrinsic parameter matrix For depth data Perform a reverse coordinate mapping to obtain the pixel coordinates in the camera coordinate system. , The corresponding three-dimensional coordinates The calculation formula is as follows:

[0052] ;

[0053] It is understandable that the coordinate data corresponding to the depth image consists of the coordinates of each pixel ( , The corresponding three-dimensional coordinates Determined, i.e., the coordinates of each pixel ( , The corresponding three-dimensional coordinates The coordinate data is then compiled based on the depth image.

[0054] Furthermore, based on the mapped point cloud coordinates and three-dimensional coordinates Determine the depth difference The calculation formula is as follows:

[0055] ;

[0056] in, This represents the L2 norm, used to calculate the Euclidean distance between two points.

[0057] Meanwhile, based on the mapped point cloud coordinates The corresponding unit normal vector and three-dimensional coordinates The corresponding unit normal vector Determine the normal deviation The calculation formula is as follows:

[0058] ;

[0059] Furthermore, based on depth difference Sum and normal deviation Calculate the matching degree The calculation formula is as follows:

[0060] ;

[0061] in, It is a natural exponential function; The standard deviation of the Gaussian kernel for location differences can be, for example, 0.02m. The standard deviation of the Gaussian kernel for the normal difference can be, for example, 0.2.

[0062] This allows the matching degree to simultaneously reflect both positional consistency and surface orientation consistency, resulting in more accurate judgments for conflicting data. Furthermore, it clarifies the depth correspondence of the vehicle surface in glass, edges, and partially occluded areas, improving the stability and continuity of the target point cloud data.

[0063] S103. Based on the matching degree, the point cloud data and depth image are fused to obtain fused point cloud data.

[0064] Among them, fusion processing refers to processing the data at corresponding positions of point cloud data and depth image according to the matching degree under the same coordinate system, and outputting unified three-dimensional data describing the geometric shape of the vehicle's outer surface.

[0065] Fusion point cloud data refers to point cloud data formed after consistency judgment and data integration, which can be directly used for vehicle contour recognition, surface completion and car wash trajectory layout.

[0066] Specifically, the matching degree is used as input to fuse point cloud data and depth image data at corresponding locations. A high matching degree indicates that the measurement results from the two types of sensors at the same surface location are similar, and the fusion result can be generated by combining their information. A low matching degree indicates significant measurement conflicts at the current location, and the impact of conflicting data on the fusion result can be reduced. If necessary, conflicting locations are marked as areas to be filled instead of being directly included in the final surface. For low-consistency regions or regions with unstable observations, erroneous surfaces are not forcibly generated; instead, these regions are left as gaps and marked as areas to be filled. Based on this, instead of simply overlaying the original point cloud and depth image, the matching degree is used to constrain the data consistency at different locations, ensuring that high-consistency regions form stable surfaces, while low-consistency regions enter the completion process. This reduces contour distortion and surface breaks caused by directly writing conflicting observations into the model.

[0067] In one possible implementation, point cloud data and depth images are fused based on the matching degree to obtain fused point cloud data. This includes: acquiring point cloud data corresponding to each spatial location on the vehicle surface, as well as coordinate data determined based on the depth image; if there is no corresponding point cloud data at a spatial location, the coordinate data corresponding to that spatial location is determined as fused point cloud data; if there is no corresponding coordinate data at a spatial location, the point cloud data corresponding to that spatial location is determined as fused point cloud data; if there are corresponding point cloud data and coordinate data at a spatial location, the point cloud data and coordinate data are fused based on the matching degree corresponding to the spatial location to obtain the fused point cloud data corresponding to that spatial location.

[0068] Specifically, for each spatial location, point cloud data corresponding to the LiDAR and coordinate data obtained by reverse mapping from the depth image are extracted. For locations where only coordinate data exists, the coordinate data is directly written into the fused point cloud data; for locations where only point cloud data exists, the point cloud data is directly retained as the fusion result; for locations where both types of data exist, the two types of data are weighted and fused based on the matching degree corresponding to the location to generate the fused point cloud data for that location.

[0069] In this way, the missing point cloud locations on the vehicle surface can be filled in by the depth image, the missing coordinate locations can be filled in by the point cloud, and for locations where both point cloud and depth image exist, fusion is completed based on the matching degree, making the fused point cloud data more suitable for filling point cloud holes and vehicle cleaning control in terms of completeness and consistency.

[0070] In one possible implementation, point cloud data and coordinate data are fused based on the matching degree corresponding to the spatial location to obtain fused point cloud data at the spatial location. This includes: determining a first confidence level corresponding to the point cloud data and a second confidence level corresponding to the coordinate data based on the matching degree corresponding to the spatial location; the second confidence level is determined based on the depth image and the color image; and performing a weighted fusion calculation on the point cloud data and coordinate data based on the first confidence level and the second confidence level to obtain fused point cloud data.

[0071] Specifically, the matching degree is mapped to the corresponding first confidence level and second confidence level. For locations with higher reliability of point cloud data, the weight of the point cloud data is increased; for locations with higher reliability of coordinate data, the weight of the coordinate data is increased. The weighted fusion calculation can be implemented using a linear weighting method, that is, the result of multiplying the point cloud data by the first confidence level and then adding it to the result of multiplying the coordinate data by the second confidence level, to obtain the fused point cloud data at that spatial location.

[0072] For example, the process of determining the first confidence level, the second confidence level, and completing the weighted fusion can be as follows:

[0073] It is understandable that the first confidence level is the confidence weight of the lidar side. The larger this value, the more reliable the point cloud data measurement results.

[0074] Based on normalized echo intensity of lidar Matching degree The first confidence level can be calculated using the following formula:

[0075] ;

[0076] in, It is a sigmoid activation function used to map the calculation results to the interval (0,1). This is the echo intensity gain coefficient, with a value of 1.0.

[0077] It is understandable that the second confidence level is the confidence weight of the RGB-D side. The larger this value, the more reliable the measurement results of the coordinate data corresponding to the depth image.

[0078] Based on depth-validation flags Local texture variance of corresponding pixels in a color image and matching degree The second confidence level can be calculated using the following formula:

[0079] ;

[0080] in, The value can be 0 or 1, where 1 indicates that the current depth data is valid; Used to characterize the strength of interference such as water mist and highlights; The standard deviation of the Gaussian kernel for texture variance is 0.1.

[0081] Furthermore, based on the first confidence level With second confidence level Calculate the total fusion weight The calculation formula is as follows:

[0082] ;

[0083] Here, ε is a very small positive number, used to avoid calculation errors caused by a denominator of 0.

[0084] Understandable, based on the first confidence level Second confidence level After completing the weighted fusion, the fused point cloud coordinates of the corresponding spatial locations are obtained. The calculation formula is as follows:

[0085] ;

[0086] It is understandable that the fused point cloud data is based on the fused point cloud coordinates. Determined, i.e., the coordinates of the merged point cloud. The data is aggregated to form a fused point cloud.

[0087] This allows more reliable data to occupy a larger proportion of the fusion result, making the fused point cloud data at spatial locations more consistent with the true geometry of the vehicle surface and reducing the impact of outliers from a single sensor on the fusion result. Since the second confidence level combines the joint judgment of depth and color images, the reliability of the coordinate data can be more accurately represented, thereby improving the stability and consistency of the fused point cloud data in complex car wash environments.

[0088] S104. Perform feature recognition processing on the color image to obtain the target features corresponding to the area to be filled in the fused point cloud data.

[0089] Feature recognition processing refers to using information such as color, texture, edge, and region distribution in color images to determine the surface attributes of the marked areas to be filled in the fused point cloud data.

[0090] Target features refer to the categories used to represent the regions to be filled.

[0091] In one possible implementation, feature recognition processing is performed on the color image to obtain the target features corresponding to the region to be filled in the fused point cloud data. This includes: performing voxel rasterization processing on the fused point cloud data to obtain multiple voxel regions; determining the region to be filled that meets preset conditions among the multiple voxel regions; performing feature recognition processing on the color image at the location of the region to be filled to obtain the target features; the preset conditions include at least one of the following: a voxel region containing fewer than a preset number of point cloud data; a voxel region whose distance from a reference region is greater than a preset distance threshold; the reference region being a voxel region in the fused point cloud data containing more than or equal to a preset number of point cloud data; and a voxel region in the corresponding color image whose texture contrast is less than a preset contrast or whose texture variance is higher than a preset texture variance.

[0092] For example, the process of identifying the region to be filled and determining the corresponding target features can be as follows:

[0093] The fused point cloud data is processed by voxel rasterization. The voxel raster size is set to, for example, 0.02m, and the fused point cloud data is divided into several uniform voxel regions.

[0094] It is understandable that the regions to be filled are selected from all voxel regions based on preset conditions. A voxel region is determined to be a region to be filled when it meets at least one of the following criteria:

[0095] The number of point clouds contained within a single voxel region is less than the preset number. ,in For example, the value can be 5;

[0096] The distance between the voxel region and the high-density reference region is greater than a preset distance threshold. ,in For example, the value can be 0.05m;

[0097] The voxel region corresponds to a color image region that is either a low texture contrast region or a high texture variance region. A low texture contrast region is a region in the color image where the texture contrast is less than a preset contrast, and a high texture variance region is a voxel region where the texture variance is greater than a preset texture variance.

[0098] It is understandable that low texture contrast areas and high texture variance areas usually correspond to interference scenes such as car windows and water mist, which are prone to point cloud loss.

[0099] It is understandable that after filtering out all voxel regions that meet the preset conditions, the region to be filled is generated.

[0100] Furthermore, semantic feature recognition processing is performed on the color image corresponding to the region to be filled. For example, a lightweight semantic segmentation network can be used to classify the color image pixel by pixel to obtain a pixel-level semantic label map. Pixel semantic labels include features for low reflectivity surfaces, features for water mist interference and / or specular interference, and features for small defects.

[0101] Among them, low-reflectivity surface features refer to glass areas such as vehicle windows and sunroofs. These areas have weak surface reflectivity, which can easily cause ranging failures of lidar and RGB-D equipment, resulting in gaps in point clouds.

[0102] Water mist interference and / or specular highlight interference features refer to areas on the vehicle body surface where water mist adheres and strong light creates specular highlights. These areas have messy image textures and significant ranging noise, which can cause local point cloud loss.

[0103] Small defects refer to localized point cloud missing areas with no fixed pattern, such as gaps in the vehicle body, tiny dents, and gaps between parts.

[0104] It can be understood that by projecting each region to be filled onto the image plane, determining the number of pixels of various semantic labels in the color image corresponding to the region to be filled, and selecting the semantic label with the highest pixel ratio as the target feature corresponding to the region to be filled.

[0105] In this way, by discretizing the fused point cloud data into voxel regions and combining the number of point clouds, spatial distance, and image texture to jointly select the regions to be filled, the target features can be kept consistent with the local surface state.

[0106] S105. Using a filling strategy corresponding to the target features, fill the area to be filled to obtain the target point cloud data of the vehicle, and control the car wash equipment to clean the vehicle based on the target point cloud data.

[0107] Among them, the filling strategy refers to the geometric reconstruction rules or point cloud generation method corresponding to the target features of the region to be filled.

[0108] Target point cloud data refers to the three-dimensional data of the vehicle's outer surface formed by regional completion based on fused point cloud data, which can be used for actual cleaning control.

[0109] Specifically, for regions with different target characteristics, a method adapted to the surface properties of the region can be used for filling to ensure that the generated result is as continuous as possible with the surrounding known surfaces and to avoid erroneous extensions across obvious boundaries. After filling is completed, the processing module can perform a consistency check on the newly generated points and the original fused points, and delete or adjust the fill points that do not meet the conditions, ultimately forming the target point cloud data of the vehicle.

[0110] In one possible implementation, a filling strategy corresponding to the target feature is used to fill the region to be filled to obtain the target point cloud data of the vehicle. This includes: if the target feature is a low-reflectivity surface feature, the region to be filled is filled using a surface fitting method with image edge constraints to obtain the target point cloud data; if the target feature is a water mist interference feature and / or a specular interference feature, the region to be filled is filled based on at least one set of point cloud data acquired before the current time and the point cloud data corresponding to the region to be filled to obtain the target point cloud data; if the target feature is a small defect feature, the region to be filled is filled using a radial basis function interpolation method to obtain the target point cloud data.

[0111] For example, the filling process for the three types of features can be implemented as follows:

[0112] It is understandable that low-reflectivity surface features correspond to glass areas such as car windows and sunroofs. These areas have weak optical reflection characteristics, which can easily lead to measurement failures when the LiDAR and RGB-D camera are synchronized, resulting in large-area regular point cloud holes. Therefore, a surface fitting method with image edge constraints is used to fill the holes.

[0113] The input to this method includes the region of the hole to be filled, the known point cloud within the edge of the hole (e.g., 3cm), and the edge gradient information of the color image. Specifically, based on the known point cloud within the edge of the hole, a least-squares method is used to fit and obtain the geometric model of the vehicle surface. A planar model or a quadratic surface model is adaptively selected according to the local surface morphology; the formula for the planar model is:

[0114] ;

[0115] The formula for the quadratic surface model is:

[0116] ;

[0117] in, , , It is a three-dimensional rectangular coordinate system; , , , , , The parameters for surface fitting can be obtained by solving for the known point cloud data at the edge of the hole.

[0118] Then, uniform grid sampling is performed in the area to be filled to generate an initial filling point cloud, and small Gaussian noise is superimposed on the coordinates of the filling point cloud to simulate the roughness of the real surface of the car body; at the same time, the edge gradient information of the color image is combined to constrain the filling boundary, suppress the step distortion of the filling result, and finally generate target point cloud data that fits the shape of the real car body.

[0119] It is understandable that water mist interference and highlight interference features mainly correspond to areas of the car body covered by water mist, foam, or direct sunlight in car wash scenarios, resulting in highlight reflections. The RGB-D depth data of these areas is severely affected by environmental interference and has low reliability; therefore, a time-series cumulative filtering method is used to fill the holes.

[0120] The input to this method includes the area to be filled and at least one set (e.g., five sets) of historical LiDAR point cloud data prior to the current moment. The specific filling process is as follows: First, only the point cloud data within the area to be filled is retained; then, the point cloud data corresponding to the spatial location in at least one set of historical point cloud data is retrieved, and the coordinates corresponding to the current spatial location are updated using the median of multiple sets of data to complete the filling; if there is no point cloud data for the corresponding location in at least one set of historical point cloud data, a surface fitting method based on low reflectivity can be used for filling, and the fitting error can be appropriately increased to accommodate slight deformation deviations in the interference area, ultimately generating stable and reliable target point cloud data.

[0121] It can be understood that small-scale defect features correspond to small, scattered local voids, specifically tiny point cloud missing regions with a size smaller than a preset size (e.g., 5cm × 5cm voxels). For this type of small-scale void, a cubic radial basis function interpolation method is used to achieve local smooth filling.

[0122] The input for this method includes the known point cloud of the region to be filled and its edges. The formula for the cubic radial basis interpolation function is as follows:

[0123] ;

[0124] in, The coordinates are for interpolation output; The three-dimensional coordinates of the position to be interpolated; This represents the total number of known points on the edge of the region to be filled. The three-dimensional coordinates of the j-th known edge point; The interpolation weight coefficients are the values ​​corresponding to the j-th known edge point. It is a cubic radial basis kernel function. The three-dimensional coordinates of the location to be interpolated and the Euclidean distance between the known edge points; These are linear polynomial terms used to smooth the interpolation results. After completion, the confidence level of all interpolated coordinates is set to a fixed confidence level (e.g., 0.5).

[0125] This allows for the use of appropriate completion models for regions to be filled based on different causes, thereby obtaining target point cloud data with higher continuity and more stable morphology.

[0126] Then, based on the target point cloud data, vehicle surface information is extracted to further generate the cleaning trajectory of the car wash equipment.

[0127] In one possible implementation, the target point cloud data includes point cloud data at each spatial location, confidence at each spatial location, and target features at each spatial location; the method further includes: generating a confidence mask based on the confidence of the target point cloud data; and controlling the car wash equipment to clean the vehicle based on the confidence mask.

[0128] Specifically, car wash equipment can constrain the cleaning range, spray trajectory, brush head contact path, or robotic arm movement boundaries based on a confidence mask. This ensures that the actuator performs preset cleaning actions only on the effective area and uses deceleration, avoidance, or paused approach control methods for restricted areas. If the confidence mask in the target point cloud data shows consecutive low confidence at certain locations, the controller can remove that location from the current cleaning arrangement or limit the movement amplitude of that area within a safe threshold, allowing the equipment to complete cleaning coverage along the trusted area.

[0129] For example, the confidence mask generation process can be as follows:

[0130] It is understandable that the target point cloud data contains Each point corresponds to three types of information: three-dimensional coordinates, confidence level, and target features. The overall representation is as follows:

[0131] ;

[0132] in, Let be the three-dimensional coordinates corresponding to the i-th point. Let be the confidence level corresponding to the i-th point. Let be the semantic label corresponding to the i-th point.

[0133] For locations where point cloud data, depth images, and color images coexist. To merge point cloud coordinates , Total fusion weight For locations where only point cloud data exists, Mapping point cloud coordinates , First confidence level For locations where only color and depth images exist, Three-dimensional coordinates For locations where point cloud data, depth images, and color images are all absent, The coordinates are generated by the corresponding filling strategy. For fixed confidence levels.

[0134] Furthermore, based on the confidence level of each point within the target point cloud data... Generate a binary confidence mask The mask distinguishes between high-confidence and low-confidence points. High-confidence points are marked as effective clean areas, while low-confidence points are marked as restricted areas requiring careful avoidance and deceleration. The generated confidence mask... Used for controlling the movement path and cleaning strategy of subsequent car wash equipment.

[0135] This allows cleaning control to be built directly on point cloud data that has been filtered for reliability, thus ensuring that the equipment's operating range is consistent with the actual state of the vehicle surface, improving the stability of cleaning control on complex surfaces, and reducing false cleaning of low-reliability areas.

[0136] Therefore, the vehicle cleaning control method of this application quantifies the depth consistency between the corresponding positions of point cloud data and depth images, introduces the quantization result into the fusion process, and then distinguishes the surface attributes of the area to be filled by combining the color image. Filling processing adapted to different area properties is then performed accordingly, ultimately generating target point cloud data that can be directly used for cleaning control. This implementation connects sensor data consistency judgment, area feature recognition, and completion into a continuous processing link. Even in car wash environments with low reflection, occlusion, scattering, and water mist interference, it can still maintain the continuity and controllability of the three-dimensional representation of the vehicle surface, and allows the trajectory layout and attitude adjustment of the car wash equipment to be based on data that more closely resembles the actual vehicle body shape.

[0137] Figure 2 A schematic diagram of the point cloud completion method provided in this application.

[0138] S201. Input point cloud data, depth image, and color image.

[0139] S202. Perform consistency verification on the input point cloud data and depth image, and calculate the point-by-point confidence scores for the two types of sensors respectively.

[0140] S203. Based on the location confidence of each point, perform weighted calculation to generate fused point cloud data.

[0141] S204. Identify the missing areas to be filled in the point cloud from the fused point cloud data, distinguish the hole types by combining the semantic features of the color image, and match the corresponding differentiated filling strategy to complete the hole filling.

[0142] S205. Integrate the original fusion points and the filled composite points to obtain complete target point cloud data. Generate a confidence mask based on the confidence of each point location, and output the target point cloud data and the confidence mask.

[0143] Figure 3 A schematic diagram of the vehicle cleaning control device provided in this application is shown below. Figure 3 As shown, the vehicle cleaning control device 30 provided in this embodiment includes:

[0144] The acquisition module 301 acquires point cloud data, depth images, and color images of the vehicle's outer surface;

[0145] The calculation module 302 is used to determine the matching degree between each point cloud data and the depth data at the same location in the depth image based on the point cloud data and the depth image.

[0146] The fusion module 303 is used to fuse point cloud data and depth image according to the matching degree to obtain fused point cloud data;

[0147] Extraction module 304 is used to perform feature recognition processing on color images to obtain target features corresponding to the areas to be filled in the fused point cloud data;

[0148] The completion module 305 is used to fill the area to be filled using a filling strategy corresponding to the target features, so as to obtain the target point cloud data of the vehicle.

[0149] The control module 307 is used to control the car wash equipment to clean the vehicle based on the target point cloud data.

[0150] In one possible implementation, the acquisition module 301 is further configured to acquire point cloud data corresponding to each spatial location on the vehicle surface, as well as coordinate data determined based on the depth image.

[0151] The fusion module 303 is also used to determine the coordinate data corresponding to the spatial location as the fused point cloud data if there is no corresponding point cloud data at the spatial location.

[0152] The fusion module 303 is also used to determine the point cloud data corresponding to the spatial location as the fused point cloud data if there is no corresponding coordinate data at the spatial location.

[0153] The fusion module 303 is also used to perform fusion processing on the point cloud data and coordinate data according to the matching degree of the spatial location if there are corresponding point cloud data and coordinate data at the spatial location, so as to obtain the fused point cloud data corresponding to the spatial location.

[0154] In one possible implementation, the fusion module 303 is further configured to determine a first confidence level corresponding to the point cloud data and a second confidence level corresponding to the coordinate data based on the matching degree corresponding to the spatial location; the second confidence level is determined based on the depth image and the color image.

[0155] The fusion module 303 is also used to perform weighted fusion calculations on point cloud data and coordinate data based on a first confidence level and a second confidence level to obtain fused point cloud data.

[0156] In one possible implementation, the calculation module 302 is also used to map the point cloud data to the camera coordinate system based on the camera intrinsic parameter matrix to obtain the mapped point cloud data.

[0157] The calculation module 302 is also used to reverse map the depth data to the camera coordinate system to obtain coordinate data based on the depth image;

[0158] The calculation module 302 is also used to calculate the depth difference and normal deviation between the mapped point cloud data and the coordinate data; the depth difference is the Euclidean space distance deviation between the mapped point cloud data and the coordinate data; the normal deviation is the angle deviation between the normal vector corresponding to the mapped point cloud data and the normal vector corresponding to the coordinate data.

[0159] The calculation module 302 is also used to determine the matching degree based on the depth difference and normal deviation.

[0160] In one possible implementation, the extraction module 304 is also used to perform voxel rasterization processing on the fused point cloud data to obtain multiple voxel regions.

[0161] The extraction module 304 is also used to determine the region to be filled that meets preset conditions in multiple voxel regions;

[0162] The extraction module 304 is also used to perform feature recognition processing on the color image at the location of the area to be filled to obtain target features;

[0163] The preset conditions include at least one of the following:

[0164] The number of point cloud data contained is less than the preset number of voxel regions;

[0165] A voxel region whose distance from the reference region is greater than a preset distance threshold; the reference region is the region in the fused point cloud data whose voxel region contains a number of point cloud data greater than or equal to a preset number.

[0166] The corresponding color image contains voxel regions whose texture contrast is less than the preset contrast or whose texture variance is higher than the preset texture variance.

[0167] In one possible implementation, the completion module 305 is also used to fill the area to be filled by using a surface fitting method with image edge constraints if the target feature is a feature of a low-reflectivity surface class.

[0168] The completion module 305 is also used to fill in the area to be filled based on at least one set of point cloud data obtained before the current time and the point cloud data corresponding to the area to be filled if the target feature is a feature of water mist interference and / or specular light interference.

[0169] The completion module 305 is also used to fill in the region to be filled if the target feature is a small defect type feature using the radial basis function interpolation method;

[0170] The completion module 305 is also used to integrate the composite points generated after completion with the original fusion points to obtain the target point cloud data of the vehicle.

[0171] In one possible implementation, the target point cloud data includes the point cloud coordinates of each spatial location, the confidence level of each spatial location, and the target features of each spatial location.

[0172] The vehicle cleaning control device 30 also includes a generation module 306;

[0173] The generation module 306 is used to generate a confidence mask based on the confidence of the target point cloud data;

[0174] The control module 307 is also used to constrain the cleaning range, spray trajectory, and robotic arm movement boundaries of the car wash equipment based on a confidence mask, so as to achieve automated and safe cleaning of vehicles.

[0175] The vehicle cleaning control device provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0176] This application provides a vehicle cleaning control system, including a control device and a car wash device, wherein the control device is communicatively connected to the car wash device; the control device is used to send control commands to the car wash device; the car wash device or the control device is used to execute the method provided above, determine the target point cloud data of the vehicle, and generate control commands based on the target point cloud data to control the cleaning of the vehicle.

[0177] This application provides a car wash device, including a body, at least one cleaning arm, a memory, and a processor; the body is connected to the cleaning arm; the cleaning arm is used to clean the surface of a vehicle; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory, causing the processor to perform the method provided above.

[0178] Figure 4 This is a structural diagram of the car wash equipment provided in this application. Figure 4 As shown, the car wash equipment 40 provided in this embodiment includes at least one processor 401 and a memory 402. Optionally, the car wash equipment 40 also includes a communication component 403. The processor 401, memory 402, and communication component 403 are connected via a bus 404.

[0179] In the specific implementation process, at least one processor 401 executes computer execution instructions stored in memory 402, causing at least one processor 401 to execute the above-described vehicle cleaning control method.

[0180] The specific implementation process of processor 401 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0181] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0182] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0183] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0184] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0185] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0186] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0187] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0188] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0189] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0190] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0191] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0192] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0193] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope.

Claims

1. A vehicle cleaning control method, characterized in that, The method includes: Acquire point cloud data, depth images, and color images of the vehicle's outer surface; Determining the matching degree between each point cloud data and the depth data at the same location in the depth image based on the point cloud data and the depth image includes: mapping the point cloud data to the camera coordinate system based on the camera intrinsic parameter matrix to obtain mapped point cloud data; mapping the depth data back to the camera coordinate system to obtain coordinate data based on the depth image; calculating the depth difference and normal deviation between the mapped point cloud data and the coordinate data; the depth difference is the Euclidean space distance deviation between the mapped point cloud data and the coordinate data; the normal deviation is the angular deviation between the normal vector corresponding to the mapped point cloud data and the normal vector corresponding to the coordinate data; and determining the matching degree based on the depth difference and the normal deviation. Based on the matching degree, the point cloud data and the depth image are fused to obtain fused point cloud data; The color image is subjected to feature recognition processing to obtain the target features corresponding to the region to be filled in the fused point cloud data; Using a filling strategy corresponding to the target features, the area to be filled is filled to obtain the target point cloud data of the vehicle, and the car wash equipment is controlled to clean the vehicle based on the target point cloud data; The step of filling the area to be filled using a filling strategy corresponding to the target feature to obtain the target point cloud data of the vehicle includes: If the target feature is a low-reflectivity surface feature, the area to be filled is filled using a surface fitting method with image edge constraints to obtain the target point cloud data; If the target feature is a water mist interference type and / or a specular interference type feature, the area to be filled is filled based on at least one set of point cloud data obtained before the current time and the point cloud data corresponding to the area to be filled, to obtain the target point cloud data; If the target feature is a small defect type, the radial basis function interpolation method is used to fill the area to be filled to obtain the target point cloud data.

2. The method according to claim 1, characterized in that, The step of fusing the point cloud data and the depth image based on the matching degree to obtain fused point cloud data includes: For each spatial location on the vehicle surface, point cloud data corresponding to the spatial location and coordinate data determined based on the depth image are acquired respectively. If there is no corresponding point cloud data at the spatial location, then the coordinate data corresponding to the spatial location is determined as the fused point cloud data; If there is no corresponding coordinate data at the spatial location, then the point cloud data corresponding to the spatial location is determined as fused point cloud data; If there are corresponding point cloud data and coordinate data at the spatial location, then the point cloud data and coordinate data are fused according to the matching degree corresponding to the spatial location to obtain the fused point cloud data corresponding to the spatial location.

3. The method according to claim 2, characterized in that, The step of fusing the point cloud data and the coordinate data according to the matching degree corresponding to the spatial location to obtain the fused point cloud data corresponding to the spatial location includes: Based on the matching degree corresponding to the spatial location, a first confidence level corresponding to the point cloud data and a second confidence level corresponding to the coordinate data are determined; the second confidence level is determined based on the depth image and the color image. Based on the first confidence level and the second confidence level, a weighted fusion calculation is performed on the point cloud data and the coordinate data to obtain the fused point cloud data.

4. The method according to any one of claims 1 to 3, characterized in that, The step of performing feature recognition processing on the color image to obtain the target features corresponding to the region to be filled in the fused point cloud data includes: The fused point cloud data is subjected to voxel rasterization to obtain multiple voxel regions; In the plurality of voxel regions, determine the regions to be filled that meet the preset conditions; The target features are obtained by performing feature recognition processing on the color image at the location of the region to be filled. The preset conditions include at least one of the following: The number of point cloud data contained is less than the preset number of voxel regions; A voxel region whose distance from a reference region is greater than a preset distance threshold; the reference region is a region in the fused point cloud data whose number of point cloud data contained in the voxel region is greater than or equal to the preset number; The corresponding color image contains voxel regions whose texture contrast is less than the preset contrast or whose texture variance is higher than the preset texture variance.

5. The method according to claim 1, characterized in that, The target point cloud data includes point cloud data at each spatial location, confidence level at each spatial location, and target features at each spatial location. The method further includes: A confidence mask is generated based on the confidence level of the target point cloud data; Based on the confidence mask, the car wash equipment is controlled to clean the vehicle.

6. A vehicle cleaning control device, characterized in that, The device includes: The acquisition module acquires point cloud data, depth images, and color images of the vehicle's outer surface; The calculation module is used to determine the matching degree between each point cloud data and the depth data at the same position in the depth image based on the point cloud data and the depth image; The calculation module is also used to map the point cloud data to the camera coordinate system based on the camera intrinsic parameter matrix to obtain mapped point cloud data; The calculation module is also used to reverse map the depth data to the camera coordinate system to obtain coordinate data based on the depth image; The calculation module is further configured to calculate the depth difference and normal deviation between the mapped point cloud data and the coordinate data; the depth difference is the Euclidean space distance deviation between the mapped point cloud data and the coordinate data; the normal deviation is the angular deviation between the normal vector corresponding to the mapped point cloud data and the normal vector corresponding to the coordinate data. The calculation module is further configured to determine the matching degree based on the depth difference and the normal deviation; The fusion module is used to perform fusion processing on the point cloud data and the depth image according to the matching degree to obtain fused point cloud data; The extraction module is used to perform feature recognition processing on the color image to obtain the target features corresponding to the area to be filled in the fused point cloud data; The completion module is used to fill the area to be filled using a filling strategy corresponding to the target feature, so as to obtain the target point cloud data of the vehicle. The completion module is also used to fill the area to be filled by using a surface fitting method with image edge constraints if the target feature is a low-reflectivity surface feature, so as to obtain the target point cloud data. The completion module is further configured to, if the target feature is a feature of water mist interference and / or specular light interference, fill in the area to be filled based on at least one set of point cloud data obtained before the current time and the point cloud data corresponding to the area to be filled, to obtain the target point cloud data. The completion module is also used to fill the area to be filled by radial basis function interpolation if the target feature is a feature with small defects, so as to obtain the target point cloud data. The control module is used to control the car wash equipment to clean the vehicle based on the target point cloud data.

7. A vehicle cleaning control system, characterized in that, It includes control equipment and car wash equipment, wherein the control equipment is communicatively connected to the car wash equipment; The control device is used to send control commands to the car wash equipment; The car wash equipment or the control equipment is used to perform the method as described in any one of claims 1-5, determine the target point cloud data of the vehicle, and generate the control command based on the target point cloud data to control the cleaning of the vehicle.

8. A car wash device, characterized in that, It includes a body, at least one cleaning arm, a memory, and a processor; the body is connected to the cleaning arm. The cleaning arm is used to clean the surface of the vehicle; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-5.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-5.

10. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method described in any one of claims 1-5.

Citation Information

Patent Citations

  • Underwater target three-dimensional registration method based on cross-modal fusion

    CN117765048A

  • Sow body condition comprehensive scoring method based on data layer fusion

    CN119540991A

  • Multi-modal data three-dimensional scene reconstruction method and device

    CN120953484A