Cleaning task generation method and robot

Abstract

The embodiment of the application provides a cleaning task generation method and a robot, and relates to the technical field of vehicles. The method comprises the following steps: acquiring point cloud data of a vehicle interior and performing semantic segmentation on the point cloud data to obtain semantic labels, wherein the semantic labels represent sub-regions of vehicle components and materials of the sub-regions; converting the point cloud data into a semantic map according to the semantic labels, wherein the semantic map indicates geometric spatial information, semantic information and a functional state of the vehicle components; and generating a cleaning task for the vehicle interior according to the semantic map. The method can improve the cleaning efficiency of the vehicle interior.

Description

Technical Field

[0001] This application relates to the field of vehicle technology, and more particularly to a method for generating cleaning tasks and a robot. Background Technology

[0002] With the rapid development of fields such as intelligent vehicles, the demand for vehicle interior cleaning in complex and unstructured environments is increasing.

[0003] Many related technologies rely on preset cleaning paths or simple sensor feedback to generate cleaning tasks inside the vehicle, resulting in low cleaning efficiency inside the vehicle. Summary of the Invention

[0004] This application provides a cleaning task generation method and a robot to improve the cleaning efficiency of vehicle interiors.

[0005] In a first aspect, this application provides a cleaning task generation method, which includes: acquiring point cloud data inside a vehicle and performing semantic segmentation on the point cloud data to obtain semantic labels, wherein the semantic labels represent sub-regions and sub-region materials of vehicle components; converting the point cloud data into a semantic map based on the semantic labels, wherein the semantic map indicates the geometric spatial information, semantic information and functional status of vehicle components; and generating a cleaning task inside the vehicle based on the semantic map.

[0006] In one possible embodiment, before performing semantic segmentation on the point cloud data to obtain semantic labels, the method further includes: acquiring multispectral information inside the vehicle; performing semantic segmentation on the point cloud data to obtain semantic labels includes: performing feature fusion processing on the point cloud data and multispectral information to obtain fused features; and using a deep learning model to perform semantic segmentation on the fused features to obtain semantic labels.

[0007] In one possible embodiment, generating a cleaning task for the interior of a vehicle based on a semantic map includes: aggregating geometrically continuous sub-regions with the same vehicle components and materials into the same cleaning unit based on the semantic map to obtain multiple cleaning units; and generating a cleaning task sequence including multiple cleaning tasks based on the spatial dependencies between the multiple cleaning units, wherein the multiple cleaning tasks are used to clean the multiple cleaning units respectively.

[0008] In one possible embodiment, the cleaning task is based on cleaning parameters tied to the vehicle components and materials corresponding to the cleaning unit.

[0009] In one possible embodiment, after generating cleaning tasks inside the vehicle based on the semantic map, the method further includes: assigning a cleaning task sequence to each of the robot's multiple execution components based on a preset cleaning task allocation strategy, thereby obtaining multiple sub-sequences corresponding to each of the multiple execution components; generating a cost map for the first execution component based on the cost data of the vehicle components corresponding to each cleaning task in the first sub-sequence, wherein the first execution component is any one of the multiple execution components; and performing collision-free path planning based on the multiple cost maps corresponding to the multiple execution components to generate multiple cleaning paths corresponding to each of the multiple execution components.

[0010] In one possible embodiment, the method further includes: during the execution of the corresponding first cleaning path by the first execution component, performing a collision warning and updating the first cleaning path by means of a collision detection component installed on the first execution component.

[0011] In one possible embodiment, the method further includes: during the execution of a contact cleaning task in a first sub-sequence by the first execution component, after the first execution component moves to the part to be cleaned by executing a first cleaning path, the first execution component adapts to the shape of the part to be cleaned and performs a contact cleaning task along the normal direction of the part to be cleaned according to a constant pressure value set in the cleaning parameters of the contact cleaning task.

[0012] In one possible embodiment, the method further includes: detecting the cleaning effect of the first cleaning unit that has been cleaned and updating the cleaning status of the first cleaning unit in the semantic map using the cleaning effect of the first cleaning unit; if the cleaning status of the first cleaning unit indicates that the cleaning effect is not up to standard, generating a cleaning task for the first cleaning unit and inserting it into the cleaning task sequence.

[0013] In one possible embodiment, the method further includes: when the cleaning status of the first cleaning unit indicates that the cleaning effect meets the standard, recording the cleaning parameters, semantic map, vehicle type, and cleaning status corresponding to the first cleaning unit, and updating the knowledge base using the cleaning parameters, semantic map, vehicle type, and cleaning status corresponding to the first cleaning unit.

[0014] Secondly, this application provides a cleaning task generation device, which includes: a semantic segmentation module for acquiring point cloud data inside a vehicle and performing semantic segmentation on the point cloud data to obtain semantic tags, wherein the semantic tags represent sub-regions and sub-region materials of vehicle components; a conversion module for converting the point cloud data into a semantic map based on the semantic tags, wherein the semantic map indicates the geometric spatial information, semantic information and functional status of vehicle components; and a generation module for generating cleaning tasks inside the vehicle based on the semantic map.

[0015] Thirdly, this application provides a robot comprising: a memory, a processor, and an execution component; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, causing the processor to perform the method as described in any of the first aspects.

[0016] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any of the first aspects.

[0017] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the method of any one of the first aspects.

[0018] In this embodiment, point cloud data of the vehicle interior is acquired and semantically segmented to obtain semantic tags. These semantic tags characterize sub-regions and materials of vehicle components, enabling fine-grained semantic segmentation of the vehicle interior. The sub-regions and materials of the vehicle components correspond to the shape, function, and material differences between regions within the vehicle interior.

[0019] By converting point cloud data into semantic maps based on semantic labels, these semantic maps can be understood as digital twin models of the vehicle's interior, enabling robots to understand and query the geometric space, semantics, and functional status of vehicle components. Cleaning tasks generated from these semantic maps can support differentiated cleaning based on the geometric space, semantics, and functional status of vehicle components, thereby improving the efficiency of vehicle interior cleaning. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0021] Figure 1 This is a schematic diagram of the system architecture of the cleaning task generation method and robot according to an embodiment of this application;

[0022] Figure 2 This is a flowchart of a cleaning task generation method according to an embodiment of this application;

[0023] Figure 3 This is a schematic diagram of the structure of the cleaning task generation device according to an embodiment of this application.

[0024] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0025] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0026] Figure 1 This is a schematic diagram of the system architecture of the cleaning task generation method and robot according to an embodiment of this application.

[0027] like Figure 1 As shown, the system architecture includes robot 1 and vehicle 2.

[0028] For example, by triggering a cleaning button on robot 1, a person can trigger robot 1 to execute the cleaning task generation method of this application embodiment to generate a cleaning task for the interior of a vehicle. This cleaning task is performed by the execution components of robot 1 to clean the interior of the vehicle.

[0029] Apart from Figure 1 In addition to the system architecture shown, in another possible embodiment, the system architecture includes electronic devices, a robot, and a vehicle. The robot generates point cloud data of the vehicle's interior and sends this data to the electronic devices. The electronic devices execute the cleaning task generation method of this application embodiment to generate a cleaning task for the vehicle's interior. The cleaning task generated by the electronic devices is sent to the robot, and the cleaning task is performed by the robot's execution components to clean the vehicle's interior.

[0030] Electronic devices can be computers, servers, etc. Both electronic devices and robots include processors and memory; robots also include actuators.

[0031] In one possible embodiment, the memory of the electronic device or robot stores a knowledge base, which specifically stores cleaning parameters, semantic maps, vehicle type, cleaning status, cleaning task sequences, cleaning task allocation strategies, cleaning paths of various execution components, etc., to support the reuse of cleaning knowledge within the vehicle.

[0032] Optionally, robots and electronic devices may also include communication components. Taking a robot comprising a processor, memory, actuator, and communication components as an example, the processor, memory, actuator, and communication components are connected via a bus.

[0033] In specific implementation, code is stored in memory, and the processor executes the code stored in memory to perform the method of the embodiments of this application. The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules in the processor.

[0034] The memory may include high-speed RAM, and may also include non-volatile storage (NVM), such as at least one disk storage.

[0035] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0036] The actuator is, for example, a robotic arm or an end effector, such as an adaptive gripper.

[0037] For example, the robot also includes a sensor array, which includes sensors such as a structured light depth sensor or a binocular stereo vision camera for 3D scanning of the vehicle interior. The sensor array also includes robotic arm joint encoders for assisting localization, and may further include auxiliary localization markers within the vehicle.

[0038] For example, the sensor group may also include a torque sensor, a ranging sensor, a miniaturized near-infrared sensor for acquiring spectral data, or a spectral imaging sensor.

[0039] In related technology 1, two-dimensional / three-dimensional point cloud data of the vehicle interior is obtained through LiDAR or monocular camera, but it lacks the semantic recognition ability of the parts of the vehicle interior to be cleaned. It can only divide the cleaning area through geometric features and cannot distinguish key information such as material differences (such as leather and fabric) and functional attributes (such as openable storage compartments and fixed panels).

[0040] In related technology 2, a preset trajectory or gridded path coverage strategy is adopted, which ignores the spatial dependence of the parts to be cleaned inside the vehicle (such as the need to remove the seats before cleaning the floor) and process differences (such as the need to gently wipe the screen while the leather needs to be pressure-adapted), resulting in a disconnect between the cleaning action and the actual needs.

[0041] In related technology 3, the robot adopts a single-arm cleaning mode and avoids obstacles by means of joint limiting or artificial potential field method. However, it is difficult to cope with the needs of multiple cleaning tasks in the complex space inside the vehicle, and there are problems such as low operation efficiency and path conflict.

[0042] In addition, in related technologies, cleaning tools (such as vacuum brushes and wiping cloths) and process parameters (such as pressure and speed) are usually preset values ​​and cannot be dynamically adjusted according to the material of the parts to be cleaned inside the vehicle (such as fragile glass and soft fabrics), which can easily lead to poor cleaning results or damage to the interior of the vehicle.

[0043] Therefore, the relevant technologies have at least one of the following problems in generating cleaning tasks for the unstructured environment inside vehicles: the cleaning area is divided in a coarse manner, the cleaning path is not planned reasonably, the cleaning process is poorly adaptable, and the efficiency of multi-robotic arm collaboration is low, which cannot meet the needs of refined and intelligent cleaning.

[0044] Based on this, the cleaning task generation method and robot provided in this application perform semantic segmentation and construct semantic maps using point cloud data inside the vehicle, deeply integrating the cleaning areas inside the vehicle with spatial and semantic information. By combining dynamic cleaning task planning with multi-robotic arm collaborative control, it achieves efficient, accurate, and safe cleaning task generation in the complex and unstructured environment inside the vehicle, thereby enabling efficient, accurate, and safe cleaning operations inside the vehicle.

[0045] The embodiments of this application are applicable to complex, unstructured cleaning scenarios inside vehicles in the field of intelligent vehicles. For example, robots used to clean vehicle interiors need to complete cleaning tasks such as wiping leather seats, polishing the surface of the center console, and vacuuming door panel fabrics in a confined space.

[0046] Of course, the embodiments of this application can be extended to the field of service robots: household or commercial cleaning robots need to handle the cleaning needs of various materials such as furniture surfaces, appliance casings, and floor crevices. For example, in industrial automation scenarios: cleaning the interiors of precision equipment needs to adapt to areas with extremely high cleaning process requirements, such as electronic components and mechanical structures.

[0047] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0048] Figure 2 This is a flowchart illustrating a cleaning task generation method according to an embodiment of this application. This embodiment can be executed by a robot or an electronic device; the following description will use robot execution as an example.

[0049] like Figure 2 As shown, the cleaning task generation method of this application embodiment includes steps S110 to S130.

[0050] S110. Obtain point cloud data inside the vehicle and perform semantic segmentation on the point cloud data to obtain semantic labels.

[0051] Point cloud data inside a vehicle refers to the set of all visible and unavoidable three-dimensional coordinate points inside the vehicle, which is a three-dimensional digital representation of the vehicle's interior.

[0052] For example, the robot can use a structured light depth sensor or a binocular stereo vision camera mounted on its actuators to move along a preset scanning path, specifically scanning from multiple angles such as the driver's seat, passenger seat, and top view, to obtain local point cloud data from each angle. Next, the robot uses an iterative nearest-point algorithm or a feature-based matching algorithm, combined with the robotic arm joint position data determined by the robotic arm joint encoder after hand-eye calibration, to stitch the local point cloud data from each angle into complete, global point cloud data of the vehicle's interior in real time.

[0053] In one possible embodiment, the point cloud data inside the vehicle is also denoised. The denoising process includes applying voxel downsampling, statistical filtering to remove outliers, etc., to obtain clear, uniform, high-quality point cloud data.

[0054] For example, the robot can input point cloud data into a deep learning model for semantic segmentation to obtain semantic labels. The deep learning model could be one of PointNet++ (Deep Hierarchical Feature Learning on Point Sets in a Metric Space), KPConv (Kernel Point Convolution), or RandLA-Net (Random LAyer Network for Efficient Point Cloud Processing). The deep learning model is pre-trained using multiple training sets of vehicle interior point cloud data with ground truth semantic labels.

[0055] PointNet++ is a deep learning model for processing large-scale point cloud data. KPConv is a deep learning model based on kernel-based convolutions that can be used to process point cloud data. RandLA-Net is a lightweight point cloud data processing model.

[0056] Semantic tags represent sub-regions and materials of vehicle components. Vehicle components include: seats, center console, floor, doors, etc. Examples of specific semantic tags are given below. For example, semantic tags include at least one of the following: main leather surface of the seat, fabric on the side of the seat, seat gaps, center console screen, piano black panel, door armrest, door storage compartment, floor mat, gear shift lever, and steering wheel.

[0057] S120. Convert point cloud data into a semantic map based on semantic labels.

[0058] Semantic maps indicate the geometric spatial information, semantic information, and functional status of vehicle components. A semantic map can be understood as a dynamic, robot-understandable, and queryable digital twin model of the vehicle's interior.

[0059] Geometric spatial information includes the 3D bounding box information of vehicle components, point cloud clusters, and surface normal vector fields. Semantic information indicates vehicle components, sub-regions, and sub-region materials. Functional status indicates the current state of vehicle components, such as whether the cup holder is open or closed, or whether the screen is upright or folded up.

[0060] For example, the robot generates an instance identifier for each vehicle component based on semantic tags. Each vehicle component's instance identifier is associated with its 3D bounding box information, point cloud clusters, surface normal vector field, sub-regions, sub-region materials, and functional status. This results in a structured semantic map.

[0061] S130. Generate a cleaning task for the vehicle interior based on the semantic map.

[0062] In this embodiment, point cloud data of the vehicle interior is acquired and semantically segmented to obtain semantic tags. These semantic tags characterize sub-regions and materials of vehicle components, enabling fine-grained semantic segmentation of the vehicle interior. The sub-regions and materials of the vehicle components correspond to the shape, function, and material differences between regions within the vehicle interior.

[0063] By converting point cloud data into semantic maps based on semantic labels, these semantic maps can be understood as digital twin models of the vehicle's interior, enabling robots to understand and query the geometric space, semantics, and functional status of vehicle components. Cleaning tasks generated from these semantic maps can support differentiated cleaning based on the geometric space, semantics, and functional status of vehicle components, thereby improving the efficiency of vehicle interior cleaning.

[0064] Unlike the previous embodiment, where the robot inputs point cloud data into a deep learning model for semantic segmentation to obtain semantic labels, in another embodiment of this application, the robot can combine point cloud data and multispectral information for semantic segmentation.

[0065] Specifically, in one possible embodiment, before performing semantic segmentation on the point cloud data in step S120 to obtain semantic labels, the cleaning task generation method further includes step S140.

[0066] S140, Acquire multispectral information inside the vehicle.

[0067] For example, the robot sends infrared light of multiple wavelengths to the interior of the vehicle through a miniaturized near-infrared sensor or a spectral imaging sensor. The intensity of the reflected light energy after the infrared light shines on the interior surface of the vehicle is the multispectral information of the interior of the vehicle.

[0068] Step S110 involves semantic segmentation of the point cloud data to obtain semantic labels, including steps S111 to S112.

[0069] S111. Perform feature fusion processing on point cloud data and multispectral information to obtain fused features.

[0070] For example, when the dimensions of point cloud data and multispectral information are consistent, the point cloud data and multispectral information can be weighted or stitched together to obtain fused features.

[0071] S112. Using a deep learning model, perform semantic segmentation on the fused features to obtain semantic labels.

[0072] For example, semantic labels can be obtained by fusing features into a deep learning model.

[0073] In this embodiment, by acquiring multispectral information from inside the vehicle, the multispectral information can reflect the characteristics of different materials inside the vehicle, because different materials inside the vehicle have different absorption and reflection properties for infrared light. That is, in this embodiment, the multispectral information from inside the vehicle can distinguish different materials inside the vehicle, facilitating subsequent combination with point cloud data to help achieve more refined semantic segmentation of the vehicle's interior. For example, different regions of the same vehicle component may have different materials, or different vehicle components inside the vehicle may have different materials.

[0074] By fusing point cloud data and multispectral information, and combining the three-dimensional spatial and material information of the vehicle's interior, the resulting fused features exhibit superior semantic segmentation representation. Furthermore, by utilizing a deep learning model to perform refined semantic segmentation on the fused features, the resulting semantic labels are more granular and accurate.

[0075] In one possible embodiment, step S130, which generates a cleaning task for the vehicle interior based on a semantic map, includes steps S131 to S132.

[0076] S131. Based on the semantic map, sub-regions with continuous geometric space and identical vehicle parts and materials are aggregated into the same cleaning unit to obtain multiple cleaning units.

[0077] For example, by using the sub-regions and materials of vehicle components indicated by semantic tags, and the geometric spatial information of vehicle components indicated by semantic maps, sub-regions that are geometrically continuous and contain the same vehicle components and materials can be identified and aggregated into the same cleaning unit. Similarly, multiple cleaning units can be obtained.

[0078] S132. Based on the spatial dependency between multiple cleaning units, generate a cleaning task sequence that includes multiple cleaning tasks, each of which is used to clean multiple cleaning units.

[0079] Spatial dependencies are determined by the physical spatial relationships between multiple cleaning units. For example, the spatial dependency between a floor mat cleaning unit and a seat cleaning unit is that the floor mat is located under the seat, which means that the cleaning task for the floor mat must be performed after the cleaning task for the seat. Alternatively, the cleaning task for the floor mat must be performed after the seat has been moved.

[0080] For example, the robot can combine the spatial dependencies between multiple cleaning units to generate a cleaning task sequence based on a preset cleaning strategy. For instance, the cleaning strategy might indicate a top-down, inside-out cleaning sequence, performing dry cleaning first, followed by wet cleaning. Dry cleaning includes vacuuming, dry wiping, and sweeping, while wet cleaning includes wet wiping and scrubbing.

[0081] For example, in vacuuming mode, the cleaning task sequence is as follows: The headliner cleaning task, dashboard cleaning task, seat surface cleaning task, door panel cleaning task, and floor cleaning task are executed sequentially. In fine cleaning mode, the cleaning task sequence is as follows: screen cleaning task, leather area cleaning task of seats, fabric area cleaning task of seats, and storage compartment cleaning task are executed sequentially.

[0082] Different vehicle components have different functional structures, and multiple sub-regions of the same vehicle component may have different materials or significantly different shapes. These factors can lead to problems such as complex vehicle interior cleaning, poor cleaning results, and easy damage to the vehicle. In this embodiment, by using a semantic map, geometrically continuous sub-regions of the same vehicle component and material are aggregated into the same cleaning unit to obtain multiple cleaning units. This allows fine-grained sub-regions to be aggregated based on their material, shape, function, and other cleaning-related characteristics, ensuring that the same cleaning unit has no abrupt changes in shape, the same function, and the same material. The absence of abrupt changes in shape ensures cleaning effectiveness, while the same function and material facilitate the use of the same cleaning process and reduce damage to the vehicle during cleaning.

[0083] By generating a cleaning task sequence comprising multiple cleaning tasks based on the spatial dependencies between multiple cleaning units, cleaning tasks can be divided at the cleaning unit level. Furthermore, the physical relationships between multiple cleaning units are combined to generate a cleaning task sequence that does not interfere with each other in physical space, ensuring cleaning effectiveness and efficiency.

[0084] In one possible embodiment, the cleaning task is based on cleaning parameters tied to the vehicle components and materials corresponding to the cleaning unit.

[0085] In this embodiment, one cleaning unit corresponds to one cleaning task, and each cleaning task can be bound to cleaning parameters, which are used specifically to clean the corresponding cleaning unit. This improves the fine-grained management of cleaning parameters, thereby enhancing the compatibility between cleaning parameters and cleaning units and improving cleaning safety, thus avoiding damage to the vehicle interior during cleaning.

[0086] For example, the cleaning parameters for the piano lacquer panel cleaning unit include: cleaning tool: microfiber cloth, cleaning pressure: 0.5N, cleaning trajectory: unidirectional straight line, and cleaning action: wiping. As another example, the cleaning parameters for the door panel storage compartment cleaning unit include: cleaning tool: small curved vacuum brush and long, thin brush, cleaning trajectory: rotation, and cleaning action: vacuuming followed by brushing.

[0087] Through the above embodiments, a cleaning task for the vehicle interior has been generated. The robot can then perform the actual cleaning task. Before performing the cleaning task, the robot first plans the cleaning path.

[0088] Specifically, in one possible embodiment, after generating the cleaning task inside the vehicle based on the semantic map in step S130, the cleaning task generation method further includes steps S140 to S160.

[0089] S140. Based on a preset cleaning task allocation strategy, a cleaning task sequence is allocated to each of the robot's multiple execution components, resulting in multiple sub-sequences corresponding to each of the multiple execution components.

[0090] In this embodiment, the robot includes multiple execution components. The cleaning task sequence can be divided into multiple sub-sequences executed by each of the multiple execution components to improve cleaning efficiency. The following explanation will use a dual-arm robotic arm as an example of the robot's multiple execution components.

[0091] For example, the cleaning task allocation strategy indicates at least one of the following: workspace balance, minimum number of cleaning tool changes, and shortest total cleaning time. Workspace balance can be understood as multiple actuators each being responsible for cleaning a portion of the space, or multiple actuators performing cleaning actions alternately and sequentially. For example, in a dual-arm system, the left arm cleans the left side of the vehicle's interior, while the right arm cleans the right side. Another example is that in a dual-arm system, one arm performs a vacuuming action, while the other arm prepares a wiping tool and performs a wiping action after the vacuuming action is completed.

[0092] S150. For the first subsequence corresponding to the first execution component, generate a cost map of the first execution component based on the cost data of the vehicle components corresponding to each cleaning task in the first subsequence. The first execution component is any one of multiple execution components.

[0093] Taking the execution of a first sub-sequence by a first execution unit as an example, the first sub-sequence can be understood as the portion of the cleaning task sequence assigned to the first execution unit. The first sub-sequence indicates at least one cleaning task for cleaning at least one cleaning unit. The cost data of each cleaning unit constitutes a hierarchical cost map of the first execution unit executing the first sub-sequence. Since cleaning units are formed by aggregating geometrically continuous sub-regions of the same vehicle parts and materials, the cleaning characteristics of the same cleaning unit are consistent. Therefore, in this embodiment, cost data is divided at the granularity of cleaning units to form a hierarchical cost map.

[0094] Cost data can be understood as the degree of risk that the first actuator poses to the cleaning unit during cleaning. The cost data for any vehicle component is determined based on the component's preset geometric obstacles, material strength, and other factors.

[0095] For example, the cost data for the seat surface cleaning unit indicates a low level of risk, meaning the first actuator can clean the seat surface directly. However, the cost data for the center console screen cleaning unit indicates a higher level of risk, requiring the first actuator to avoid or lightly touch the center console screen for cleaning.

[0096] S160. Collision-free path planning is performed based on multiple cost maps corresponding to multiple execution components to generate multiple clean paths corresponding to each execution component.

[0097] For example, robots can use sampling algorithms such as Rapidly-exploring Random Trees (RRT) or Batch Informed Trees (BIT) for collision-free path planning.

[0098] In this embodiment, a cleaning task sequence is assigned to each of the robot's multiple execution components based on a preset cleaning task allocation strategy, resulting in multiple sub-sequences corresponding to each execution component. This allows the multiple execution components to collaboratively execute the cleaning task sequence, improving cleaning efficiency. By generating a cost map for the first execution component based on the cost data of the vehicle components corresponding to each cleaning task in the first sub-sequence, each execution component's cost map provides cost data for each cleaning unit, indicating the risk level of the execution component performing cleaning on that unit. The cleaning path obtained by collision-free path planning based on the multiple cost maps corresponding to multiple execution components is safe and efficient.

[0099] For example, the robot can also perform smoothing and path optimization on the multiple cleaning paths corresponding to the multiple execution components obtained through steps S150 to S160 to ensure that the movement of each execution component is coherent and efficient, and meets mechanical constraints.

[0100] In one possible embodiment, the cleaning task generation method further includes: during the execution of the corresponding first cleaning path by the first execution component, performing collision warning and updating the first cleaning path by using a collision detection component installed on the first execution component.

[0101] For example, the collision avoidance detection component includes one of a vision sensor, a torque sensor, and a ranging sensor. Taking the ranging sensor as an example, if the ranging sensor detects that the distance between the first actuator and the obstacle is lower than a preset threshold, the robot can issue a collision warning. This triggers the replanning of the first cleaning path, and the original first cleaning path is updated using the replanned first cleaning path.

[0102] In this embodiment of the application, during the execution of the corresponding first cleaning path by the first execution component, a collision warning and update of the first cleaning path are performed by a collision detection component installed on the first execution component. This enables real-time obstacle avoidance during the execution of the first cleaning path by the first execution component, ensuring the safety of the first execution component in executing the first cleaning path and adapting to the dynamic in-vehicle environment.

[0103] In one possible embodiment, the robot can also detect the paths of multiple execution components in real time, and ensure that the multiple execution components do not interfere with or collide with each other through artificial potential field method or preset priority strategy.

[0104] In one possible embodiment, the cleaning task generation method further includes: during the execution of a contact cleaning task in a first sub-sequence by the first execution component, after the first execution component moves to the part to be cleaned by executing a first cleaning path, the first execution component adapts to the shape of the part to be cleaned and performs the contact cleaning task along the normal direction of the part to be cleaned according to the constant pressure value set in the cleaning parameters of the contact cleaning task.

[0105] For example, the robot can detect the shape of the part to be cleaned in real time by using sensors such as a vision camera installed on the first execution component, and perform a contact cleaning task along the normal direction of the part to be cleaned according to the constant pressure value set in the cleaning parameters of the contact cleaning task.

[0106] The embodiments of this application can adapt to the shape of the part to be cleaned through the above operations, and perform flexible and constant force cleaning. This not only adapts to the edge and shape changes of the part to be cleaned, ensuring the cleaning effect, but also prevents damage to the vehicle due to uneven cleaning pressure.

[0107] In one possible embodiment, the cleaning task generation method further includes: detecting the cleaning effect of a first cleaning unit that has been cleaned and updating the cleaning status of the first cleaning unit in the semantic map using the cleaning effect of the first cleaning unit; and generating a cleaning task for the first cleaning unit and inserting it into the cleaning task sequence if the cleaning status of the first cleaning unit indicates that the cleaning effect is not up to standard.

[0108] For example, taking the cleaning of a first cleaning unit by a first execution component as an example, in the above embodiment, the first execution component uses the cleaning parameters bound to the first cleaning unit and executes a first cleaning path to complete the cleaning of the first cleaning unit. Next, the robot can acquire a cleaning image of the completed first cleaning unit using a camera mounted on the first execution component and input the cleaning image into a deep learning model to determine the cleaning effect of the completed first cleaning unit.

[0109] In this embodiment, by detecting the cleaning effect of the completed first cleaning unit and updating its cleaning status in the semantic map using that effect, the cleaning process of the first cleaning unit can be monitored and feedback can be provided. Specifically, this feedback is fed back to the semantic map, enabling it to indicate not only the geometric spatial information, semantic information, and functional status of vehicle components, but also the cleaning status of each cleaning unit. This further allows the semantic map to guide the cleaning progress of any single cleaning unit. If the cleaning status of the first cleaning unit indicates that the cleaning effect is not up to standard, generating a cleaning task for that unit and inserting it into the cleaning task sequence can overcome the limitations of an open-ended approach to cleaning the vehicle interior, ensuring the cleaning effect of each cleaning unit.

[0110] It should be noted that there is a limit to the number of times each cleaning unit can be cleaned. When the cleaning number of any cleaning unit reaches the limit, the robot can send a manual intervention request to request manual cleaning of the cleaning unit that has not met the cleaning standard.

[0111] In one possible embodiment, the cleaning task generation method further includes: when the cleaning status of the first cleaning unit indicates that the cleaning effect meets the standard, recording the cleaning parameters, semantic map, vehicle type, and cleaning status corresponding to the first cleaning unit, and updating the knowledge base using the cleaning parameters, semantic map, vehicle type, and cleaning status corresponding to the first cleaning unit.

[0112] If the cleaning status of the first cleaning unit indicates that the cleaning effect has met the standards, it means that the cleaning of the first cleaning unit was effective. In this embodiment, by recording the cleaning parameters, semantic map, vehicle type, and cleaning status corresponding to the first cleaning unit, and updating the knowledge base using these parameters, the relevant knowledge of the effective cleaning can be reused, improving the reliability of the robot cleaning the interior of the vehicle. Furthermore, the updated knowledge base allows for iterative updates of the relevant knowledge of effective cleaning, supporting the robot's continuous learning capabilities.

[0113] For example, after completing the overall cleaning of the vehicle interior, the robot can also record multiple corresponding cleaning tasks and cleaning task sequences, cleaning task allocation strategies, and cleaning paths of each execution component, and update the knowledge base using these multiple cleaning tasks and cleaning task sequences, cleaning task allocation strategies, and cleaning paths of each execution component.

[0114] In summary, this application's embodiments construct a 3D semantic map of the vehicle's interior through multi-view point cloud fusion and deep learning semantic segmentation, combining the geometric spatial information of the cleaning area with various attributes such as material and function. Based on this, sub-regions of vehicle components are aggregated and dependency analyzed using semantic tags to generate an optimized cleaning task sequence and dynamically bind cleaning parameters. For the dual-arm robot, a collision-free and efficient cleaning path is generated for each execution component based on obstacle information in the semantic map and the cleaning task allocation strategy. During cleaning, a collision warning is provided in real time through a collision avoidance detection component, triggering cleaning path replanning. Furthermore, a closed-loop feedback mechanism and a knowledge base optimization mechanism are used to continuously update cleaning-related knowledge, ultimately achieving efficient and precise cleaning operations within the complex space of the vehicle's interior.

[0115] Figure 3 This is a schematic diagram of the cleaning task generation device according to an embodiment of this application. Figure 3 As shown, the cleaning task generation device provided in this application embodiment includes: a semantic segmentation module 210, a conversion module 220, and a generation module 230.

[0116] The semantic segmentation module 210 is used to acquire point cloud data inside the vehicle and perform semantic segmentation on the point cloud data to obtain semantic labels. The semantic labels represent the sub-regions and materials of the vehicle components.

[0117] The conversion module 220 is used to convert point cloud data into a semantic map based on semantic labels. The semantic map indicates the geometric spatial information, semantic information, and functional status of vehicle components.

[0118] The generation module 230 is used to generate cleaning tasks for the interior of the vehicle based on the semantic map.

[0119] In one possible embodiment, the device further includes: an acquisition module for acquiring multispectral information inside the vehicle; a semantic segmentation module including a fusion submodule for performing feature fusion processing on point cloud data and multispectral information to obtain fused features; and a segmentation submodule for using a deep learning model to perform semantic segmentation on the fused features to obtain semantic labels.

[0120] In one possible embodiment, the generation module includes: an aggregation submodule, configured to aggregate sub-regions that are geometrically continuous and have the same vehicle parts and materials into the same cleaning unit based on a semantic map, so as to obtain multiple cleaning units; and a generation submodule, configured to generate a cleaning task sequence including multiple cleaning tasks based on the spatial dependencies between the multiple cleaning units, wherein the multiple cleaning tasks are used to clean the multiple cleaning units respectively.

[0121] In one possible embodiment, the cleaning task is based on cleaning parameters tied to the vehicle components and materials corresponding to the cleaning unit.

[0122] In one possible embodiment, the device further includes: an allocation module, configured to allocate a cleaning task sequence to each of the robot's multiple execution components based on a preset cleaning task allocation strategy, thereby obtaining multiple sub-sequences corresponding to each of the multiple execution components; a cost map generation module, configured to generate a cost map for a first execution component based on the cost data of vehicle components corresponding to each cleaning task in the first sub-sequence corresponding to the first execution component, wherein the first execution component is any one of the multiple execution components; and a path planning module, configured to perform collision-free path planning based on the multiple cost maps corresponding to the multiple execution components, thereby generating multiple cleaning paths corresponding to each of the multiple execution components.

[0123] In one possible embodiment, the device further includes: a warning module, used to provide collision warning and update the first cleaning path by means of a collision detection component installed on the first execution component during the execution of the corresponding first cleaning path by the first execution component.

[0124] In one possible embodiment, the device further includes: an execution module, configured to, during the execution of a contact cleaning task in a first sub-sequence by the first execution component, after the first execution component moves to the part to be cleaned by executing a first cleaning path, the first execution component adapts to the shape of the part to be cleaned and performs a contact cleaning task along the normal direction of the part to be cleaned according to a constant pressure value set in the cleaning parameters of the contact cleaning task.

[0125] In one possible embodiment, the device further includes: an update module, configured to detect the cleaning effect of the first cleaning unit that has been cleaned and update the cleaning status of the first cleaning unit in the semantic map using the cleaning effect of the first cleaning unit; and an insertion module, configured to generate a cleaning task for the first cleaning unit and insert it into the cleaning task sequence when the cleaning status of the first cleaning unit indicates that the cleaning effect is not up to standard.

[0126] In one possible embodiment, the device further includes a recording module, configured to record the cleaning parameters, semantic map, vehicle type, and cleaning status corresponding to the first cleaning unit when the cleaning status of the first cleaning unit indicates that the cleaning effect meets the standard, and update the knowledge base using the cleaning parameters, semantic map, vehicle type, and cleaning status corresponding to the first cleaning unit.

[0127] This application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the methods described in the above-described method embodiments.

[0128] The aforementioned computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0129] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0130] This application provides a computer program product, including a computer program that, when executed by a processor, implements the methods provided in any of the embodiments described above.

[0131] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0132] It should be further noted that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0133] It should be understood that the above-described device embodiments are merely illustrative, and the device of this application can also be implemented in other ways. For example, the division of units / modules in the above embodiments is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units, modules, or components may be combined, or integrated into another system, or some features may be ignored or not executed.

[0134] Furthermore, unless otherwise specified, the functional units / modules in the various embodiments of this application can be integrated into one unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated together. The integrated units / modules described above can be implemented in hardware or as software program modules.

[0135] When integrated units / modules are implemented in hardware, the hardware can be digital circuits, analog circuits, etc. The physical implementation of the hardware structure includes, but is not limited to, transistors, memristors, etc. Unless otherwise specified, the processor can be any suitable hardware processor, such as a CPU, GPU, FPGA, DSP, and ASIC, etc. Unless otherwise specified, the storage unit can be any suitable magnetic or magneto-optical storage medium, such as Resistive Random Access Memory (RRAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Enhanced Dynamic Random Access Memory (EDRAM), High-Bandwidth Memory (HBM), Hybrid Memory Cube (HMC), etc.

[0136] If the integrated unit / module is implemented as a software program module and sold or used as an independent financial product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software financial product. This computer software financial product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0137] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0138] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0139] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method of generating a cleaning task, characterized by, The method includes: Obtain point cloud data of the vehicle interior and perform semantic segmentation on the point cloud data to obtain semantic labels. The semantic labels represent the sub-regions and sub-region materials of vehicle components. The point cloud data is converted into a semantic map based on the semantic tags, and the semantic map indicates the geometric spatial information, semantic information and functional status of the vehicle components; Based on the semantic map, a cleaning task for the vehicle interior is generated.

2. The method according to claim 1, characterized in that, Before performing semantic segmentation on the point cloud data to obtain semantic labels, the method further includes: acquiring multispectral information inside the vehicle; The semantic segmentation of the point cloud data to obtain semantic labels includes: The point cloud data and the multispectral information are subjected to feature fusion processing to obtain fused features; Using a deep learning model, the fused features are semantically segmented to obtain the semantic labels.

3. The method according to claim 1, characterized in that, The step of generating a cleaning task for the vehicle interior based on the semantic map includes: Based on the semantic map, sub-regions with continuous geometric space and identical vehicle parts and materials are aggregated into the same cleaning unit to obtain multiple cleaning units; Based on the spatial dependencies between the multiple cleaning units, a cleaning task sequence is generated, which includes multiple cleaning tasks, each of which is used to clean the multiple cleaning units.

4. The method according to claim 3, characterized in that, The cleaning task is based on cleaning parameters bound to the vehicle components and materials corresponding to the cleaning unit.

5. The method according to claim 4, characterized in that, After generating the vehicle interior cleaning task based on the semantic map, the method further includes: Based on a preset cleaning task allocation strategy, the cleaning task sequence is allocated to each of the robot's multiple execution components, resulting in multiple sub-sequences corresponding to each of the multiple execution components; For the first subsequence corresponding to the first execution component, a cost map of the first execution component is generated based on the cost data of the vehicle components corresponding to each cleaning task in the first subsequence. The first execution component is any one of the plurality of execution components. Collision-free path planning is performed based on multiple cost maps corresponding to the multiple execution components to generate multiple clean paths corresponding to each of the multiple execution components.

6. The method according to claim 5, characterized in that, The method further includes: During the execution of the corresponding first cleaning path by the first execution component, a collision warning is issued and the first cleaning path is updated by a collision detection component installed on the first execution component.

7. The method according to claim 6, characterized in that, The method further includes: During the execution of the contact cleaning task in the first sub-sequence by the first execution unit, after the first execution unit moves to the part to be cleaned by executing the first cleaning path, the first execution unit adapts to the shape of the part to be cleaned and executes the contact cleaning task along the normal of the part to be cleaned according to the constant pressure value set in the cleaning parameters of the contact cleaning task.

8. The method according to claim 3, characterized in that, The method further includes: The cleaning effect of the first cleaning unit that has been cleaned is detected and the cleaning status of the first cleaning unit in the semantic map is updated using the cleaning effect of the first cleaning unit. If the cleaning status of the first cleaning unit indicates that the cleaning effect is not up to standard, a cleaning task for the first cleaning unit is generated and inserted into the cleaning task sequence.

9. The method according to claim 8, characterized in that, The method further includes: When the cleaning status of the first cleaning unit indicates that the cleaning effect meets the standard, the cleaning parameters, semantic map, vehicle type, and cleaning status corresponding to the first cleaning unit are recorded, and the knowledge base is updated using the cleaning parameters, semantic map, vehicle type, and cleaning status corresponding to the first cleaning unit.

10. A robot, characterized in that, include: Memory, processor, and execution unit; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-9.

Images