Control method and device and robot

Abstract

The invention provides a control method and device and a robot. The method comprises the steps that a corresponding operation script is determined according to vehicle type identification information of a target vehicle, and the operation script is used for controlling a vehicle door handle to execute vehicle door operation; sending first control information to an executive component of the robot through a first state machine based on the operation script, wherein the first control information is used for controlling the executive component of the robot to sequentially execute a plurality of operation elements corresponding to the operation script; and in the process that the robot executes the operation primitive, the sensor data of the robot and the preset condition of the operation primitive are compared, and the first state machine is driven to jump according to the comparison result. The method can adapt to structural differences of different vehicle types and reduce the failure rate of vehicle door operation.

Description

Technical Field

[0001] This application relates to the field of vehicle technology, and more particularly to a control method, device and robot. Background Technology

[0002] In the fields of automobile manufacturing, maintenance, and intelligent cleaning, robots are required to frequently handle door operation tasks for different car models.

[0003] In related technologies, robots generally rely on preset fixed operating procedures or general algorithms, which cannot flexibly cope with the structural differences between multiple car models, resulting in a high failure rate of door operation and low efficiency in door operation tasks. Summary of the Invention

[0004] This application provides a control method, device, and robot that can adapt to the structural differences of different vehicle models and reduce the failure rate of door operation.

[0005] In a first aspect, this application provides a control method, comprising: determining a corresponding operation script based on the vehicle model identification information of the target vehicle, the operation script being used to control the vehicle door handle to perform door operation; sending first control information to the execution component of a robot through a first state machine based on the operation script, the first control information being used to control the execution component of the robot to sequentially execute multiple operation primitives corresponding to the operation script; during the execution of the operation primitives by the robot, comparing the sensor data of the robot with the preset conditions of the operation primitives and driving the first state machine to jump according to the comparison result.

[0006] In one possible embodiment, during the robot's execution of an operation primitive, comparing the robot's sensor data with the preset conditions of the operation primitive and driving the first state machine to jump based on the comparison result includes: collecting sensor data when the robot executes the operation primitive; comparing the sensor data with the preset success conditions and / or preset failure conditions of the operation primitive to obtain a comparison result; if the comparison result meets the preset success conditions, driving the first state machine to jump to the next operation primitive; if the comparison result meets the preset failure conditions, driving the first state machine to jump to the exception handling state.

[0007] In one possible embodiment, after driving the first state machine to jump to the exception handling state, the method further includes: in the exception handling state, sending second control information to the robot's execution component according to the exception handling strategy corresponding to the preset failure condition, the second control information being used to control the robot's execution component to perform an operation so that the operation primitive that meets the preset failure condition is restored to the preset state.

[0008] In one possible embodiment, the method further includes: recording the execution parameters and operation results of the operation primitives during the execution of the operation primitives by the robot's execution components; and adjusting and updating preset success conditions and / or preset failure conditions based on the execution parameters and operation results through a machine learning model.

[0009] In one possible embodiment, after the door operation is a door opening operation and the last operation primitive in the operation script is successfully executed, the method further includes: determining the corresponding cleaning strategy based on the vehicle model identification information; and sending third control information to the robot's execution component through a second state machine based on the cleaning strategy, wherein the third control information is used to control the robot's execution component to sequentially execute multiple operation primitives corresponding to the cleaning strategy.

[0010] In one possible embodiment, after the door operation is a door opening operation and the last operation primitive of the operation script is successfully executed, the method further includes: acquiring vehicle features corresponding to the door opening operation, the vehicle features including at least one of the following: maximum door opening angle, interior exposed area, and door movement trajectory. Sending third control information to the robot's execution component via a second state machine based on the cleaning strategy includes: determining at least one cleaning sub-task and a cleaning trajectory corresponding to each cleaning sub-task based on the cleaning strategy and vehicle features; generating third control information based on the cleaning sub-tasks and their corresponding cleaning trajectories and sending the third control information to the robot's execution component.

[0011] In one possible embodiment, at least one of the operation script, cleaning strategy, exception handling strategy, and vehicle features corresponding to the door opening operation is stored in a knowledge base and can be invoked.

[0012] In one possible embodiment, the cleaning strategy specifies a cleaning sub-task for each exposed interior area, as well as the cleaning tools, cleaning parameters, and cleaning trajectory for each cleaning sub-task.

[0013] Secondly, this application provides a control device, comprising: an operation script determination module, configured to determine a corresponding operation script based on the vehicle model identification information of the target vehicle, the operation script being used to control the vehicle door handle to perform door operation; a sending module, configured to send first control information to the execution component of a robot through a first state machine based on the operation script, the first control information being used to control the execution component of the robot to sequentially execute multiple operation primitives corresponding to the operation script; and a comparison module, further configured to compare the robot's sensor data with the preset conditions of the operation primitives during the robot's execution of the operation primitives and drive the first state machine to jump based on the comparison result.

[0014] Thirdly, this application provides an electronic device, including: a memory, a processor, and an execution unit; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, causing the processor to perform the method as described in any of the first aspects.

[0015] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any of the first aspects.

[0016] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the method of any one of the first aspects.

[0017] In this embodiment, expert experience can be transformed into an executable operation script. By determining the corresponding operation script based on the target vehicle's model identification information, it can be ensured that the process of controlling the vehicle door handle to perform door operations is compatible with the structural characteristics of the target vehicle's door handle. Furthermore, each operation primitive in each operation script can be reused, thereby avoiding the lengthy development cycle of requiring separate programming for each vehicle model to perform door operations for fixed process control.

[0018] According to the operation script, the first control information is sent to the robot's execution components via the first state machine, which drives the robot's execution components to sequentially execute multiple operation primitives corresponding to the operation script. Simultaneously, by comparing the robot's sensor data with the preset conditions of the operation primitives and driving the first state machine to jump based on the comparison result, the execution state of each operation primitive can be detected in real time, and the first state machine can be accurately driven to jump based on the execution state. This process enables closed-loop control from vehicle model recognition to the completion of the door operation task, improving the robot's adaptability to differences in vehicle model structures and reducing the door operation failure rate.

[0019] In addition, the first state machine can detect and manage the process of the robot executing the operation script, enabling continuous operation of door operation tasks in a flexible and automated manner. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0021] Figure 1 This is a schematic diagram illustrating an application scenario of the control method according to an embodiment of this application;

[0022] Figure 2 This is a flowchart of the control method according to an embodiment of this application;

[0023] Figure 3 This is a flowchart of a control method according to another embodiment of this application;

[0024] Figure 4 This is a schematic diagram of the control device according to an embodiment of this application.

[0025] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0026] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0027] Figure 1 This is a schematic diagram illustrating the application scenarios of the control method, apparatus, and robot according to embodiments of this application.

[0028] like Figure 1 As shown, maintenance personnel maintain knowledge related to vehicle door handle operation in the knowledge base through electronic devices. The robot can invoke knowledge from the knowledge base and execute the control method of this embodiment to perform door operations on the vehicle.

[0029] exist Figure 1 In the examples, the electronic device can be a computer, server, etc. In the embodiments of this application, the robot includes a processor, a memory, and an execution component.

[0030] For example, the knowledge base may be stored in the memory of an electronic device or a robot. Figure 1 This is an example of a knowledge base stored on an electronic device.

[0031] Optionally, the robot also includes a communication component. The processor, memory, execution component, and communication component are connected via a bus.

[0032] In specific implementation, code is stored in memory, and the processor executes the code stored in memory to perform the method of the embodiments of this application. The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules in the processor.

[0033] The memory may include high-speed RAM, and may also include non-volatile storage (NVM), such as at least one disk storage.

[0034] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0035] The actuator is used to perform specific door operations. The actuator may be, for example, a robotic arm or an end effector, such as an adaptive gripper.

[0036] For example, the robot also includes a sensor array, which includes at least one of a global / local vision camera, an end effector six-dimensional force / torque sensor, a joint encoder, a gripper opening / closing sensor, and an acoustic sensor.

[0037] In related technology 1, operating steps are preset for each car model through manual programming to drive the robot to perform door operations. However, a separate program needs to be developed for each car model, resulting in high development costs and difficulty in scaling to new car models.

[0038] In related technology 2, rule-based decision logic is used to drive the robot to perform door operations, but it lacks modeling of vehicle-specific operation knowledge and cannot handle complex scenarios.

[0039] The control method, device, and robot provided in this application store vehicle-specific operation scripts in a knowledge base and adjust execution strategies based on sensor data to achieve closed-loop control of the entire process from vehicle model recognition to completion of the door operation task. Specifically, expert experience can be transformed into programmable operation scripts and preset conditions for successful or failed execution. A state machine monitors the execution status in real time, and an exception handling strategy enables self-recovery, thereby solving the problems of poor operational adaptability, insufficient robustness, and low collaborative efficiency of door operation tasks in multi-vehicle scenarios.

[0040] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0041] Figure 2 This is a flowchart illustrating a control method according to an embodiment of this application. The control method described in this application can be applied to robots.

[0042] like Figure 2 As shown, the control method of this application embodiment includes steps S110 to S130.

[0043] S110. Determine the corresponding operation script based on the vehicle model identification information of the target vehicle.

[0044] The target vehicle can be understood as any vehicle whose door operation is currently to be performed.

[0045] Vehicle identification information refers to information used to identify the model of a target vehicle. Vehicle identification information can be determined through at least one of the following: vehicle outline, vehicle logo, Vehicle Identification Number (VIN), or vehicle model code entered by the user.

[0046] For example, images of the target vehicle are acquired by an image acquisition device (such as a camera) set up by the robot, and the vehicle outline features and logo are extracted based on the image. The vehicle model identification information of the target vehicle is determined by combining the vehicle outline features and logo.

[0047] The operation script is used to control the vehicle door handles to perform door operations.

[0048] For example, an operation script includes a sequence of operation primitives corresponding to multiple operation primitives. An operation primitive refers to an atomic operation unit that constitutes an operation script, and each operation primitive includes execution parameters and preset conditions. The preset conditions include preset success conditions and / or preset failure conditions. The preset success conditions and / or preset failure conditions are represented by force / torque thresholds, displacement / angle thresholds, time thresholds, etc.

[0049] For example, each operation primitive can be reused for flexible combination to obtain different operation scripts. Execution parameters and preset conditions can also be modified.

[0050] For example, door operations include door opening and door closing. In this embodiment, when the door operation is a door opening operation, the robot's execution component can clean the vehicle interior. After cleaning is complete, the door closing operation can be performed.

[0051] S120. Based on the operation script, send the first control information to the robot's execution components through the first state machine.

[0052] The first state machine is used to describe the robot's behavior as it transitions between different states while completing its operation script.

[0053] The first control information is used to control the robot's execution components to sequentially execute multiple operation primitives corresponding to the operation script.

[0054] S130. During the execution of the operation primitive by the robot, the robot's sensor data and the preset conditions of the operation primitive are compared, and the first state machine is driven to jump according to the comparison result.

[0055] Understandably, the robot is equipped with a sensor array, through which sensor data is obtained. This sensor data indicates the robot's current state.

[0056] Different vehicle models have different structural characteristics for their door handles, including handle type (e.g., pull-out, lift-up, electronic button, concealed, semi-concealed) and geometric parameters (e.g., handle length, width). In this embodiment, expert experience can be translated into executable operation scripts. By determining the corresponding operation script based on the target vehicle's model identification information, it can be ensured that the process of controlling the vehicle's door handle to perform door operations is adapted to the structural characteristics of the target vehicle's door handle. Furthermore, the operation primitives of each operation script can be reused, thereby avoiding the lengthy development cycle of requiring separate programming for each vehicle model to perform door operations.

[0057] According to the operation script, the first control information is sent to the robot's execution components via the first state machine, which drives the robot's execution components to sequentially execute multiple operation primitives corresponding to the operation script. Simultaneously, by comparing the robot's sensor data with the preset conditions of the operation primitives and driving the first state machine to jump based on the comparison result, the execution state of each operation primitive can be detected in real time, and the first state machine can be accurately driven to jump based on the execution state. This process enables closed-loop control from vehicle model recognition to the completion of the door operation task, improving the robot's adaptability to differences in vehicle model structures and reducing the door operation failure rate.

[0058] In addition, the first state machine can detect and manage the process of the robot executing the operation script, enabling continuous operation of door operation tasks in a flexible and automated manner.

[0059] Figure 3 This is a flowchart illustrating a control method according to yet another embodiment of this application. Figure 3 As shown, in one possible embodiment, step S130 includes steps S131 to S134.

[0060] S131. Collect sensor data when the robot executes operation primitives.

[0061] S132. Compare the sensor data with the preset success conditions and / or preset failure conditions of the operation element to obtain the comparison results.

[0062] S133. If the comparison result meets the preset success condition, then drive the first state machine to jump to the next operation primitive.

[0063] S134. If the comparison result meets the preset failure condition, the first state machine is driven to jump to the exception handling state.

[0064] In this embodiment, sensor data collected during robot operation can be used to obtain the real-time status of the operation. The sensor data is compared with preset success and / or failure conditions of the operation, and the comparison result is used to provide feedback on whether the current operation has succeeded or failed. In the case of success, the first state machine is driven to jump to the next operation to sequentially execute the sequence of operation elements in the operation script, thereby completing the door operation task. In the case of failure, the first state machine is driven to jump to an exception handling state for exception handling. This avoids the high failure rate caused by the lack of a feedback mechanism in fixed-flow control.

[0065] like Figure 3 As shown, in one possible embodiment, after the first state machine is driven to jump to the exception handling state in step S134, the control method further includes step S135.

[0066] S135. In the abnormal handling state, the second control information is sent to the robot's execution component according to the abnormal handling strategy corresponding to the preset failure condition.

[0067] It should be noted that each preset failure condition can be associated with a corresponding exception handling strategy. Therefore, based on the preset failure condition, the corresponding exception handling strategy can be determined. Exception handling strategies may include, for example, one of the following: retry, jump to a specific state, or abort the door operation task.

[0068] The second control information is used to control the robot's execution components to perform operations so that the operation primitives that meet the preset failure conditions are restored to the preset state.

[0069] In this embodiment, during anomaly handling, a second control message is sent to the robot's execution components according to the anomaly handling strategy corresponding to the preset failure conditions, enabling fine-grained handling of various preset failure conditions. The second control message controls the robot's execution components to perform operations so that the operation primitives that meet the preset failure conditions return to their preset states. This achieves self-recovery from the anomaly handling state and improves the success rate of door operation tasks.

[0070] In one possible embodiment, at least one of the operation script and exception handling strategy is stored in a knowledge base and can be invoked. This can be understood as the knowledge in the knowledge base including at least one of the operation script and exception handling strategy.

[0071] The step S110 above, which determines the corresponding operation script based on the target vehicle's model identification information, may include: calling the corresponding model's operation script from the knowledge base based on the target vehicle's model identification information.

[0072] The step S135 above, which involves sending the second control information to the robot's execution component according to the exception handling strategy corresponding to the preset failure condition, includes: calling the corresponding exception handling strategy from the knowledge base according to the preset failure condition and sending the second control information to the robot's execution component according to the exception handling strategy.

[0073] In this embodiment, a knowledge base can store operation scripts for various vehicle models and various exception handling strategies. These operation scripts and exception handling strategies are reusable. Maintenance personnel can maintain the operation scripts and exception handling strategies in the knowledge base to expand the operation scripts for new vehicle models and modify exception handling strategies to adapt to various application scenarios.

[0074] The following example illustrates the door operation task performed using the first state machine:

[0075] The target vehicle model corresponds to an outward-pull door handle type.

[0076] The states of the first state machine include: idle state, initialization state (used to load the knowledge base), approach state (detecting the approaching vehicle door handle through the view camera), positioning state (positioning the vehicle door handle), execution of the current operation primitive state, detection state (used to detect whether the preset success condition or preset failure condition has been met), operation primitive execution successful state, operation primitive execution failed state, exception handling state, door operation completed state, and door operation task aborted state.

[0077] Specifically, after determining the vehicle model identification information of the target vehicle, the first state machine transitions from the idle state to the initialization state to load the knowledge base and the corresponding operation script. Then, the first state machine detects the current position and sequentially reaches and transitions to the positioning state. Next, the first state machine controls the robot's execution components to execute the sequence of operation primitives from the operation script.

[0078] For example, the sequence of operation primitives included in the operation script is as follows: move to the pre-grab pose (MOVE_TO_PREGRIP), slide the probe along the path (SLIDE_PROBE), adaptive clamp (ADAPTIVE_GRIP), pull along the translation direction (PULL_TRANSLATE), pull around the axis by rotation (PULL_ROTATE), press the button (PUSH_BUTTON), and wait for a specific signal WAIT_FOR_SIGNAL.

[0079] Let's illustrate this with some specific examples of operation primitives. For instance, the execution parameters of the operation primitive "Move to Pre-grasp Position" include a preset pose, which is represented by the position x in the x-direction, the position y in the y-direction, the position z in the z-direction, and the attitude angle. The corresponding preset success condition is that the robot's current pose meets the threshold corresponding to the preset pose, and the preset failure condition is that the robot's current pose does not meet the threshold corresponding to the preset pose.

[0080] The execution parameters of the sliding probe operation primitive are the sliding path and the maximum force. The corresponding preset success condition is that the path of the robot's current position relative to the initial position reaches the threshold corresponding to the sliding path, and the preset failure condition is that the path of the robot's current position relative to the initial position does not reach the threshold corresponding to the sliding path.

[0081] The execution parameters of the adaptive clamping operation primitive are the desired force and the clamping time. The corresponding preset success condition is that the force of the robot clamping the door handle of the vehicle reaches the desired force threshold, and the corresponding preset failure condition is that the duration of the robot clamping the door handle of the vehicle exceeds the clamping time and the force of the robot clamping the door handle of the vehicle does not reach the desired force threshold.

[0082] The execution parameters for the operation primitive that waits for a specific signal are the signal type and duration. When the signal type is a force signal, the corresponding preset success condition is a forceful drop. When the signal type is an audio signal, the corresponding preset success condition is a preset sound, which indicates that the door opening operation was successful.

[0083] During the execution of any operation primitive in the operation script by the robot's actuators, the first state machine synchronously triggers a detection state. This allows the robot to simultaneously compare sensor data with the preset conditions of the current operation primitive. If the current operation primitive is executed successfully, the first state machine transitions to the operation primitive execution success state, and the robot can load and execute the next operation primitive. If the current operation primitive fails or times out, the first state machine transitions to the operation primitive execution failure state, and then transitions to the exception handling state.

[0084] In the exception handling state, the robot searches the knowledge base for and executes the corresponding exception handling strategy based on the preset failure condition corresponding to the current operation primitive. This exception handling strategy may include, for example, instructing the first state machine to return to the previous state for retry, jumping to a specific state (such as the positioning state), or jumping to the door operation task abort state.

[0085] To illustrate the exception handling strategy, consider the following specific examples: When the current operation primitive is adaptive clamping, and the preset failure condition for this operation primitive indicates clamping slippage, the corresponding exception handling strategy includes the following steps executed sequentially: releasing the execution component (such as the gripper), reverting to the operation primitive that moves to the pre-grabbing pose, increasing the expected force threshold of the adaptive clamping operation primitive by 10%, and retrying adaptive clamping. The number of retries is limited to a maximum of two.

[0086] If the preset failure condition indication for the current operation primitive is stuck, the corresponding exception handling strategy includes the following steps executed in sequence: release the execution unit, report the stuck state, and terminate.

[0087] If the visual tracking is lost due to the preset failure condition indicated by the current operation primitive, the corresponding exception handling strategy includes the following steps executed in sequence: maintain the current force and re-enter the positioning state.

[0088] After the last operation primitive in the operation script is successfully executed, the first state machine can enter the door operation completed state. Alternatively, after the last operation primitive in the operation script is successfully executed and the door operation verification conditions are met, the first state machine can enter the door operation completed state.

[0089] When the door operation is an opening operation, the corresponding door operation verification condition includes the visual camera detecting that the gap between the door and the vehicle is greater than or equal to a threshold. When the door operation is a closing operation, the corresponding door operation verification condition includes the visual camera detecting that the gap between the door and the vehicle is less than a threshold.

[0090] In one possible embodiment, the control method may further include: recording the execution parameters and operation results of the operation primitives during the execution of operation primitives by the robot's execution components; and adjusting and updating preset success conditions and / or preset failure conditions based on the execution parameters and operation results through a machine learning model.

[0091] For example, the machine learning model is a convolutional neural network (CNN) or a long short-term memory (LSTM) network.

[0092] The machine learning model is obtained by training an initial machine learning model. The initial machine learning model and the initial machine learning model have the same model structure but different model parameters. During the robot's execution of operational primitives, the execution parameters and operation results of the primitives are recorded to generate training samples. These training samples have parameter threshold labels indicating preset success or failure conditions. The training samples are input into the initial machine learning model, which predicts the parameter thresholds for the preset success or failure conditions. A loss function is used to evaluate the difference between the parameter thresholds predicted by the initial machine learning model and the parameter threshold labels of the training samples. If the loss function does not converge, the next round of iterative training is performed. If the loss function converges, the model parameters of the current initial machine learning model are fixed, resulting in the final machine learning model.

[0093] The trained machine learning model learns the success and failure characteristics of operational primitives under various conditions. Based on this, during the robot's execution of operational primitives, the execution parameters and operation results of the primitives are recorded. Based on the execution parameters and operation results, the machine learning model adjusts and updates the preset success and / or preset failure conditions, enabling adaptive updates to the preset success and / or preset failure conditions to ensure the accuracy of the robot's success or failure in executing operational primitives.

[0094] It should be noted that during the same execution of the control method of this application embodiment, the preset success condition and / or preset failure condition remain consistent. After adjusting and updating the preset success condition and / or preset failure condition through a machine learning model, the robot can apply the updated preset success condition and / or preset failure condition to execute the control method of this application embodiment on subsequent target vehicles.

[0095] The above embodiment is a detailed description of controlling the door handle to perform door operations. When the door operation is a door opening operation, the robot can also clean the interior of the vehicle.

[0096] Specifically, such as Figure 2 As shown, in one possible embodiment, after the door operation is a door opening operation and the last operation primitive of the operation script is successfully executed, the control method further includes steps S140 to S150.

[0097] S140. Determine the corresponding cleaning strategy based on vehicle model identification information.

[0098] S150: Based on the cleaning strategy, send third control information to the robot's execution components through the second state machine.

[0099] The third control information is used to control the robot's execution components to sequentially execute multiple operation primitives corresponding to the cleaning strategy.

[0100] The second state machine is used to describe the robot's behavior as it transitions between different states during the execution of a cleaning strategy.

[0101] In this embodiment, a corresponding cleaning strategy is determined based on vehicle model identification information, adapting to differences in the location, size, and material of cleanable areas within different vehicles. By sending third control information to the robot's execution components via a second state machine based on the cleaning strategy, the robot's execution components can be controlled to sequentially execute multiple operation primitives corresponding to the cleaning strategy. This enables automated and efficient cleaning of vehicles.

[0102] In addition, the cleaning process of the robot's execution components controlled by the second state machine is similar to the process of the robot's execution components controlled by the first state machine to perform the door operation task, which can ensure that the operation primitives corresponding to each cleaning strategy support detection and recovery.

[0103] like Figure 2 As shown, in one possible embodiment, after the door operation is a door opening operation and the last operation primitive of the operation script is successfully executed, the method further includes step S160. Step S160 may be executed before step S140.

[0104] S160. Obtain the vehicle features corresponding to the door opening operation. The vehicle features include at least one of the following: the maximum opening angle of the door, the exposed area of ​​the interior, and the door movement trajectory.

[0105] The exposed interior areas refer to the areas where the robot's execution components perform the cleaning strategy. These areas include: dashboard, center console, seats, floor, etc.

[0106] When different materials are used in various exposed interior areas, different cleaning tools and parameters can be applied to each area based on the material.

[0107] Cleaning tools include: soft brush, sprayer, vacuum cleaner, and microfiber cloth.

[0108] Cleaning parameters include: detergent dosage, brushing pressure, dwell time, and suction intensity.

[0109] Step S150, which involves sending third control information to the robot's execution component via the second state machine based on the cleaning strategy, includes: determining at least one cleaning sub-task and the cleaning trajectory corresponding to each cleaning sub-task based on the cleaning strategy and vehicle characteristics; generating third control information based on the cleaning sub-task and the corresponding cleaning trajectory; and sending the third control information to the robot's execution component.

[0110] For example, the cleaning strategy specifies the cleaning sub-task for each exposed interior area, as well as the cleaning tools, cleaning parameters, and cleaning trajectory for each cleaning sub-task.

[0111] To illustrate with a concrete example: multiple cleaning sub-tasks are determined based on the cleaning strategy and vehicle characteristics, with each sub-task corresponding to an independent exposed interior area. For instance, multiple sub-tasks could be used to clean the dashboard, center console, seats, and floor, respectively.

[0112] Taking the cleaning sub-task of a car seat as an example, the corresponding operation units for the cleaning strategy are as follows: spraying cleaning agent, circular brushing, vacuuming, and wiping along the cleaning path. Among these, the cleaning agent spraying operation unit is related to the duration and area of ​​the spray. The circular brushing operation unit is related to the brushing pressure and frequency. The vacuuming operation unit is related to the vacuuming power. The wiping along the cleaning path operation unit is related to the cleaning path.

[0113] In this embodiment, vehicle features corresponding to the door opening operation are acquired. These features include at least one of the following: the maximum door opening angle, the exposed interior area, and the door movement trajectory. The maximum door opening angle is used to determine the space within the vehicle's interior for the robot's actuators. The door movement trajectory is used to determine the avoidance path for the robot's actuators, thereby determining the cleaning trajectory of the robot's actuators. This avoids interference between the robot's actuators and the vehicle interior during cleaning and accurately identifies the areas within the vehicle interior that require cleaning.

[0114] By determining at least one cleaning sub-task and its corresponding cleaning trajectory based on cleaning strategies and vehicle characteristics, efficient cleaning sub-tasks can be generated based on the material and location of exposed interior areas. For example, different cleaning tools can be used for exposed interior areas made of different materials to avoid damaging the vehicle. Adjacent exposed interior areas can be cleaned continuously (this continuous cleaning characteristic can be reflected in the cleaning trajectory), preventing previously cleaned areas from being re-contaminated.

[0115] By generating third control information based on the cleaning sub-tasks and their corresponding cleaning trajectories, and sending this third control information to the robot's execution components, the robot's execution components can be driven to execute the cleaning strategy along the cleaning trajectory. It can be understood that executing the cleaning strategy means executing the corresponding cleaning sub-task.

[0116] It should be noted that, in addition to determining at least one cleaning sub-task and the corresponding cleaning trajectory for each cleaning sub-task based on the cleaning strategy and vehicle characteristics as described in the above embodiments, the robot can determine the cleaning space of its execution components based on vehicle characteristics; and adjust and optimize the order of operation primitives for the cleaning sub-tasks based on parameters such as the cleaning space and the replacement order of various cleaning tools, in order to improve cleaning effectiveness and reduce cleaning time. For example, the optimized order of operation primitives could be vacuuming first and then wiping.

[0117] In one possible embodiment, at least one of the cleaning strategy and the vehicle features corresponding to the door opening operation is stored in a knowledge base and can be accessed. This can be understood as the knowledge in the knowledge base including at least one of the cleaning strategy and the vehicle features corresponding to the door opening operation.

[0118] The step S140 above, which determines the corresponding cleaning strategy based on the vehicle model identification information, may include: retrieving the corresponding vehicle model cleaning strategy from the knowledge base based on the target vehicle's vehicle model identification information.

[0119] In this embodiment, a knowledge base can store cleaning strategies for various vehicle models and various vehicle characteristics, which can be reused. Maintenance personnel can maintain the cleaning strategies and vehicle characteristics in the knowledge base to expand cleaning strategies for new vehicle models and update vehicle characteristics, adapting to various application scenarios.

[0120] In one possible embodiment, the knowledge base also stores the vehicle door handle type, the positioning reference point of the vehicle door handle, and the geometric parameters of the vehicle door handle for each vehicle model. The positioning reference point refers to the approximate coordinates of the vehicle door handle relative to vehicle feature points (such as the wheel center or door gap), used to guide the robot's actuator to move to that position. The geometric parameters refer to the dimensions of the vehicle door handle, which can be represented by parameters of a 3D model, such as length, width, and recess depth. These geometric parameters guide the robot's actuator in setting the pre-angle and insertion depth.

[0121] In summary, the control method, apparatus, and robot of this application embodiment can decompose complex, unstructured door operation tasks into programmable operation primitives, quantifiable preset success conditions and / or preset failure conditions, and preset exception handling strategies, and achieve strict process control through a state machine.

[0122] Building upon this, it also supports controlling the robot's actuators to perform cleaning tasks inside vehicles. Of course, in addition to cleaning, it can be extended to other tasks such as inspecting vehicle interiors, demonstrating broad application prospects.

[0123] Figure 4 This is a schematic diagram of the structure of the control device provided in the embodiments of this application, such as... Figure 4 As shown in the figure, this application embodiment provides a control device, which includes: an operation script determination module 210, a sending module 220, and a comparison module 230.

[0124] The operation script determination module 210 is used to determine the corresponding operation script based on the vehicle model identification information of the target vehicle. The operation script is used to control the vehicle door handle to perform door operation.

[0125] The sending module 220 is used to send first control information to the robot's execution component through a first state machine based on the operation script. The first control information is used to control the robot's execution component to execute multiple operation primitives corresponding to the operation script in sequence.

[0126] The comparison module 230 is also used to compare the robot's sensor data with the preset conditions of the operation primitive during the process of the robot executing the operation primitive, and drive the first state machine to jump according to the comparison result.

[0127] In one possible embodiment, the comparison module includes: a data acquisition submodule for acquiring sensor data when the robot executes an operation primitive; a comparison submodule for comparing the sensor data with preset success conditions and / or preset failure conditions of the operation primitive to obtain a comparison result; a first jump rotor module for driving a first state machine to jump to the next operation primitive if the comparison result meets the preset success conditions; and a second jump rotor module for driving the first state machine to jump to an exception handling state if the comparison result meets the preset failure conditions.

[0128] In one possible embodiment, the sending module is further configured to send second control information to the robot's execution component in an abnormal handling state according to an abnormal handling strategy corresponding to a preset failure condition. The second control information is used to control the robot's execution component to perform an operation so that the operation primitive that meets the preset failure condition is restored to a preset state.

[0129] In one possible embodiment, the control device further includes: a recording module, used to record the execution parameters and operation results of the operation primitives during the execution of the operation primitives by the robot's execution components; and an updating module, used to adjust and update preset success conditions and / or preset failure conditions based on the execution parameters and operation results through a machine learning model.

[0130] In one possible embodiment, the control device further includes: a cleaning strategy determination module, used to determine the corresponding cleaning strategy based on vehicle model identification information; and a sending module, used to send third control information to the robot's execution component through a second state machine based on the cleaning strategy, wherein the third control information is used to control the robot's execution component to sequentially execute multiple operation primitives corresponding to the cleaning strategy.

[0131] In one possible embodiment, the control device further includes: a vehicle feature acquisition module, used to acquire vehicle features corresponding to the door opening operation, the vehicle features including at least one of the following: maximum door opening angle, interior exposed area, and door movement trajectory; the sending module is specifically used to determine at least one cleaning sub-task and a cleaning trajectory corresponding to each cleaning sub-task based on the cleaning strategy and vehicle features; and to generate third control information and send the third control information to the robot's execution component based on the cleaning sub-task and the corresponding cleaning trajectory.

[0132] In one possible embodiment, at least one of the operation script, cleaning strategy, exception handling strategy, and vehicle features corresponding to the door opening operation is stored in a knowledge base and can be invoked.

[0133] In one possible embodiment, the cleaning strategy specifies a cleaning sub-task for each exposed interior area, as well as the cleaning tools, cleaning parameters, and cleaning trajectory for each cleaning sub-task.

[0134] This application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the methods described in the above-described method embodiments.

[0135] The aforementioned computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0136] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0137] This application provides a computer program product, including a computer program that, when executed by a processor, implements the methods provided in any of the embodiments described above.

[0138] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0139] It should be further noted that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0140] It should be understood that the above-described device embodiments are merely illustrative, and the device of this application can also be implemented in other ways. For example, the division of units / modules in the above embodiments is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units, modules, or components may be combined, or integrated into another system, or some features may be ignored or not executed.

[0141] Furthermore, unless otherwise specified, the functional units / modules in the various embodiments of this application can be integrated into one unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated together. The integrated units / modules described above can be implemented in hardware or as software program modules.

[0142] When integrated units / modules are implemented in hardware, the hardware can be digital circuits, analog circuits, etc. The physical implementation of the hardware structure includes, but is not limited to, transistors, memristors, etc. Unless otherwise specified, the processor can be any suitable hardware processor, such as a CPU, GPU, FPGA, DSP, and ASIC, etc. Unless otherwise specified, the storage unit can be any suitable magnetic or magneto-optical storage medium, such as Resistive Random Access Memory (RRAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Enhanced Dynamic Random Access Memory (EDRAM), High-Bandwidth Memory (HBM), Hybrid Memory Cube (HMC), etc.

[0143] If the integrated unit / module is implemented as a software program module and sold or used as an independent financial product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software financial product. This computer software financial product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0144] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0145] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0146] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A control method, characterized in that, include: The corresponding operation script is determined based on the vehicle model identification information of the target vehicle. The operation script is used to control the vehicle door handle to perform door operation. Based on the operation script, a first control message is sent to the robot's execution component through a first state machine. The first control message is used to control the robot's execution component to execute multiple operation primitives corresponding to the operation script in sequence. During the execution of the operation primitive by the robot, the robot's sensor data and the preset conditions of the operation primitive are compared, and the first state machine is driven to jump according to the comparison result.

2. The method according to claim 1, characterized in that, The process of comparing the robot's sensor data with the preset conditions of the operation primitive during the robot's execution, and driving the first state machine to jump based on the comparison result, includes: Collect sensor data when the robot executes the operation element; The sensor data is compared with the preset success conditions and / or preset failure conditions of the operation element to obtain the comparison results; If the comparison result meets the preset success condition, then the first state machine is driven to jump to the next operation primitive; If the comparison result meets the preset failure condition, the first state machine is driven to jump to the exception handling state.

3. The method according to claim 2, characterized in that, After driving the first state machine to jump to the exception handling state, the method further includes: In the abnormal handling state, a second control message is sent to the robot's execution component according to the abnormal handling strategy corresponding to the preset failure condition. The second control message is used to control the robot's execution component to perform an operation so that the operation primitive that meets the preset failure condition is restored to the preset state.

4. The method according to claim 2, characterized in that, The method further includes: During the execution of the operation primitive by the robot's execution component, the execution parameters and operation results of the operation primitive are recorded; Based on the execution parameters and operation results, the preset success conditions and / or the preset failure conditions are adjusted and updated through a machine learning model.

5. The method according to claim 3, characterized in that, After the door operation is a door opening operation and the last operation primitive of the operation script is successfully executed, the method further includes: Based on the vehicle model identification information, determine the corresponding cleaning strategy; Based on the cleaning strategy, a third control message is sent to the robot's execution component via a second state machine. The third control message is used to control the robot's execution component to sequentially execute multiple operation primitives corresponding to the cleaning strategy.

6. The method according to claim 5, characterized in that, After the door operation is a door opening operation and the last operation primitive of the operation script is successfully executed, the method further includes: Obtain the vehicle features corresponding to the door opening operation, wherein the vehicle features include at least one of the following: maximum door opening angle, interior exposed area, and door movement trajectory; The step of sending third control information to the robot's execution components via a second state machine based on the cleaning strategy includes: Based on the cleaning strategy and the vehicle characteristics, at least one cleaning sub-task and the cleaning trajectory corresponding to each cleaning sub-task are determined. Based on the cleaning sub-task and the corresponding cleaning trajectory, third control information is generated and sent to the robot's execution components.

7. The method according to claim 6, characterized in that, The operation script, the cleaning strategy, the exception handling strategy, and at least one of the vehicle features corresponding to the door opening operation are stored in the knowledge base and can be called.

8. The method according to claim 6, characterized in that, The cleaning strategy specifies the cleaning sub-tasks for each exposed interior area, as well as the cleaning tools, cleaning parameters, and cleaning trajectory for each cleaning sub-task.

9. A control device, characterized in that, include: The operation script determination module is used to determine the corresponding operation script based on the vehicle model identification information of the target vehicle. The operation script is used to control the vehicle door handle to perform door operation. The sending module is used to send first control information to the robot's execution component through a first state machine based on the operation script. The first control information is used to control the robot's execution component to execute multiple operation primitives corresponding to the operation script in sequence. The comparison module is also used to compare the robot's sensor data with the preset conditions of the operation primitive during the process of the robot executing the operation primitive, and drive the first state machine to jump according to the comparison result.

10. A robot, characterized in that, include: Memory, processor, and execution unit; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-8.

Images