Octopus tentacle imitating cooking mechanical arm control method, device and system

Abstract

The invention discloses a control method, device and system for a cooking mechanical arm simulating an octopus tentacle. The method comprises the steps that visual information sent by a sub-controller of the tentacle is received; the visual information comprises target food material information and cooking environment information; determining a cooking action sequence of the cooking mechanical arm based on the visual information; the cooking action sequence comprises at least one cooking action of at least one cooking stage; splitting the cooking action sequence into a plurality of sub-task packets; each subtask package comprises task information; sending the plurality of sub-task packets to sub-controllers of a plurality of tentacles; receiving competition information sent by the sub-controller; the competition information is determined by the sub-controller based on the task information and the current state information; and distributing a plurality of sub-task packets based on the competition information corresponding to the plurality of tentacles. According to the embodiment of the invention, the sub-controllers are arranged on the flexible octopus-like tentacles, so that a distributed system is formed, and flexible and continuous cooking operation is realized.

Description

Technical Field

[0001] This application relates to the field of cooking robot technology, and in particular to a cooking robotic arm control method, device and system that mimics octopus tentacles. Background Technology

[0002] With the rapid development of smart home and automation technologies, the demand for intelligent equipment in kitchen settings is increasing. As the core execution unit of kitchen automation, robotic arms have been gradually applied to basic cooking tasks such as food handling, stir-frying, and mixing.

[0003] Currently, most existing robotic arms for cooking employ rigid mechanical structures and operate using pre-programmed instructions. However, the rigid structure and fixed joint design make them ill-suited to the complex environment of a kitchen, resulting in insufficient flexibility. Furthermore, most current robotic arms rely on a central controller to uniformly schedule all joints and actuators based on a pre-programmed sequence. If the central processor malfunctions, experiences communication interruptions, or fails due to computing power overload, the entire robotic arm will become paralyzed and unable to complete subsequent operations. Therefore, existing robotic arms for cooking struggle to achieve flexible and safe continuous cooking operations. Summary of the Invention

[0004] To address the technical problem of existing robotic arms struggling to perform flexible and safe continuous cooking operations, this invention provides a control method, device, and system for a robotic arm with tentacles resembling octopus tentacles. By setting sub-controllers on the flexible octopus tentacles, multiple sub-controllers form a distributed system with the main controller. This distributed intelligent system ensures that if one tentacle fails, the others can continue to work, improving system reliability. The sub-controllers collect information and execute specific tasks, while the main controller is responsible for unified task scheduling, effectively improving the efficiency of cooking task execution and ultimately achieving flexible and continuous cooking operations.

[0005] In a first aspect, embodiments of this application provide a control method for a cooking robotic arm that mimics octopus tentacles, applied to the main controller of the cooking robotic arm, comprising: Receive visual information sent by the sub-controller of the tentacle; the visual information includes target ingredient information and cooking environment information; The cooking motion sequence of the cooking robot arm is determined based on visual information; the cooking motion sequence includes at least one cooking motion in at least one cooking stage; The cooking action sequence is broken down into multiple sub-task packages; each sub-task package includes task information. Send multiple subtask packages to the sub-controllers of multiple tentacles; Receive contention information sent by the sub-controller; the contention information is determined by the sub-controller based on task information and current status information; Multiple subtask packages are allocated based on the competitive information corresponding to multiple tentacles.

[0006] In one optional embodiment, determining the cooking motion sequence of the cooking robotic arm based on visual information includes: Obtain a user cooking habit model; the user cooking habit model is trained based on historical cooking data. Visual information is input into the user's cooking habit model to obtain the cooking action sequence of the cooking robot arm.

[0007] In one optional embodiment, the competition information includes the current tentacle's task matching status and task score; the task matching status is used to characterize whether the current tentacle and the subtask package match. Multiple sub-task packages are allocated based on the competitive information corresponding to multiple tentacles, including: Execute for each of the multiple subtask packages: The currently executing subtask package is designated as the target subtask package; Based on the task matching status, at least one executable tentacle that matches the target subtask package is identified among multiple tentacles; Based on the task score of at least one executable tentacle, a target tentacle is determined among at least one executable tentacle; the task score of the target tentacle is greater than or equal to the task scores of the other tentacles among at least one executable tentacle besides the target tentacle; Assign the target subtask package to the target tentacle.

[0008] In an optional embodiment, the method further includes: If a parameter request message is received from the sub-controller, the drive command is determined based on the parameter request message; the parameter request message is determined by the sub-controller based on tactile information. Send drive commands to the sub-controller that sent the parameter request information.

[0009] Secondly, embodiments of this application provide a control method for a cooking robotic arm mimicking octopus tentacles, applied to sub-controllers of multiple tentacles of the cooking robotic arm, including: The robot acquires and sends visual information to the main controller of the cooking robotic arm; the visual information includes target ingredient information and cooking environment information; the visual information is used to determine the cooking action sequence. Receives multiple sub-task packets sent by the main controller; each sub-task packet includes task information; the sub-task packets are obtained by the main controller based on the cooking action sequence. Obtain the current state information of multiple tentacles; Based on the current state information and task information, determine the competition information of multiple tentacles; Send contention information to the main controller; the contention information is used to assign multiple subtask packages to multiple tentacles; Receive subtask packages assigned by the main controller and perform cooking operations based on the subtask packages.

[0010] In one optional embodiment, the current state information includes current posture information, remaining degrees of freedom, and fault flag information; based on the current state information and task information, competition information among multiple tentacles is determined, including: Execute on each of the multiple tentacles: Define the currently executing tentacle as the current tentacle; Based on the current posture information, remaining degrees of freedom, and multiple task information of the current tentacle, determine the task score of the current tentacle and each task information; If the task score is less than the preset score threshold, or the fault flag indicates that the current tentacle is in a fault state, it is determined that the task matching state of the current tentacle is not a match; or, if the task score is greater than or equal to the preset score threshold, and the fault flag indicates that the current tentacle is in a normal state, it is determined that the task matching state of the current tentacle is a match. Based on the task score and task matching status of each current tentacle, the competition information of multiple tentacles is determined.

[0011] In an optional embodiment, the method further includes: Obtain tactile information about objects; Determine the surface friction coefficient and stiffness information of an object based on tactile information; Adsorption force parameters are determined based on surface friction coefficient and contact model; the contact model is established based on a micro-suction cup array and a tactile sensing module array set on multiple tentacles; Envelope force parameters are determined based on stiffness information and contact model; Based on the adsorption force parameters and the envelope force parameters, determine the parameter request information; Send parameter request information to the main controller; If a drive command is received from the main controller based on parameter request information, the micro suction cup array will be controlled based on the drive command.

[0012] In one alternative embodiment, the sub-task package includes an ideal motion trajectory; the method further includes: Obtain the current motion trajectory; Determine the trajectory deviation based on the current trajectory and the ideal trajectory; If the deviation of the motion trajectory is greater than the preset deviation threshold, the motion trajectory deviation is sent to the main controller; the motion trajectory deviation is used to update the user's cooking habit model; the user's cooking habit model is trained based on historical cooking data and is used to determine the cooking action sequence.

[0013] Thirdly, embodiments of this application provide a control device for a cooking robotic arm that mimics octopus tentacles, applied to the main controller of the cooking robotic arm, comprising: The first receiving module is used to receive visual information sent by the sub-controller of the tentacle; the visual information includes target food information and cooking environment information; The first determining module is used to determine the cooking action sequence of the cooking robot arm based on visual information; the cooking action sequence includes at least one cooking action in at least one cooking stage; The task splitting module is used to break down a cooking action sequence into multiple sub-task packages; each sub-task package includes task information. The first sending module is used to send multiple subtask packages to the sub-controllers of multiple tentacles; The second receiving module is used to receive the contention information sent by the sub-controller; the contention information is determined by the sub-controller based on the task information and the current status information. The task allocation module is used to allocate multiple sub-task packages based on the competition information corresponding to multiple tentacles.

[0014] Fourthly, embodiments of this application provide a control device for a cooking robotic arm resembling octopus tentacles, characterized in that a sub-controller applied to multiple tentacles of the cooking robotic arm includes: The second sending module is used to acquire and send visual information to the main controller of the cooking robotic arm; the visual information includes target ingredient information and cooking environment information; the visual information is used to determine the cooking action sequence. The third receiving module is used to receive multiple sub-task packets sent by the main controller; the sub-task packets include task information; the sub-task packets are obtained by the main controller based on the cooking action sequence. The first acquisition module is used to acquire the current status information of multiple tentacles; The second determining module is used to determine the competition information of multiple tentacles based on the current state information and task information; The third sending module is used to send contention information to the main controller; the contention information is used to allocate multiple sub-task packages to multiple tentacles. The task execution module is used to receive sub-task packages assigned by the main controller and perform cooking operations based on the sub-task packages.

[0015] Fifthly, embodiments of this application provide a cooking robotic arm system that mimics octopus tentacles, including a main controller and a cooking robotic arm; The cooking robotic arm includes multiple tentacles; each tentacle includes a sub-controller, a vision acquisition module, a tactile acquisition module, and a micro suction cup array; the sub-controller is communicatively connected to the vision acquisition module and the tactile acquisition module, and receives visual and tactile information collected and sent by the vision acquisition module and the tactile acquisition module; Multiple sub-controllers communicate with the main controller, sending visual and tactile information to the main controller and receiving tasks assigned by the main controller based on the visual and tactile information; the sub-controllers also communicate with the micro suction cup array, controlling the micro suction cup array to perform tasks.

[0016] Sixthly, embodiments of this application provide an intelligent kitchen appliance, which includes a processor and a memory. The memory stores at least one instruction, at least one program, code set, or instruction set. The processor loads and executes the at least one instruction, at least one program, code set, or instruction set to implement the octopus-tentacle-like cooking robotic arm control method of the first aspect.

[0017] In a seventh aspect, embodiments of this application provide a computer-readable storage medium storing at least one instruction or at least one program, wherein the at least one instruction or at least one program is loaded and executed by a processor to implement the octopus-tentacle-like cooking robotic arm control method of the first aspect.

[0018] Eighthly, embodiments of this application provide a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the octopus-tentacle-like cooking robotic arm control method of the first aspect.

[0019] The cooking robotic arm control method, device, and system with octopus-tentacle-like tendrils provided in this application have the following technical effects: The system receives visual information sent by the sub-controllers of the tentacles; the visual information includes target ingredient information and cooking environment information; determines the cooking action sequence of the cooking robot arm based on the visual information; the cooking action sequence includes at least one cooking action in at least one cooking stage; divides the cooking action sequence into multiple sub-task packages; each sub-task package includes task information; sends the multiple sub-task packages to the sub-controllers of the multiple tentacles; receives competition information sent by the sub-controllers; the competition information is determined by the sub-controllers based on the task information and current state information; and allocates multiple sub-task packages based on the competition information corresponding to the multiple tentacles. In this embodiment, by setting sub-controllers on the flexible octopus-like tentacles, multiple sub-controllers and the main controller form a distributed system. The distributed intelligent system ensures that when one tentacle fails, the other tentacles can continue to work, improving system reliability. The sub-controllers collect information and specifically execute tasks, while the main controller is responsible for unified task scheduling, effectively improving the efficiency of cooking task execution and ultimately achieving flexible and continuous cooking operations. Attached Figure Description

[0020] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a schematic diagram of an application environment provided in an embodiment of this application; Figure 2 This is a flowchart illustrating a cooking robotic arm control method that mimics octopus tentacles, as provided in an embodiment of this application. Figure 1 ; Figure 3 This is a flowchart illustrating a cooking robotic arm control method that mimics octopus tentacles, as provided in an embodiment of this application. Figure 2 ; Figure 4 This is a flowchart illustrating a cooking robotic arm control method that mimics octopus tentacles, as provided in an embodiment of this application. Figure 3 ; Figure 5 This is a flowchart illustrating a cooking robotic arm control method that mimics octopus tentacles, as provided in an embodiment of this application. Figure 4 ; Figure 6 This is a flowchart illustrating a cooking robotic arm control method that mimics octopus tentacles, as provided in an embodiment of this application. Figure 5 ; Figure 7 This is a schematic diagram of the structure of a cooking robotic arm control device that mimics octopus tentacles, provided in an embodiment of this application. Figure 1 ; Figure 8 This is a schematic diagram of the structure of a cooking robotic arm control device that mimics octopus tentacles, provided in an embodiment of this application. Figure 2 ; Figure 9 This is a hardware structure block diagram of a server for a cooking robotic arm control method that mimics octopus tentacles, as provided in an embodiment of this application. Detailed Implementation

[0022] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0023] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0024] Please see Figure 1 , Figure 1 This is a schematic diagram of an application environment provided in an embodiment of this application, a cooking robotic arm system that mimics octopus tentacles, including a main controller 101 and a cooking robotic arm 102.

[0025] In this embodiment of the application, the cooking robotic arm 102 includes a plurality of tentacles 103. Each of the plurality of tentacles 103 includes a sub-controller 1031, a vision acquisition module 1032, a tactile acquisition module 1033, and a micro suction cup array 1034.

[0026] Specifically, the tentacles 103 of the cooking robotic arm 102 are made of flexible materials, such as silicone or shape memory alloys, and have embedded multi-degree-of-freedom (DOF) actuators, such as micro hydraulic or pneumatic actuators. The tentacles 103 can achieve various movements such as bending, twisting, and extension through these multi-DOF actuators. The multi-DOF actuators can include multiple drive units, each responsible for one degree of freedom of movement. Each drive unit is powered by a hydraulic or pneumatic system, ensuring the flexibility of the tentacles 103's movement.

[0027] In this embodiment of the application, the visual acquisition module 1032 includes a camera for real-time identification of the type, shape and state of food, and a depth sensor for real-time acquisition of the depth information of the item. Through the camera and the depth sensor, visual information (RGB-D in three-dimensional perception data format) containing three-channel color image information and depth image information can be acquired.

[0028] In this embodiment of the application, the tactile acquisition module 1033 can be specifically configured as tactile sensors evenly distributed on the surface of the tentacle 103, used to sense the texture, weight and shape of the object.

[0029] In this embodiment, the micro-suction cup array 1034 consists of multiple micro-suction cups evenly distributed on the surface of the tentacle 103, and the spacing between the suction cups can be dynamically adjusted according to the shape of the object. The suction strength of the suction cups and the degree of envelopment of the tentacle can be dynamically adjusted based on tactile information to achieve better object handling.

[0030] In one possible embodiment, the sub-controller 1031 is communicatively connected to the vision acquisition module and the tactile acquisition module, and receives visual information and tactile information acquired and sent by the vision acquisition module and the tactile acquisition module.

[0031] Multiple sub-controllers 1031 are communicatively connected to the main controller 101, sending visual and tactile information to the main controller 101 and receiving tasks assigned by the main controller 101 based on the visual and tactile information; the sub-controllers are communicatively connected to the micro suction cup array 1034, controlling the micro suction cup array 1034 to perform tasks.

[0032] In this application, the sub-controller 1031 and the main controller 101 communicate with each other via a high-speed communication interface (such as Wi-Fi or Bluetooth).

[0033] In one possible embodiment, the main controller 101 is configured to receive visual information sent by the sub-controllers 1031 of the tentacles 103; the visual information includes target ingredient information and cooking environment information; determine the cooking action sequence of the cooking robotic arm 102 based on the visual information; the cooking action sequence includes at least one cooking action in at least one cooking stage; divide the cooking action sequence into multiple sub-task packages; each sub-task package includes task information; send the multiple sub-task packages to the sub-controllers 1031 of the multiple tentacles; receive competition information sent by the sub-controllers 1031; the competition information is determined by the sub-controllers 1031 based on the task information and current state information; and allocate multiple sub-task packages based on the competition information corresponding to the multiple tentacles 103.

[0034] In one possible embodiment, the sub-controller 1031 is used to acquire and send visual information to the main controller of the cooking robotic arm 102; the visual information includes target ingredient information and cooking environment information; the visual information is used to determine the cooking action sequence; receive multiple sub-task packages sent by the main controller 101; the sub-task packages include task information; the sub-task packages are obtained by the main controller 101 based on the cooking action sequence; acquire the current state information of multiple tentacles 103; determine the competition information of multiple tentacles 103 based on the current state information and task information; send the competition information to the main controller 101; the competition information is used to allocate multiple sub-task packages to multiple tentacles 103; receive the sub-task packages allocated by the main controller 101, and perform cooking operations based on the sub-task packages.

[0035] In this embodiment, sub-controllers 1031 are set on flexible octopus-like tentacles. Multiple sub-controllers 1031 and the main controller 101 form a distributed system. The distributed intelligent system ensures that when one tentacle 103 fails, the other tentacles 103 can continue to work, improving system reliability. The sub-controllers 1031 collect information and execute specific tasks, while the main controller 101 is responsible for unified task scheduling, effectively improving the efficiency of cooking task execution and ultimately achieving flexible and continuous cooking operations.

[0036] The following describes a specific embodiment of the control method for a cooking robotic arm that mimics octopus tentacles, as described in this application. Figure 2 This is a flowchart illustrating a method for controlling a cooking robotic arm resembling octopus tentacles, as provided in an embodiment of this application. This specification provides the method operation steps as shown in the embodiments or flowcharts, but based on conventional or non-inventive labor, more or fewer operation steps may be included. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only possible execution order. In actual system or server products, the method can be executed sequentially according to the embodiments or drawings, or in parallel (e.g., in a parallel processor or multi-threaded processing environment). Specifically, as shown... Figure 2 As shown, this method, applied to the main controller of a cooking robotic arm, may include: S201: Receives visual information sent by the sub-controller of the tentacle; the visual information includes target ingredient information and cooking environment information.

[0037] S202: Determine the cooking action sequence of the cooking robot arm based on visual information; the cooking action sequence includes at least one cooking action in at least one cooking stage.

[0038] S203: Break down the cooking action sequence into multiple sub-task packages; each sub-task package includes task information.

[0039] S204: Send multiple subtask packages to the sub-controllers of multiple tentacles.

[0040] S205: Receive contention information sent by the sub-controller; the contention information is determined by the sub-controller based on task information and current status information.

[0041] S206: Allocate multiple subtask packages based on the competition information corresponding to multiple tentacles.

[0042] Figure 3 This is a flowchart illustrating a cooking robotic arm control method that mimics octopus tentacles, as provided in an embodiment of this application. Figure 2 The method may include: S301: Receives visual information sent by the sub-controller of the tentacle.

[0043] In this embodiment of the application, the visual information includes target ingredient information and cooking environment information.

[0044] Specifically, the vision system can collect visual information including image and depth information. By inputting the visual information into the recognition model, we can obtain the type and freshness of the ingredients, as well as the spatial pose of the ingredients and kitchen utensils in the cooking environment.

[0045] S302: Determine the cooking motion sequence of the cooking robot arm based on visual information.

[0046] In this embodiment of the application, the cooking action sequence includes at least one cooking action in at least one cooking stage. For example, in the food preparation stage, the actions of grabbing ingredients, washing ingredients, and cutting ingredients into preset sizes are performed; in the cooking stage, the actions of grabbing ingredients, stir-frying ingredients, and adding seasonings are performed.

[0047] In one optional embodiment, determining the cooking motion sequence of the cooking robotic arm based on visual information includes: S3021: Obtain a user's cooking habit model.

[0048] In this embodiment, the user cooking habit model is trained based on historical cooking data and stored locally or in the cloud.

[0049] S3022: Input visual information into the user's cooking habit model to obtain the cooking action sequence of the cooking robot arm.

[0050] In another alternative embodiment, this application may also provide a user interaction interface, allowing users to customize the operation mode via voice or a mobile application, and adjust or generate a series of cooking action sequences based on user input instructions.

[0051] Determining the cooking action sequence through a user's cooking habit model is more intelligent, while adjusting it by user input instructions better meets personalized needs.

[0052] S303: Break down the cooking action sequence into multiple sub-task packages.

[0053] In this embodiment, the main controller breaks down the cooking action sequence into multiple sub-task packages. Each sub-task package includes task information, such as target location, required degrees of freedom, time limit, and load requirements, which describes the capabilities required to execute the corresponding sub-task.

[0054] S304: Send multiple subtask packets to the sub-controllers of multiple tentacles.

[0055] In this embodiment, the main controller broadcasts multiple subtask packages to all tentacle sub-controllers via a high-speed bus, allowing the tentacles to bid for and receive the subtasks.

[0056] S305: Receives contention information sent by the sub-controller.

[0057] In one possible embodiment, the competition information is determined by the sub-controller based on task information and current state information. The competition information includes the current task matching status and task score of the tentacle. The task matching status is used to characterize whether the current tentacle and the sub-task package match.

[0058] S306: Assign multiple subtask packages based on the competition information corresponding to multiple tentacles.

[0059] In one possible implementation, the following is performed for each of the multiple subtask packages: S3061: Determine the currently executing subtask package as the target subtask package; S3062: Based on the task matching status, determine at least one executable tentacle among the multiple tentacles that matches the target subtask package.

[0060] In this embodiment of the application, the task matching status includes matching and non-matching. The task matching status can be used to filter out at least one executable tentacle that can perform the target sub-task.

[0061] S3063: Based on the task score of at least one executable tentacle, identify the target tentacle among at least one executable tentacle.

[0062] In the embodiments of this application, the task score of the target tentacle is greater than or equal to the task score of at least one of the executable tentacles other than the target tentacle. In other words, the target tentacle is the tentacle with the highest score among the executable tentacles.

[0063] S3064: Assign the target subtask package to the target tentacle.

[0064] In one possible embodiment, the target subtask package is sent to the target tentacle.

[0065] Through the bidding mechanism described above, when a problem is encountered, all tentacles will send out competitive information to apply for the job at the same time, broadcasting their quantitative scores on whether they can complete the task faster and better. Then, the highest score is awarded the contract after comprehensive consideration. The entire bidding process is completed within 1ms, ultimately achieving more efficient task allocation.

[0066] If a tentacle malfunctions, its task package is immediately re-acquired by an adjacent or redundant tentacle to ensure the continuity of action. In other words, if a tentacle encounters a problem or malfunction and cannot continue to execute the next step, it will find the nearest tentacle that does not have a next step in the process to take over the work, ensuring the smooth execution of the operation.

[0067] After the task is assigned and during task execution, dynamic adjustments can be made to the tentacle gripping mechanism, and so on. Figure 3 As shown, it includes the following steps: S307: Receive parameter request information sent by the sub-controller.

[0068] In one possible embodiment, the parameter request information is determined by the sub-controller based on tactile information, and the tentacle that grasps the object can collect tactile information in real time for dynamic adjustment.

[0069] Specifically, the tactile information of the object is acquired through the tactile acquisition module, which is the tactile sensor. Based on the tactile information, the surface friction coefficient and stiffness information of the object are determined. Based on the surface friction coefficient and contact model, the adsorption force parameters are determined. Based on the stiffness information and contact model, the envelope force parameters are determined. Based on the adsorption force parameters and envelope force parameters, parameter request information is determined and sent to the main controller.

[0070] S308: Determines the driver instruction based on parameter request information.

[0071] S309: Send drive instructions to the sub-controller that sent the parameter request information.

[0072] In one possible embodiment, the main controller determines the drive command based on the parameter request information and sends the drive command to the sub-controller that sent the parameter request information to control the micro suction cup array.

[0073] After task allocation and during task execution, the model can be updated online, continuing as follows: Figure 3 As shown, it includes the following steps: S310: Receives motion trajectory deviation sent by the sub-controller.

[0074] S311: Update the user cooking habit model based on motion trajectory deviation.

[0075] In one possible embodiment, the motion trajectory deviation is determined by the sub-controller based on the acquired current motion trajectory and the ideal motion trajectory. If the motion trajectory deviation exceeds a preset deviation threshold, the deviation is sent to the main controller, triggering online reinforcement learning, updating the local policy network, and synchronizing the weights to the cloud-based user model.

[0076] After the corresponding action sequence is completed, all the sub-controllers of the tentacles send a "task completion code" to the main controller as task completion information. The main controller confirms that there is no abnormality and issues a "low power sleep" command.

[0077] Figure 4 This is a flowchart illustrating a cooking robotic arm control method that mimics octopus tentacles, as provided in an embodiment of this application. Figure 3A sub-controller for multiple tentacles of a cooking robotic arm, the method may include: S401: Acquire and send visual information to the main controller of the cooking robot arm.

[0078] In this embodiment, the visual information includes target ingredient information and cooking environment information; the visual information is used to determine the cooking action sequence.

[0079] S402: Receives multiple subtask packets sent by the main controller.

[0080] In this embodiment, the subtask package includes task information; the subtask package is obtained by the main controller based on the cooking action sequence.

[0081] S403: Obtain the current status information of multiple tentacles.

[0082] In this embodiment of the application, the current state information includes current attitude information, remaining degrees of freedom, and fault flag information.

[0083] Specifically, current attitude information refers to the position and orientation of the tentacle in space; degrees of freedom (DOF) refers to the number of directions in which the tentacle can move independently; remaining degrees of freedom refers to the number of currently unoccupied and available degrees of freedom; and fault flag information is used to characterize whether the tentacle is in a fault state.

[0084] S404: Based on the current state information and task information, determine the competition information of multiple tentacles.

[0085] In one possible embodiment, based on current state information and task information, competition information among multiple tentacles is determined, including: Execute on each of the multiple tentacles: S4041: Determine the currently executing tentacle as the current tentacle.

[0086] S4042: Based on the current posture information, remaining degrees of freedom, and multiple task information of the current tentacle, determine the task score of the current tentacle and each task information.

[0087] In this embodiment, based on the current posture information, remaining degrees of freedom, and multiple task information of the current tentacle, the task score between the current tentacle and each subtask is calculated one by one. Specifically, the position and direction deviation are calculated using the current posture information and the target position to measure whether the tentacle tip can reach the target position and maintain the correct orientation by adjustment. It is also assessed whether the remaining degrees of freedom of the tentacle are sufficient to cover the task requirements and reserve adjustment space. Then, the matching degree between the movement time and the allowed task time is calculated to determine whether the tentacle can complete the action within the task requirement time, and it is verified whether the maximum load of the tentacle can meet the task requirements. Finally, a quantified score is obtained as the task score.

[0088] S4043: Determine the current task matching status of the tentacle based on task score and fault flag information.

[0089] Specifically, if the task score is less than the preset score threshold, or if the fault flag indicates that the current tentacle is in a fault state, the task matching status of the current tentacle is determined to be mismatched.

[0090] If the task score is less than the preset score threshold, it means that the tentacle is not suitable to perform the corresponding subtask. The fault flag information indicates that the current tentacle is in a fault state and cannot perform the corresponding subtask. Therefore, the task matching status of these two types of tentacles is mismatched.

[0091] If the task score is greater than or equal to the preset score threshold, and the fault flag information indicates that the current tentacle is in a normal state, the task matching status of the current tentacle is determined.

[0092] S4044: Based on the task score and task matching status of each current tentacle, determine the competition information of multiple tentacles.

[0093] S405: Sends contention information to the main controller.

[0094] In one possible embodiment, competition information is used to assign multiple subtask packages to multiple tentacles.

[0095] S406: Receives subtask packages assigned by the main controller and performs cooking operations based on the subtask packages.

[0096] Figure 5 This is a flowchart illustrating a cooking robotic arm control method that mimics octopus tentacles, as provided in an embodiment of this application. Figure 4 This method, applied to the sub-controller of the tentacle, may include: S501: Obtain tactile information about an object.

[0097] In this embodiment of the application, during the process of performing a task and taking an item, the sub-controller obtains the tactile information of the item through the tactile sensor on the surface of the touch.

[0098] S502: Determine the surface friction coefficient and stiffness information of an object based on tactile information.

[0099] S503: Adsorption force parameters are determined based on surface friction coefficient and contact model.

[0100] In this embodiment of the application, the contact model is established based on an array of micro-suction cups and an array of tactile sensing modules disposed on multiple tentacles, and is used to characterize the correspondence between surface friction coefficient and adsorption force, stiffness information and envelope force (or area).

[0101] S504: Determine the envelope force parameters based on stiffness information and contact model.

[0102] Therefore, the contact model established through the collaboration of a micro-suction cup array and a tactile sensor array can calculate the required adsorption force and envelope force (or area) based on real-time acquired surface friction coefficient and stiffness information, and make dynamic adjustments to achieve flexible and firm grasping of objects, ensuring the stability and safety of the grasp and avoiding damage to objects due to excessive force. For example, for smooth objects, the suction cups will increase the adsorption force; for fragile objects, the tentacles will reduce the force and increase the envelope area.

[0103] S505: Determine parameter request information based on adsorption force parameters and envelope force parameters.

[0104] S506: Send parameter request information to the main controller.

[0105] S507: Receives drive commands sent by the main controller based on parameter request information.

[0106] S508: Micro suction cup array controlled by drive commands.

[0107] In one possible embodiment, the sub-controller can control the spacing of the micro-suction cup array and the suction force of the suction cups based on drive commands, and can also control the built-in drive device to adjust the bending degree of the tentacles based on drive commands.

[0108] Figure 6 This is a flowchart illustrating a cooking robotic arm control method that mimics octopus tentacles, as provided in an embodiment of this application. Figure 5 The method may include: S601: Obtain the current motion trajectory.

[0109] In this embodiment of the application, during the execution of the task, the current motion trajectory is continuously acquired by sampling at 30 Hz through the visual sampling module and the tactile sampling module.

[0110] S602: Determine the trajectory deviation based on the current trajectory and the ideal trajectory.

[0111] In this embodiment, the ideal motion trajectory is a trajectory planned based on a user's cooking habit model.

[0112] S603: Determine whether the deviation of the motion trajectory is greater than the preset deviation threshold. If yes, execute S604; otherwise, execute S601.

[0113] S604: Sends the motion trajectory deviation to the main controller.

[0114] In this embodiment, the motion trajectory deviation is used to update the user's cooking habit model; the user's cooking habit model is trained based on historical cooking data and is used to determine the cooking action sequence.

[0115] Through continuous updates and learning, a more accurate and efficient model of user cooking habits can be obtained, enabling more accurate and efficient cooking operations in the subsequent control of the cooking robotic arm.

[0116] This application also provides a control device for a cooking robotic arm that mimics octopus tentacles. Figure 7 This is a schematic diagram of the structure of a cooking robotic arm control device that mimics octopus tentacles, provided in an embodiment of this application. Figure 1 ,like Figure 7 As shown, the device 700 is used in the main controller of a cooking robotic arm and includes: The first receiving module 701 is used to receive visual information sent by the sub-controller of the tentacle; the visual information includes target ingredient information and cooking environment information; The first determining module 702 is used to determine the cooking action sequence of the cooking robot arm based on visual information; the cooking action sequence includes at least one cooking action in at least one cooking stage; The task splitting module 703 is used to split the cooking action sequence into multiple sub-task packages; each sub-task package includes task information. The first sending module 704 is used to send multiple subtask packets to the sub-controllers of multiple tentacles; The second receiving module 705 is used to receive the contention information sent by the sub-controller; the contention information is determined by the sub-controller based on the task information and the current status information. The task allocation module 706 is used to allocate multiple sub-task packages based on the competition information corresponding to multiple tentacles.

[0117] In an optional embodiment, it further includes: The first acquisition module is used to acquire the user's cooking habit model; the user's cooking habit model is trained based on historical cooking data. The model prediction module is used to input visual information into the user's cooking habit model to obtain the cooking action sequence of the cooking robot arm.

[0118] In an optional embodiment, the competition information includes the current tentacle's task matching status and task score; the task matching status is used to characterize whether the current tentacle and the sub-task package match; it also includes: Execute for each of the multiple subtask packages: The third determination module is used to determine the currently executing subtask package as the target subtask package; The fourth determination module is used to determine at least one executable tentacle that matches the target subtask package among multiple tentacles based on the task matching status; The fifth determining module is used to determine the target tentacle among at least one executable tentacle based on the task score of at least one executable tentacle; the task score of the target tentacle is greater than or equal to the task scores of the other tentacles among at least one executable tentacle besides the target tentacle; The second task allocation module is used to assign target subtask packages to target tentacles.

[0119] In an optional embodiment, it further includes: The fourth receiving module is used to determine the driving instruction based on the parameter request information sent by the sub-controller if it receives the parameter request information; the parameter request information is determined by the sub-controller based on tactile information. The fourth sending module is used to send drive instructions to the sub-controller that sends parameter request information.

[0120] This application also provides a control device for a cooking robotic arm that mimics octopus tentacles. Figure 8 This is a schematic diagram of the structure of a cooking robotic arm control device that mimics octopus tentacles, provided in an embodiment of this application. Figure 2 ,like Figure 8 As shown, the device 800 is used as a sub-controller for multiple tentacles of a cooking robotic arm, including: The second sending module 801 is used to acquire and send visual information to the main controller of the cooking robotic arm; the visual information includes target ingredient information and cooking environment information; the visual information is used to determine the cooking action sequence. The third receiving module 802 is used to receive multiple sub-task packets sent by the main controller; the sub-task packets include task information; the sub-task packets are obtained by the main controller based on the cooking action sequence. The first acquisition module 803 is used to acquire the current status information of multiple tentacles; The second determining module 804 is used to determine the competition information of multiple tentacles based on the current state information and task information; The third sending module 805 is used to send contention information to the main controller; the contention information is used to allocate multiple sub-task packages to multiple tentacles; The task execution module 806 is used to receive sub-task packages assigned by the main controller and perform cooking operations based on the sub-task packages.

[0121] In one optional embodiment, the current state information includes current attitude information, remaining degrees of freedom, and fault flag information; it also includes: Execute on each of the multiple tentacles: The sixth determination module is used to determine the currently executing tentacle as the current tentacle; The seventh determination module is used to determine the task score of the current tentacle and each task information based on the current posture information, remaining degrees of freedom and multiple task information of the current tentacle. The eighth determination module is used to determine the task matching state of the current tentacle as non-match if the task score is less than the preset score threshold or the fault flag information indicates that the current tentacle is in a fault state; or if the task score is greater than or equal to the preset score threshold and the fault flag information indicates that the current tentacle is in a normal state, it is determined as the task matching state of the current tentacle. The ninth determination module is used to determine the competition information of multiple tentacles based on the task score and task matching status of each current tentacle.

[0122] In an optional embodiment, it further includes: The second acquisition module is used to acquire tactile information of the object; The tenth determination module is used to determine the surface friction coefficient and stiffness information of an object based on tactile information; The eleventh determination module is used to determine the adsorption force parameters based on the surface friction coefficient and the contact model; the contact model is established based on the array of micro suction cups and the array of tactile sensing modules set on multiple tentacles. The twelfth determination module is used to determine the envelope force parameters based on stiffness information and contact model; The thirteenth determination module is used to determine parameter request information based on adsorption force parameters and envelope force parameters; The fifth sending module is used to send parameter request information to the main controller; The fifth receiving module is used to control the micro suction cup array based on the driving command sent by the main controller based on the parameter request information if it receives the driving command.

[0123] In one alternative embodiment, the sub-task package includes an ideal motion trajectory; and further includes: The third acquisition module is used to acquire the current motion trajectory; The fourteenth determination module is used to determine the deviation of the motion trajectory based on the current motion trajectory and the ideal motion trajectory; The sixth sending module is used to send the motion trajectory deviation to the main controller if the motion trajectory deviation is greater than a preset deviation threshold; the motion trajectory deviation is used to update the user cooking habit model; the user cooking habit model is trained based on historical cooking data and is used to determine the cooking action sequence.

[0124] The apparatus and method embodiments in this application are based on the same application concept.

[0125] The methods and embodiments provided in this application can be executed on a computer terminal, server, or similar computing device. Taking running on a server as an example, Figure 9 This is a hardware structure block diagram of a server for a cooking robotic arm control method that mimics octopus tentacles, as provided in an embodiment of this application. Figure 9 As shown, the server 900 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 910 (CPUs 910 may include, but are not limited to, microprocessors such as MCUs or programmable logic devices such as FPGAs), a memory 930 for storing data, and one or more storage media 920 (e.g., one or more mass storage devices) for storing application programs 923 or data 922. The memory 930 and storage media 920 may be temporary or persistent storage. The program stored in the storage media 920 may include one or more modules, each module may include a series of instruction operations on the server. Furthermore, the CPU 910 may be configured to communicate with the storage media 920 and execute the series of instruction operations stored in the storage media 920 on the server 900. Server 900 may also include one or more power supplies 960, one or more wired or wireless network interfaces 950, one or more input / output interfaces 940, and / or one or more operating systems 921, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0126] The input / output interface 940 can be used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of server 900. In one example, the input / output interface 940 includes a network interface controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the input / output interface 940 may be a radio frequency (RF) module used for wireless communication with the Internet.

[0127] Those skilled in the art will understand that Figure 9 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, server 900 may also include... Figure 9 The more or fewer components shown, or having the same Figure 9 The different configurations shown.

[0128] This application provides an intelligent kitchen appliance, which includes a processor and a memory. The memory stores at least one instruction, at least one program, code set, or instruction set. The processor loads and executes the at least one instruction, at least one program, code set, or instruction set to implement the above-described data processing method.

[0129] Embodiments of this application also provide a computer-readable storage medium, which can be disposed in a server to store at least one instruction, at least one program, code set, or instruction set related to implementing a cooking robotic arm control method for octopus tentacles in the method embodiments. The at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the aforementioned cooking robotic arm control method for octopus tentacles.

[0130] Optionally, in this embodiment, the storage medium may be located at at least one of the multiple network servers in a computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0131] As can be seen from the embodiments of the octopus-tentacle-like cooking robotic arm control method, device, and system provided in this application, this application receives visual information sent by the sub-controllers of the tentacles; the visual information includes target ingredient information and cooking environment information; the cooking action sequence of the cooking robotic arm is determined based on the visual information; the cooking action sequence includes at least one cooking action in at least one cooking stage; the cooking action sequence is divided into multiple sub-task packages; each sub-task package includes task information; multiple sub-task packages are sent to the sub-controllers of multiple tentacles; competition information sent by the sub-controllers is received; the competition information is determined by the sub-controllers based on the task information and current state information; multiple sub-task packages are allocated based on the competition information corresponding to multiple tentacles. In the embodiments of this application, by setting sub-controllers on the flexible octopus-tentacle-like tentacles, multiple sub-controllers and the main controller form a distributed system. The distributed intelligent system ensures that when one tentacle fails, other tentacles can still continue to work, improving system reliability. The sub-controllers collect information and specifically execute tasks, while the main controller is responsible for unified task scheduling, effectively improving the efficiency of cooking task execution, and ultimately achieving flexible and continuous cooking operations.

[0132] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, specific embodiments have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0133] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0134] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0135] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An octopus tentacle-imitating cooking robot arm control method, characterized by, A main controller applied to a cooking mechanical arm, comprising: receiving visual information sent by a sub-controller of a touch hand; the visual information comprises target food material information and cooking environment information; determining a cooking action sequence of the cooking mechanical arm based on the visual information; the cooking action sequence comprises at least one cooking action of at least one cooking stage; splitting the cooking action sequence into a plurality of sub-task packages; each of the sub-task packages comprises task information; sending the plurality of sub-task packages to a plurality of sub-controllers of the touch hands; receiving competition information sent by the sub-controllers; the competition information is determined by the sub-controllers based on task information and current state information; allocating the plurality of sub-task packages based on the competition information corresponding to the plurality of touch hands.

2. The control method of the cooking robot arm imitating octopus tentacles according to claim 1, wherein, The method further comprises: if receiving parameter request information sent by the sub-controller, determining a driving instruction based on the parameter request information; the parameter request information is determined by the sub-controller based on tactile information; sending the driving instruction to the sub-controller sending the parameter request information.

3. The control method of the cooking robotic arm imitating octopus tentacles according to claim 1, characterized in that, A sub-controller applied to a plurality of touch hands of a cooking mechanical arm, comprising: obtaining and sending visual information to a main controller of the cooking mechanical arm; the visual information comprises target food material information and cooking environment information; the visual information is used to determine a cooking action sequence; receiving a plurality of sub-task packages sent by the main controller; the sub-task package comprises task information; the sub-task package is obtained by the main controller based on the cooking action sequence; obtaining current state information of the plurality of touch hands; determining competition information of the plurality of touch hands based on the current state information and the task information; sending the competition information to the main controller; the competition information is used to allocate the plurality of sub-task packages for the plurality of touch hands; receiving the sub-task package allocated by the main controller, and performing a cooking operation based on the sub-task package. ​ 4. The control method of the cooking robot arm imitating octopus tentacles according to claim 1, wherein, ​ ​ ​ 5. An octopus tentacle-imitating cooking robot arm control method, characterized by, ​ ​ ​ ​ ​ ​ ​ 6. The control method of claim 5, wherein, The current state information includes current posture information, residual degrees of freedom, and fault flag information; The competition information of the multiple tentacles is determined based on the current state information and the task information, including: For each of the multiple tentacles, the following is performed: A tentacle currently being executed is determined as a current tentacle; Based on the current posture information, the residual degrees of freedom of the current tentacle, and the multiple task information, a task score of the current tentacle and each of the task information is determined; If the task score is less than a preset score threshold, or the fault flag information indicates that the current tentacle is in a fault state, it is determined that the task matching state of the current tentacle is not matched; or if the task score is greater than or equal to the preset score threshold, and the fault flag information indicates that the current tentacle is in a normal state, it is determined that the task matching state of the current tentacle is matched; Based on the task score and the task matching state of each of the current tentacles, the competition information of the multiple tentacles is determined.

7. The control method of claim 5, wherein the plurality of octopus tentacle-like cooking robot arms are controlled to move in a spiral pattern. The method further includes: Obtaining tactile information of an object; Based on the tactile information, surface friction coefficient and stiffness information of the object are determined; Based on the surface friction coefficient and a contact model, adsorption force parameters are determined; the contact model is established based on a micro-suction cup array and a tactile acquisition module array arranged on the multiple tentacles; Based on the stiffness information and the contact model, envelope force parameters are determined; Based on the adsorption force parameters and the envelope force parameters, parameter request information is determined; The parameter request information is sent to the main controller; If a driving instruction sent by the main controller based on the parameter request information is received, the micro-suction cup array is controlled based on the driving instruction.

8. The control method of claim 5, wherein, The sub-task package includes an ideal motion trajectory; the method further includes: Obtaining a current motion trajectory; Based on the current motion trajectory and the ideal motion trajectory, a motion trajectory deviation is determined; If the motion trajectory deviation is greater than a preset deviation threshold, the motion trajectory deviation is sent to the main controller; the motion trajectory deviation is used to update a user cooking habit model; the user cooking habit model is trained based on historical cooking data and is used to determine the cooking action sequence.

9. An octopus tentacle-imitating cooking robot arm control device characterized by comprising: A main controller applied to a cooking robot arm includes: A first receiving module configured to receive visual information sent by a sub-controller of a tentacle; the visual information includes target food material information and cooking environment information; A first determining module configured to determine a cooking action sequence of the cooking robot arm based on the visual information; the cooking action sequence includes at least one cooking action of at least one cooking stage; A task splitting module configured to split the cooking action sequence into multiple sub-task packages; each of the sub-task packages includes task information; A first sending module configured to send the multiple sub-task packages to sub-controllers of the multiple tentacles; A second receiving module configured to receive competition information sent by the sub-controllers; the competition information is determined by the sub-controllers based on task information and current state information; A task allocation module is configured to allocate the multiple sub-task packages based on the competition information corresponding to the multiple tentacles.

10. An octopus tentacle-imitating cooking robot arm control device characterized by comprising: A sub-controller applied to multiple tentacles of a cooking mechanical arm, comprising: A second sending module is configured to acquire and send visual information to a main controller of the cooking mechanical arm; the visual information includes target food material information and cooking environment information; the visual information is used to determine a cooking action sequence; A third receiving module is configured to receive multiple sub-task packages sent by the main controller; the sub-task packages include task information; the sub-task packages are obtained by the main controller based on the cooking action sequence; A first acquiring module is configured to acquire current state information of the multiple tentacles; A second determining module is configured to determine competition information of the multiple tentacles based on the current state information and the task information; A third sending module is configured to send the competition information to the main controller; the competition information is used to allocate the multiple sub-task packages for the multiple tentacles; A task execution module is configured to receive the sub-task packages allocated by the main controller, and execute a cooking operation based on the sub-task packages.

11. An octopus tentacle-mimicking cooking robotic arm system, characterized by, The cooking mechanical arm comprises a main controller and a cooking mechanical arm; The cooking mechanical arm comprises multiple tentacles; each of the multiple tentacles comprises a sub-controller, a visual acquisition module, a tactile acquisition module, and a micro-suction disc array; the sub-controller is in communication connection with the visual acquisition module and the tactile acquisition module, and receives visual information and tactile information collected and sent by the visual acquisition module and the tactile acquisition module; The multiple sub-controllers are in communication connection with the main controller, send the visual information and the tactile information to the main controller, and receive tasks allocated by the main controller based on the visual information and the tactile information; The sub-controller is in communication connection with the micro-suction disc array, and controls the micro-suction disc array to execute the tasks.

Images