Intelligent catering robot system based on ai visual recognition and action method

Abstract

This invention discloses an intelligent catering robot system and motion method based on AI visual recognition. This invention relates to the field of intelligent catering robot technology. It matches target body models with a cloud database and confirms the object to be clamped based on a verification benchmark constructed from a built-in midpoint and farthest point, achieving high precision and robustness in target recognition. Compared to traditional feature matching methods, this method accurately locates the object to be clamped by calculating the mean coordinates in a three-dimensional coordinate system and comparing them with a single-body benchmark. It filters feasible trajectories through interference detection and calculates trajectory features based on the vector length and preset rate of the running nodes, ultimately locking in the time-optimal running scheme, balancing the safety and efficiency of the motion. On one hand, interference detection completely avoids the risk of collisions between the robotic arm and other objects in the environment, ensuring the stability of the operation process. On the other hand, the optimal standard with the lowest time value as the core significantly improves the robot's operating efficiency.

Description

Technical Field

[0001] This invention relates to the field of intelligent catering robot technology, specifically to an intelligent catering robot system and its operation method based on AI visual recognition. Background Technology

[0002] With the rapid development of intelligent catering robot technology, motion planning based on visual recognition has become one of the core capabilities for robots to achieve autonomous operation.

[0003] Application CN111958596B discloses a motion planning system and method for a highly intelligent robot. The motion planning system includes at least a highly intelligent robot and a remotely connected operating terminal for executing environmental intervention actions that cause changes in the external world environment. The highly intelligent robot specifically includes an environmental perception module, a first recognition module, a path planning module, a drive module, a second recognition module, a motion planning module, and an execution module. This technical solution enables autonomous planning of robot actions that cause changes in the external world environment. In particular, for the action tasks of opening and closing doors, the robot autonomously determines the target object characteristics of the door and whether the task has been completed through environmental recognition on the robot side, eliminating the need for complex remote control by humans. It has high intelligence and wide applicability for autonomous task execution.

[0004] In the trajectory planning stage, existing technologies often struggle to balance safety and efficiency. Some solutions can generate collision-free trajectories through path search algorithms, but they fail to consider the dynamic characteristics of the robotic arm's operating nodes, resulting in excessively long trajectory execution times. Other solutions overemphasize efficiency, neglecting the interference risks of dynamic obstacles in complex environments, which can easily lead to collision accidents and affect operational stability. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides an intelligent catering robot system and motion method based on AI visual recognition, which solves the problem of excessively long trajectory execution time caused by not considering the dynamic characteristics of the robotic arm's operating nodes.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a motion method for an intelligent catering robot based on AI visual recognition, comprising the following steps:

[0007] Step 1: Based on the environment of the intelligent catering robot and the AI ​​visual recognition equipment, generate an environmental model of the environment, and record the generated environmental model as a pending model. The specific method is as follows:

[0008] Based on the set AI visual recognition equipment, the environmental images of the surrounding environment of the intelligent catering robot are acquired, and at the same time, the sound wave detection equipment is used to confirm the sound wave characteristics associated with the surrounding environment. Based on the acquired environmental images and sound wave characteristics, an environmental model associated with the current intelligent catering robot is generated.

[0009] Step 2: Based on the input target command, locate the target volume model from the cloud database, and determine the object to be clamped from the undetermined model based on the individual features of the target volume model. The specific method is as follows:

[0010] Based on the confirmed target instructions, the target body model associated with the target instructions is confirmed in the cloud database;

[0011] The built-in midpoint is identified from the target body model. The built-in midpoint has been preset in the corresponding target body model. Then, the farthest point from the built-in midpoint is identified from the target body model. The built-in midpoint and the farthest point are connected to generate a verification benchmark belonging to the target body model.

[0012] Next, feature verification is performed on different individual models within the target model: the individual model is placed in a three-dimensional coordinate system, and the three-dimensional coordinates associated with multiple plane points in the corresponding plane are confirmed and averaged. The average coordinates are then confirmed, and the average points associated with the average coordinates are marked within the individual model as the center point associated with the individual model. From the plane of the individual model, the plane point that is farthest from the center point in a straight line is identified and connected to confirm the individual model reference. The individual model reference corresponding to the individual model is compared and verified with the verification reference of the target model to identify whether the individual model completely overlaps with the target model. The individual model that completely overlaps is recorded as the object to be clamped. Otherwise, the confirmation of the object to be clamped is continued for other individual models.

[0013] Step 3: Based on the confirmed location of the object to be gripped and the robotic arm, determine the robotic arm's running trajectory. Then, from the confirmed sets of running trajectories, select feasible trajectories and confirm the travel plans associated with each different feasible trajectory to determine the optimal running plan. The specific method is as follows:

[0014] The position of the robotic arm in space is confirmed as the starting point, and the position of the object to be gripped in space is confirmed as the ending point. Starting from the starting point, the robotic arm moves towards the ending point, generating several sets of corresponding robotic arm running trajectories.

[0015] Based on the generated sets of running trajectories, it is confirmed whether the corresponding running trajectory interferes with other individual models in the undetermined model. If there is interference, the corresponding running trajectory is removed. If there is no interference, the corresponding running trajectory is marked as a feasible trajectory.

[0016] Place the feasible trajectory in a three-dimensional spatial coordinate system. From the three-dimensional spatial coordinate system, confirm the plane vector associated with the feasible trajectory in a single plane. Based on the starting point and ending point of the feasible trajectory, directly proceed from the starting point to the ending point in the two-dimensional plane of the corresponding three-dimensional spatial coordinate system. Confirm the plane vector associated with the corresponding two-dimensional plane. Then, based on the running nodes associated with the corresponding plane, record the multiple sets of plane vectors associated with the feasible trajectory as the running vectors of the corresponding running nodes.

[0017] Based on the different parallel vectors recorded by different running nodes, confirm the length data L associated with the corresponding parallel vector. i Where i represents different parallel vectors, and then the running speed V of the corresponding running node is determined. i V i All values ​​are preset values, using: L i ÷V i =T i From different T associated with different running nodes i In the middle, select T i max is the trajectory feature of the current feasible trajectory;

[0018] Then, sequentially, the T associated with different feasible trajectories... i Max confirms sequentially, from the confirmed T... i In the max, the minimum value is selected, and the feasible trajectory associated with the minimum value is recorded as the optimal running plan. The intelligent catering robot is then controlled to run according to the confirmed optimal running plan.

[0019] Step 4: Based on the confirmed object to be clamped, and according to the planar features associated with the surface of the object, lock the clamping point on the object and control the corresponding mechanical gripper of the intelligent catering robot to perform the grasping process. The specific method is as follows:

[0020] Record the center point marked inside the clamping body as the reference point, then confirm the outer plane of the clamping body, and based on the outline of the corresponding outer plane, confirm the midpoint of the inner plane of the corresponding outer plane.

[0021] Arbitrarily select a set of built-in plane midpoints and associated reference points, generate a line connecting the built-in plane midpoints and reference points, extend this line, confirm the extended point on another plane, lock the total line distance JL between the built-in plane midpoints and the extended point, and then lock the gripping distance JQ of the robotic gripper.

[0022] If JQ≤JL, select another set of external planes and confirm the midpoint and extension points of the built-in plane, then confirm the distance and identify whether the confirmed JQ satisfies JQ>JL. If it satisfies, use the midpoint and extension points of the built-in plane confirmed in the corresponding process as clamping points. If it does not satisfy, generate an error signal for display.

[0023] If JQ > JL, then the currently confirmed midpoint of the built-in plane and the extended point will be used as the clamping point.

[0024] This invention provides an intelligent catering robot system and its motion method based on AI visual recognition. Compared with existing technologies, it has the following advantages:

[0025] This invention matches target models with a cloud database and confirms the object to be clamped based on a verification benchmark constructed from the built-in midpoint and farthest point, achieving high accuracy and robustness in target recognition. Compared to traditional feature matching methods, this approach accurately locates the object to be clamped by calculating the mean coordinates in a three-dimensional coordinate system and comparing them with a single benchmark. It filters feasible trajectories through interference detection and calculates trajectory features based on the vector length and preset rate of the running nodes, ultimately locking in the time-optimal running scheme, balancing the safety and efficiency of the operation. On the one hand, interference detection can completely avoid the risk of collision between the robotic arm and other objects in the environment, ensuring the stability of the operation process. On the other hand, the optimal standard with the lowest time value as the core significantly improves the robot's operating efficiency.

[0026] In the gripping operation, the gripping point is calculated by using the reference point and external plane features, and the matching judgment of the gripping distance of the mechanical claw and the total connection distance is combined to realize the adaptive adjustment of the gripping action. This design not only ensures the rationality of the gripping point selection and ensures that the mechanical claw can stably grasp the object to be gripped, but also identifies ungraspable scenarios in advance through distance verification, avoids invalid operations, reduces equipment wear and the probability of task interruption, and improves the reliability and flexibility of robot operation. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of the method flow of the present invention;

[0028] Figure 2 This is a schematic diagram of the principle framework of the present invention. Detailed Implementation

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] Example 1

[0031] Please see Figure 1 This application provides a motion method for an intelligent catering robot based on AI visual recognition, including the following steps:

[0032] Step 1: Based on the environment of the intelligent catering robot and the AI ​​visual recognition equipment, generate an environmental model of the environment and record the generated environmental model as a pending model.

[0033] The specific method for generating the environment model is as follows:

[0034] Based on the set AI visual recognition equipment, the environmental images of the surrounding environment of the intelligent catering robot are acquired, and at the same time, the sound wave detection equipment is used to confirm the sound wave characteristics associated with the surrounding environment. Based on the acquired environmental images and sound wave characteristics, an environmental model associated with the current intelligent catering robot is generated. Since the method of generating a model based on images and sound waves is common in existing technologies, it will not be elaborated on here.

[0035] By combining AI visual recognition equipment with acoustic detection equipment to generate an environmental model, the limitations of single visual perception are overcome. This multi-source information fusion method can capture environmental features more comprehensively, reduce model errors caused by factors such as changes in lighting and visual occlusion, provide more reliable basic data for subsequent target recognition and action planning, and improve the robot's adaptability to complex environments.

[0036] Step 2: Based on the input target command, locate the target volume model from the cloud database, and determine the object to be clamped from the undetermined model based on the individual characteristics of the target volume model. The specific method for determining the object to be clamped is as follows:

[0037] Based on the confirmed target instruction (input in advance by the operator), the target body model associated with the target instruction is confirmed in the cloud database (different target body models correspond to different target instructions, so the corresponding target body model can be quickly identified based on the corresponding target instruction).

[0038] Identify the built-in midpoint on the target body model. The built-in midpoint has been preset in the corresponding target body model. Then, identify the farthest point (the plane point with the farthest straight-line distance) from the built-in midpoint on the target body model. Connect the built-in midpoint and the farthest point to generate a verification benchmark belonging to the target body model (if there are multiple sets of farthest points, multiple sets of connection vectors will be formed during the connection process).

[0039] Next, feature verification is performed on different individual models within the target model: the individual model is placed in a three-dimensional coordinate system, and the three-dimensional coordinates associated with multiple plane points in the corresponding plane are confirmed and averaged. The average coordinates are then confirmed, and the average points associated with the average coordinates are marked within the individual model as the center point associated with the individual model. From the plane of the individual model, the plane point that is farthest from the center point in a straight line is identified and connected to confirm the individual model reference. The individual model reference corresponding to the individual model is compared and verified with the verification reference of the target model to identify whether the individual model completely overlaps with the target model. The individual model that completely overlaps is recorded as the object to be clamped. Otherwise, the confirmation of the object to be clamped is continued for other individual models.

[0040] Specifically, based on the confirmed model that needs to be processed, the target body model associated with the corresponding target instruction is identified from the corresponding cloud database. Then, the target body model is compared and verified with the individual model units existing in the corresponding pending model. From the specific process of verification, the body to be gripped in the pending model can be locked. Then, the corresponding intelligent catering robot can automatically complete the gripping process of the body to be gripped. Based on the location of the body to be gripped, the subsequent gripping actions of the intelligent catering robot are planned and the planning process is completed.

[0041] Step 3: Based on the confirmed location of the object to be gripped and the robotic arm, confirm the robotic arm's running trajectory, then select feasible trajectories from the confirmed sets of running trajectories, and confirm the travel plan associated with each different feasible trajectory, thereby locking in the optimal running plan. Specifically, the optimal running plan is the best plan that the corresponding robotic arm can achieve during the gripping process at different running nodes, with its running time characteristics at the lowest value. At the same time, during its running process, it can effectively ensure that the corresponding robotic arm can effectively grip the object to be gripped and complete the corresponding gripping process.

[0042] The specific method for locking the optimal operating plan is as follows:

[0043] The position of the robotic arm in space is confirmed as the starting point, and the position of the object to be gripped in space is confirmed as the ending point. Starting from the starting point, the robotic arm moves towards the ending point, generating several sets of corresponding robotic arm running trajectories.

[0044] Based on the generated sets of running trajectories, it is confirmed whether the corresponding running trajectory interferes with other individual models within the undetermined model. If interference exists, the corresponding running trajectory is eliminated. If there is no interference, the corresponding running trajectory is marked as a feasible trajectory. Interference means that the corresponding running trajectory intersects with the corresponding individual model. If the corresponding robotic arm runs according to the corresponding running trajectory, it will collide with the corresponding individual model, which is considered an interference situation. In this case, it is necessary to confirm the feasible trajectory from multiple sets of running trajectories.

[0045] Place the feasible trajectory in a three-dimensional spatial coordinate system. From the three-dimensional spatial coordinate system, confirm the plane vector associated with the feasible trajectory in a single plane. Based on the starting point and ending point of the feasible trajectory, directly advance from the starting point to the ending point in the two-dimensional plane of the corresponding three-dimensional spatial coordinate system. Confirm the plane vector associated with the corresponding two-dimensional plane. Then, according to the running nodes associated with the corresponding plane (which have been preset in advance), record the multiple sets of plane vectors associated with the feasible trajectory as the running vectors of the corresponding running nodes.

[0046] Based on the different parallel vectors recorded by different running nodes, confirm the length data L associated with the corresponding parallel vector. i Where i represents different parallel vectors, and then the running speed V of the corresponding running node is determined. i V i All values ​​are preset and determined in advance by the operator based on experience. The following values ​​are used: L i ÷V i =T i From different T associated with different running nodes i In the middle, select T i max is the trajectory feature of the current feasible trajectory;

[0047] Then, sequentially, the T associated with different feasible trajectories... i Max confirms sequentially, from the confirmed T... i In the max, the minimum value is selected, and the feasible trajectory associated with the minimum value is recorded as the optimal running plan. Subsequently, the intelligent catering robot is controlled to run according to the confirmed optimal running plan to realize the process of gripping the object to be gripped.

[0048] Specifically, within the corresponding running trajectory, the running characteristics associated with different running trajectories can be identified, and feasible trajectories can be locked from the identified running characteristics. Then, the node running scheme of the feasible trajectory is confirmed. From the corresponding running vector, the running process of each running node during the running process is identified. Under the condition of ensuring the synchronous running of multiple running nodes, the running node with the longest running time is identified, and the time value of the longest running time is used as the time feature of the corresponding running process. In this way, the specific features of the corresponding process are locked, and the optimal trajectory can be identified from several sets of feasible trajectories. In this way, the corresponding optimal running scheme is locked and executed.

[0049] In the target recognition and localization stage, the target model is matched with a cloud database, and the object to be grasped is confirmed based on a verification benchmark constructed with the built-in midpoint and farthest point, achieving high accuracy and strong robustness in target recognition. Compared with traditional feature matching methods, this approach can accurately locate the object to be grasped by calculating the mean coordinates in a three-dimensional coordinate system and comparing them with a single benchmark, effectively avoiding the problem of misidentification of similar objects. This ensures that the robot can quickly locate targets in multi-object mixed scenes, improving the accuracy of task execution.

[0050] The trajectory planning and optimal solution selection mechanism uses interference detection to screen feasible trajectories and calculates trajectory features (Timax) based on the vector length of the running nodes and a preset rate, ultimately locking in the time-optimal running solution, balancing the safety and efficiency of the operation. On the one hand, interference detection can completely avoid the risk of collision between the robotic arm and other objects in the environment, ensuring the stability of the operation process; on the other hand, the optimal standard with the lowest time value as the core significantly improves the robot's operating efficiency, making it particularly suitable for scenarios with high time requirements, such as industrial sorting and service retrieval.

[0051] Step 4: Based on the confirmed object to be clamped, and according to the planar features associated with the surface of the object, lock the clamping point on the object, and control the corresponding mechanical gripper of the intelligent catering robot to perform the grasping process. The specific method for locking the clamping point is as follows:

[0052] Record the center point marked inside the clamping body as the reference point, then confirm the outer plane of the clamping body, and based on the outline of the corresponding outer plane, confirm the midpoint of the inner plane of the corresponding outer plane.

[0053] Arbitrarily select a set of built-in plane midpoints and associated reference points, generate a line connecting the built-in plane midpoints and reference points, extend this line, confirm the extended point on another plane (i.e., the opposite plane), lock the total line distance JL between the built-in plane midpoints and the extended point, and then lock the gripping distance JQ of the robotic gripper.

[0054] If JQ≤JL, then select another set of external planes and confirm the midpoint and extension points of the built-in plane, then confirm the distance, and identify whether the confirmed JQ satisfies JQ>JL. If it satisfies, then the midpoint and extension points of the built-in plane confirmed in the corresponding process are used as clamping points. If it does not satisfy, then generate an error signal for display. When external personnel see the error signal, it means that the current intelligent catering robot cannot clamp the corresponding object to be clamped, and it is necessary to reselect the object to be clamped, re-enter the target command, lock the corresponding clamping point, and perform the gripping process.

[0055] If JQ > JL, then the currently confirmed midpoint of the built-in plane and the extended point will be used as the clamping point;

[0056] Specifically, to achieve optimal processing, the clamping point is calculated using a reference point and external planar features. This, combined with a matching judgment of the gripper's gripping distance (JQ) and the total connecting distance (JL), enables adaptive adjustment of the gripping action. This design ensures the rationality of the clamping point selection, guaranteeing stable gripping of the object, while also proactively identifying ungripable scenarios through distance verification. This avoids invalid operations, reduces equipment wear and tear and the probability of task interruption, and improves the reliability and flexibility of robot operations.

[0057] Example 2

[0058] Combination Figure 2 The intelligent catering robot system based on AI visual recognition includes:

[0059] On the model generation end, based on the environment in which the intelligent catering robot is located and the AI ​​visual recognition equipment, an environmental model of the environment in which it is located is generated, and the generated environmental model is recorded as a model to be determined.

[0060] The object to be clamped generates the object by locking the target model from the cloud database according to the input target instructions, and determining the object to be clamped from the model based on the individual characteristics of the target model.

[0061] The operation plan confirmation end confirms the operation trajectory of the robotic arm based on the confirmed position of the object to be clamped and the robotic arm. Then, it locks out the feasible trajectory from the confirmed set of several operation trajectories and confirms the movement plan associated with each different feasible trajectory, thereby locking out the optimal operation plan.

[0062] The gripping control processing end, based on the confirmed object to be gripped, locks the clamping point on the object according to the planar features associated with the surface of the object, and controls the corresponding mechanical claw of the intelligent catering robot to perform gripping processing.

[0063] Also includes:

[0064] Communication layer: Enables low-latency interconnection between the control robot and catering equipment such as noodle stoves, cooking machines, coffee machines, dishwashers, multi-functional steam ovens, fume purifiers, and fruit and vegetable cleaners;

[0065] Equipment Abstraction Layer: Unifies the encapsulation of robots, heating elements, and sensor driver modules, facilitating rapid integration of new equipment;

[0066] Motion control layer: controls trajectory interpolation, speed / acceleration limiting, and multi-axis synchronous motion;

[0067] Application layer: includes process script parsing and task scheduling, visualization interface, log monitoring and fault alarm.

[0068] The overall system architecture includes:

[0069] The hardware layer includes a camera, multiple actuators, a lifting motor module, a rotary slide, an embedded controller (STM32) with an MQTT communication module, and an SD card point storage module.

[0070] The software layer includes:

[0071] Visual perception framework: OpenCV + TensorFlow / PyTorch (YOLOv5);

[0072] Robot middleware: Self-developed C++ / Python scheduling framework (based on MQTT);

[0073] Motion planning library: DLite (Dynamic Incremental Replanning) performs dynamic incremental planning on pre-generated raster maps;

[0074] Collaborative control terminal: based on self-developed action services and topics.

[0075] The data layer includes camera intrinsic and extrinsic parameters, label calibration results, DH parameters for each axis, velocity / acceleration limits, and raster maps (automatically generated).

[0076] The system uses AI machine vision to complete the process of equipment identification and calibration, including:

[0077] Use YOLOv5 to detect kitchen equipment and QR code / ArUco tags in real time, maintaining ≥30 FPS to complete the equipment detection process;

[0078] The center of the detection box is mapped to the camera coordinate system, thus completing the pixel coordinate mapping process.

[0079] Call cv2.solvePnP(), combine with camera intrinsic parameters to obtain the 6DoF transformation matrix of the label to complete the PnP pose transformation;

[0080] The error filtering operation is completed by performing moving average or statistical filtering after sampling multiple frames and outputting the final captured pose.

[0081] The environmental mapping and localization process includes:

[0082] By employing the DLite algorithm, dynamic incremental replanning is suitable for frequently occurring moving obstacles; when the state changes or obstacles are updated, only the affected area is recalculated locally, improving efficiency; and it interfaces with grid maps to output discrete path sequences.

[0083] The path planning and obstacle avoidance process includes:

[0084] The PlannerManager centrally switches between global DLite planning and ORCA speed correction. Costmap updates are implemented: the end-effector depth camera / laser sensor acquires dynamic obstacles at a 10Hz frequency and marks them on a 2D grid: hard obstacles (cost=∞), soft obstacles (cost=C). high ), Free Zone (cost=1);

[0085] Then, global planning is performed: DLite is run on the above costmap, incrementally recalculating only the affected areas and outputting discrete grid path;

[0086] Then perform ORCA speed correction: In multi-device scenarios, only the speed vector of each device is adjusted without changing the global path topology, ensuring decentralized real-time collision avoidance;

[0087] Then perform path smoothing or feasibility verification: Generate a C² continuous curve using cubic / quintic B-splines, and perform feasibility checks in conjunction with machine dynamics constraints (speed, acceleration, joint limits);

[0088] Execute abnormal rollback process: If planning fails N times in a row, automatically roll back to the last safe point or restart global planning, and trigger alarms or human intervention when necessary.

[0089] The automatic path calculation and obstacle avoidance process for height-adjustable devices includes:

[0090] 1. Sensor data acquisition: A depth camera or laser rangefinder is installed at the end point, focusing only on the measured minimum obstacle height Z(i,j); the depth value corresponding to each map grid (i,j) is updated to the estimated height of that grid.

[0091] 2. Classify grid status: Set a "low obstacle threshold" h low (Requires elevation to pass) and "Impassable threshold" ck;

[0092] The associated signal for each grid cell (i,j) is evaluated:

[0093] If Z(i,j)≥ ck, marked as "barrier" (cost=∞);

[0094] If h low ≤Z(i,j)≤ ck is marked as "passable but requires elevation" (cost=C) high );

[0095] If Z(i,j)<h low It is marked as "free passage" (cost=1).

[0096] 3. Execute the 2D DLite planning process: Run DLite on the above 2D grid (containing only passage costs); when obstacles are added or removed, or when costs change, only update the corresponding grid, triggering local replanning, and output the grid path {(x k ,y k )}.

[0097] 4. Embedded lifting logic: During path traversal, if the next grid point is "traversable but requires lifting"; pause XY motion and execute Z-axis lifting h. low + After lifting is complete, XY path tracking is restored; after passing through the area, you can choose to automatically lower back to the working height.

[0098] 5. Execution path smoothing and speed planning process: Use cubic B-spline interpolation on the DLite discrete path to generate a smooth curve; distribute the speed on the curve according to the trapezoidal speed curve, accelerate → constant speed → decelerate, and reduce mechanical shock.

[0099] 6. Real-time online replanning: During operation, depth sensor data is continuously read and grid cost is updated; if the current path is blocked by a new obstacle, DLite automatically performs local replanning and smoothly connects.

[0100] Some of the data in the above formulas are numerical calculations with dimensions removed, and the contents not described in detail in this specification are all prior art known to those skilled in the art.

[0101] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. A motion method for an intelligent catering robot based on AI visual recognition, characterized in that, Includes the following steps: Step 1: Based on the environment of the intelligent catering robot and the AI ​​visual recognition equipment, generate an environmental model of the environment and record the generated environmental model as a pending model. Step 2: Based on the input target instructions, locate the target body model from the cloud database, and determine the body to be clamped from the undetermined model based on the individual characteristics of the target body model; Step 3: Based on the confirmed location of the object to be gripped and the robotic arm, determine the robotic arm's trajectory, and then select a feasible trajectory from the confirmed trajectories. The specific method is as follows: The position of the robotic arm in space is confirmed as the starting point, and the position of the object to be gripped in space is confirmed as the ending point. Starting from the starting point, the robotic arm moves towards the ending point, generating several sets of corresponding robotic arm running trajectories. Based on the generated sets of running trajectories, it is confirmed whether the corresponding running trajectory interferes with other individual models in the undetermined model. If there is interference, the corresponding running trajectory is removed. If there is no interference, the corresponding running trajectory is marked as a feasible trajectory. Then, the travel plans associated with each different feasible trajectory are confirmed to determine the optimal operating plan. The specific method is as follows: Place the feasible trajectory in a three-dimensional spatial coordinate system. From the three-dimensional spatial coordinate system, confirm the plane vector associated with the feasible trajectory in a single plane. Based on the starting point and ending point of the feasible trajectory, directly proceed from the starting point to the ending point in the two-dimensional plane of the corresponding three-dimensional spatial coordinate system. Confirm the plane vector associated with the corresponding two-dimensional plane. Then, based on the running nodes associated with the corresponding plane, record the multiple sets of plane vectors associated with the feasible trajectory as the running vectors of the corresponding running nodes. Based on the different parallel vectors recorded by different running nodes, confirm the length data L associated with the corresponding parallel vector. i Where i represents different parallel vectors, and then the running speed V of the corresponding running node is determined. i V i All values ​​are preset values, using: L i ÷V i =T i From different T associated with different running nodes i In the middle, select T i max is the trajectory feature of the current feasible trajectory; Then, sequentially, the T associated with different feasible trajectories... i Max confirms sequentially, from the confirmed T... i In the max, the minimum value is selected, and the feasible trajectory associated with the minimum value is recorded as the optimal running plan. The intelligent catering robot is then controlled to run according to the confirmed optimal running plan. Step 4: Based on the confirmed object to be clamped, lock the clamping point on the object according to the planar features associated with the surface of the object, and control the corresponding mechanical claw of the intelligent catering robot to perform the grasping process.

2. The motion method for an intelligent catering robot based on AI visual recognition according to claim 1, characterized in that, In step one, the specific method for generating the environment model is as follows: Based on the set AI visual recognition equipment, the environmental images of the surrounding environment of the intelligent catering robot are acquired, and at the same time, the sound wave detection equipment is used to confirm the sound wave characteristics associated with the surrounding environment. Based on the acquired environmental images and sound wave characteristics, an environmental model associated with the current intelligent catering robot is generated.

3. The motion method for an intelligent catering robot based on AI visual recognition according to claim 1, characterized in that, In step two, the specific method for determining the clamping body is as follows: Based on the confirmed target instructions, the target body model associated with the target instructions is confirmed in the cloud database; The built-in midpoint is identified from the target body model. The built-in midpoint has been preset in the corresponding target body model. Then, the farthest point from the built-in midpoint is identified from the target body model. The built-in midpoint and the farthest point are connected to generate a verification benchmark belonging to the target body model. Next, feature verification is performed on different individual models within the target model: the individual model is placed in a three-dimensional coordinate system, and the three-dimensional coordinates associated with multiple plane points in the corresponding plane are confirmed and averaged. The average coordinates are then confirmed, and the average points associated with the average coordinates are marked within the individual model as the center point associated with the individual model. From the plane of the individual model, the plane point that is farthest from the center point in a straight line is identified and connected to confirm the individual model reference. The individual model reference corresponding to the individual model is compared and verified with the verification reference of the target model to identify whether the individual model completely overlaps with the target model. The individual model that completely overlaps is recorded as the object to be clamped.

4. The motion method for an intelligent catering robot based on AI visual recognition according to claim 3, characterized in that, If a single model does not completely overlap with the target model, the process of confirming the other single models to be sandwiched continues.

5. The motion method for an intelligent catering robot based on AI visual recognition according to claim 1, characterized in that, In step four, the specific method for locking the clamping point is as follows: Record the center point marked inside the clamping body as the reference point, then confirm the outer plane of the clamping body, and based on the outline of the corresponding outer plane, confirm the midpoint of the inner plane of the corresponding outer plane. Arbitrarily select a set of built-in plane midpoints and associated reference points, generate a line connecting the built-in plane midpoints and reference points, extend this line, confirm the extended point on another plane, lock the total line distance JL between the built-in plane midpoints and the extended point, and then lock the gripping distance JQ of the robotic gripper. If JQ≤JL, select another set of external planes and confirm the midpoint and extension points of the built-in plane, then confirm the distance and identify whether the confirmed JQ satisfies JQ>JL. If it satisfies, use the midpoint and extension points of the built-in plane confirmed in the corresponding process as clamping points. If it does not satisfy, generate an error signal for display.

6. The motion method for an intelligent catering robot based on AI visual recognition according to claim 5, characterized in that, If JQ > JL, then the currently confirmed midpoint of the built-in plane and the extended point will be used as the clamping point.

7. An intelligent catering robot system based on AI visual recognition, wherein the system operates according to the action method of the intelligent catering robot based on AI visual recognition as described in any one of claims 1-6, characterized in that, include: On the model generation end, based on the environment in which the intelligent catering robot is located and the AI ​​visual recognition equipment, an environmental model of the environment in which it is located is generated, and the generated environmental model is recorded as a model to be determined. The object to be clamped generates the object by locking the target model from the cloud database according to the input target instructions, and determining the object to be clamped from the model based on the individual characteristics of the target model. The operation plan confirmation end confirms the operation trajectory of the robotic arm based on the confirmed position of the object to be clamped and the robotic arm. Then, it locks out the feasible trajectory from the confirmed set of several operation trajectories and confirms the movement plan associated with each different feasible trajectory, thereby locking out the optimal operation plan. The gripping control processing end, based on the confirmed object to be gripped, locks the clamping point on the object according to the planar features associated with the surface of the object, and controls the corresponding mechanical claw of the intelligent catering robot to perform gripping processing.

Images