Method and system for controlling robot through cloud

Abstract

The invention provides a cloud control robot method and system, and relates to an intelligent control technology, a robot terminal obtains environment data and object state information based on a sensor module, and uploads the environment data and the object state information to a cloud server based on a communication module, and the cloud server determines an object specification according to the object state information and sends the object specification to a cloud server. The cloud server determines a constraint road section according to the environmental data, calculates a track offset value according to the object specification and the road section parameters, generates a control instruction according to the grabbing point and the track offset value, sends the control instruction to the robot terminal, and obtains real-time state data when the robot terminal executes the control instruction; and the cloud server generates action adjustment information based on the real-time state data to perform instruction updating on the robot terminal, so that more efficient and more flexible object carrying control on the robot can be realized.

Description

Technical Field

[0001] The present invention relates to intelligent control technology, and in particular, to a method and system for controlling a robot in the cloud. Background Art

[0002] In today's diverse and complex working scenarios, handling operations by robots are widely applied in multiple fields such as industrial production and logistics warehousing, and these scenarios have extremely high requirements for the efficiency and flexibility of robot handling.

[0003] Currently, in the prior art, when controlling a robot to carry an object, it mainly relies on the mode of the robot terminal executing a preset program. In this mode, technicians usually write key information such as detailed handling processes, for example, precise handling paths, accurate grasping positions, and accurate placing positions, into the local control system of the robot terminal through programming. The robot terminal performs corresponding object handling operations according to the program written in the local control system. However, this traditional preset program control method has many drawbacks that cannot be ignored. Among them, the biggest problem is the serious lack of flexibility. Once the position and layout of the objects in the working environment change, or objects of different specifications need to be carried, it is necessary to rewrite and debug the program again. This process not only takes a long time but also requires a large amount of manpower for repeated debugging, resulting in low handling efficiency of the robot.

[0004] Therefore, how to achieve more efficient and flexible object handling control for the robot has become an urgent problem to be solved. Summary of the Invention

[0005] The present invention provides a method and system for controlling a robot in the cloud, which can achieve more efficient and flexible object handling control for the robot.

[0006] In the first aspect of the present invention, a method for controlling a robot in the cloud is provided, which is characterized by including: The robot terminal obtains environmental data and object state information based on the sensor module, and uploads the environmental data and object state information to the cloud server based on the communication module; The cloud server determines the object specifications according to the object state information and calculates the grasping points according to the object specifications; The cloud server determines the restricted sections according to the environmental data, calculates the trajectory offset value according to the object specifications and section parameters, and generates a control instruction according to the grasping points and the trajectory offset value and sends it to the robot terminal; Obtain the real-time state data of the robot terminal when executing the control instruction, and the cloud server generates action adjustment information based on the real-time state data to update the instruction for the robot terminal.

[0007] Optionally, in a possible implementation of the first aspect, the cloud server determines the object specification according to the object state information and calculates the grasping point according to the object specification, including: The cloud server determines the object specification according to the image data of each perspective, and the object state information includes the image data of each perspective; Obtain the maximum arm span length of the robot terminal, determine the semi-perimeter according to the object specification, and when the maximum arm span length is greater than the semi-perimeter, determine the center point of the opposite object side as the grasping point; When the maximum arm span length is less than the semi-perimeter, multi-robot collaboration is triggered, the handling surfaces corresponding to each robot terminal are determined, and the points at the grasping offset distance from the center point in the handling surface are determined as the grasping points corresponding to each robot terminal.

[0008] Optionally, in a possible implementation of the first aspect, when the maximum arm span length is less than the semi-perimeter, multi-robot collaboration is triggered, the handling surfaces corresponding to each robot terminal are determined, and the points at the grasping offset distance from the center point in the handling surface are determined as the grasping points corresponding to each robot terminal, including: Determine the semi-arm span length of the robot terminal, and obtain the grasping range corresponding to each object side for the semi-arm span length; When there is a grasping range smaller than the range threshold of the handling surface, determine the number of terminals for multi-robot collaboration as the preset number of four corners, and sequentially determine the adjacent object sides as the handling surfaces corresponding to the respective robot terminals; When there is no grasping range smaller than the range threshold of the handling surface, determine the number of terminals for multi-robot collaboration as the preset number of diagonals, and determine the diagonally adjacent object sides as the handling surfaces corresponding to the respective robot terminals; Determine the points at the grasping offset distance from the center point in the handling surface as the grasping points corresponding to each robot terminal.

[0009] Optionally, in a possible implementation of the first aspect, the grasping offset distance is determined through the following steps, including: Calculate the object volume according to the object specification, and obtain the offset coefficient based on the ratio of the reference volume to the object volume; Perform weighted processing on the product of the offset coefficient and the reference offset distance to obtain the grasping offset distance.

[0010] Optionally, in a possible implementation of the first aspect, the cloud server identifies the passable area and the non-passable area according to the pixel classification result in the environmental data, and the environmental data includes the image data corresponding to the front view; Obtain the path center line of the passable area, and when the angle between the path center line and the current driving direction of the robot terminal is greater than the threshold, retrieve the regional map; Determine the positioning point of the robot terminal in the area map, and determine the road section where the positioning point is located and the robot terminal has not passed through as the constraint road section; Obtain the turning angle of the constraint road section, and calculate the turning speed of the robot terminal from the current driving direction to the direction corresponding to the turning angle according to the item specifications. The road section parameters include the turning angle, and the trajectory offset value includes the turning speed.

[0011] Optionally, in a possible implementation manner of the first aspect, obtaining the turning angle of the constraint road section includes: Obtain the action trajectory of the robot terminal in the area map within a preset time period, and determine the tangent direction of the action trajectory end point as the driving direction of the robot terminal; Determine the center line of the constraint road section. When the tangent directions of all path points on the center line are the same, determine the included angle between the center line and the driving direction as the turning angle, and determine the direction from the driving direction to the center line as the turning direction; When there are path points on the center line with inconsistent tangent directions, determine the path point on the center line closest to the positioning point of the robot terminal as the starting point; Taking the starting point as the traversal starting point, obtain the path sub-section composed of path points with the same tangent direction as the starting point, determine the included angle between the path sub-section and the driving direction as the turning angle, and determine the direction from the driving direction to the path sub-section as the turning direction.

[0012] Optionally, in a possible implementation manner of the first aspect, calculating the turning speed of the robot terminal from the current driving direction to the direction corresponding to the turning angle according to the item specifications includes: Calculate the object volume based on the item specifications, and perform weighted processing on the reciprocal of the object volume to obtain the first adjustment coefficient; Obtain the item attribute. When the item attribute is a vulnerable attribute, obtain the preset coefficient corresponding to the risk level of the target item as the second adjustment coefficient; Perform comprehensive calculation on the first adjustment coefficient and the second adjustment coefficient to obtain the total adjustment coefficient, and obtain the turning speed according to the product of the total adjustment coefficient and the reference speed.

[0013] Optionally, in a possible implementation manner of the first aspect, after the cloud server determines the constraint road section according to the environmental data, it further includes: Obtain the item width corresponding to the target item in the direction perpendicular to the current driving direction, and obtain the road section width of the constraint road section; When the item width is greater than or equal to the road section width, generate a reference model corresponding to the item specifications, adjust the reference model according to the preset rotation angle, and obtain the reference width corresponding to the reference model; Determine the rotation angle whose reference width is less than the road section width as the target angle, and send the target angle to the robot terminal for angle adjustment of the item.

[0014] Optionally, in a possible implementation manner of the first aspect, obtain the real-time status data of the robot terminal when executing the control instruction, and the cloud server generates action adjustment information based on the real-time status data to update the instruction of the robot terminal, including: Obtain the vibration value based on the sensor module of the robot terminal, and the real-time status data includes the vibration value; When the vibration value is greater than the vibration threshold, obtain the vibration difference between the vibration value and the reference vibration value, and calculate the weighted value of the ratio of the vibration difference to the reference difference to obtain the speed adjustment value; Obtain the action adjustment information by subtracting the speed adjustment value from the current speed of the robot terminal, and update the instruction of the speed of the robot terminal based on the action adjustment information.

[0015] In the second aspect of the present invention, a system for cloud controlling a robot is provided, including: An upload module, configured to enable the robot terminal to obtain environmental data and object state information based on the sensor module, and upload the environmental data and object state information to the cloud server based on the communication module; A determination module, configured to enable the cloud server to determine the object specification according to the object state information, and calculate the grasping point according to the object specification; A calculation module, configured to enable the cloud server to determine the constraint road section according to the environmental data, calculate the trajectory offset value according to the object specification and road section parameters, and generate a control instruction according to the grasping point and the trajectory offset value and send it to the robot terminal; A generation module, configured to obtain the real-time status data of the robot terminal when executing the control instruction, and the cloud server generates action adjustment information based on the real-time status data to update the instruction of the robot terminal.

[0016] The beneficial effects of the present invention are as follows: 1. The present invention can centrally process and analyze the environmental data and object state information uploaded by the robot terminal through the cloud server, and issue precise control instructions based on the analysis results, so as to realize flexible and efficient handling control of the robot. Through the efficient cooperation between the cloud server and the robot terminal, the robot terminal can achieve more accurate and flexible object handling operations.

[0017] 2. The present invention can transmit environmental data and object status information to the cloud server through the communication module, ensuring that the environmental data and object status information can be uploaded from the robot terminal to the cloud server in real time and reliably, enabling the cloud server to obtain the latest status of the environment and objects around the robot in a timely manner. Moreover, the cloud server can determine the object specifications based on the object status information containing multi-view image data, and calculate the optimal grasping point in combination with the maximum arm reach length of the robot terminal, which can ensure the stability and balance of the robot terminal when grasping objects, and avoid tilting or dropping of the objects during handling.

[0018] 3. In the present invention, the cloud server can comprehensively and meticulously analyze the surrounding environment of the robot terminal based on the environmental data uploaded by the robot terminal in real time, accurately determine the restricted sections. By accurately identifying the restricted sections, the robot terminal can make preparations in advance to avoid collisions caused by failure to detect the restricted sections in a timely manner, improving the safety of the robot terminal during the process of handling objects. After determining the restricted sections, the cloud server can calculate the trajectory offset value by comprehensively considering the object specifications and section parameters, thereby effectively reducing the risk of the object falling or being damaged during handling, and at the same time improving the passing efficiency of the robot terminal in the restricted sections.

[0019] 4. In the present invention, during the process of the robot terminal executing control instructions to handle objects, its sensor module continuously and real-time collects the real-time status data during handling, and can continuously send its own real-time status data to the cloud server. The cloud server can analyze the received real-time status data, especially analyze the vibration data. When abnormal vibration data is detected, the cloud server can generate corresponding action adjustment information, and update the corresponding instructions for the speed of the robot terminal according to the action adjustment information, thereby improving the stability of the robot terminal during object handling. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a schematic flowchart of a method for controlling a robot by the cloud according to an embodiment of the present invention; Figure 2 is a schematic diagram for determining the handling surface corresponding to the robot terminal according to an embodiment of the present invention; Figure 3 is a schematic diagram for determining the turning direction according to an embodiment of the present invention; Figure 4 is a schematic structural diagram of a system for controlling a robot by the cloud according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0022] The technical solutions of the present invention will be described in detail below with specific embodiments. These specific embodiments may be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0023] Refer to Figure 1 , which is a schematic diagram of a method for controlling a robot by the cloud provided by an embodiment of the present invention. Figure 1 The execution subject of the method shown may be a software and / or hardware device. The execution subject of the present application may include, but is not limited to, at least one of the following: user equipment, network equipment, etc. Among them, the user equipment may include, but is not limited to, a computer, a smart phone, a personal digital assistant (Personal Digital Assistant, abbreviated as: PDA), and the above-mentioned electronic devices, etc. The network equipment may include, but is not limited to, a single network server, a server group composed of multiple network servers, or a cloud composed of a large number of computers or network servers based on cloud computing. Among them, cloud computing is a type of distributed computing, which is composed of a group of loosely coupled computers to form a super virtual computer. This embodiment does not make any restrictions. It includes steps S1 to S4, specifically as follows: S1, the robot terminal obtains environmental data and object state information based on the sensor module, and uploads the environmental data and object state information to the cloud server based on the communication module.

[0024] Among them, the robot terminal refers to the robot device that executes specific tasks. It integrates a sensor module and a communication module and can perform object handling operations autonomously or remotely controlled. The sensor module refers to the components on the robot terminal used to sense the external environment, including but not limited to visual sensors. The environmental data refers to the spatial information of the robot's surrounding environment. The object status information refers to the omnidirectional image information of the object to be handled obtained by taking pictures of the object to be handled from multiple different perspectives through the visual sensor in the sensor module. The communication module refers to the module that connects the robot terminal to the cloud server. Through this module, the robot terminal can upload the environmental data and object status information collected by the sensor module to the cloud server in real time and reliably, and at the same time receive the control instructions issued by the cloud server. The cloud server refers to the central platform that can centrally process the data uploaded by the robot terminal, has powerful computing and storage capabilities, can use advanced technologies such as big data analysis and machine learning to deeply analyze the received data, generate accurate control instructions, and issue them to the robot terminal through the communication network.

[0025] In practical applications, robots have been deeply integrated into many key fields such as industrial production, logistics distribution, medical services, and home care for the elderly. For example, in large logistics warehouses, a large number of goods are flowing continuously, and a large number of robots are required to work together. They not only have to accurately and efficiently complete core tasks such as goods handling and sorting in a complex environment where the layout of goods stacking changes constantly, but also have to skillfully avoid interference from other operating equipment such as forklifts to ensure the smooth progress of the entire logistics process. The traditional robot control method is mainly pre-programmed control. In relatively fixed scenarios with less environmental changes, such as the production line handling link of specific components, the robot can grasp objects from the specified position according to the pre-programmed procedure, move along the established path and place them at the target location. However, once the position and layout of the objects in the working environment change, or objects of different specifications and shapes need to be handled, the program must be rewritten and debugged again, which is time-consuming and laborious. For example, in a logistics warehouse, there are a wide variety of goods and the stacking positions change constantly, and it is difficult for robots relying on pre-programmed procedures to adapt. This solution can centrally process and analyze the data uploaded by the robot terminal through the cloud server, and issue accurate control instructions based on the analysis results, so as to achieve flexible and efficient handling control of the robot. Through the efficient cooperation between the cloud server and the robot terminal, the robot can achieve more accurate and flexible object handling operations, and at the same time have stronger environmental adaptability and task execution efficiency.

[0026] Specifically, when the robot is performing an object handling task, the robot terminal can collect real-time environmental data through its equipped sensor module, which can be, for example, a vision sensor. The environmental data refers to the spatial information of the robot's surrounding environment, including the position, shape, and distance of obstacles, as well as the distribution of passable areas. For example, during the handling process, the robot can scan the surrounding environment through the vision sensor to obtain information such as whether there are obstacles ahead, whether the passage is clear, and the position of turning points. These environmental data are crucial for the robot to plan the handling path, avoid obstacles, and dynamically adjust the motion trajectory. By collecting environmental data in real time, it is possible to accurately determine whether the robot needs to turn or adjust its speed, etc., thereby improving the safety and efficiency of the handling task. Moreover, the sensor module can take pictures of the object to be handled from multiple different perspectives. By taking pictures of the object from various angles, it is possible to obtain comprehensive image information of the object to be handled, that is, object state information. These image information cover all sides, the top, and the bottom of the object, etc. After obtaining the environmental data and object state information, the robot terminal can upload these data to the cloud server through its communication module. The communication module usually supports high-speed and low-latency data transmission protocols to ensure that the environmental data and object state information can be transmitted to the cloud in real time and reliably. The environmental data and object state information can provide a sufficient information basis for the cloud server for subsequent data analysis.

[0027] Through the above implementation, it can be ensured that the environmental data and object state information can be transmitted to the cloud server in real time and reliably.

[0028] S2. The cloud server determines the object specifications based on the object state information and calculates the grasping points according to the object specifications.

[0029] After receiving the environmental data and object state information uploaded by the robot terminal, the cloud server can perform corresponding analysis on the received data. Specifically, by analyzing the object state information, it is possible to determine data such as the length, width, and height of the object to be handled, and then determine the object specifications. After obtaining the object specifications, it is possible to calculate the specific position where the end effector of the robot, such as the gripper of the robotic arm, should contact and grasp the object when performing the grasping action, that is, the grasping point. Among them, the object specifications refer to the dimensions such as the length, width, and height of the object to be handled, and the grasping point refers to the specific position where the end effector of the robot, such as the gripper of the robotic arm, should contact and grasp the object when performing the grasping action. The selection of this position is crucial for successfully grasping and stably handling the object.

[0030] In some embodiments, step S2 includes S21 to S23, specifically as follows: S21. The cloud server determines the object specifications based on the image data from each perspective, and the object state information includes the image data from each perspective.

[0031] Specifically, the vision sensor of the robot terminal can capture an object from multiple angles to obtain image data of the object from various perspectives. The cloud server can perform in-depth processing on this image data through advanced image analysis algorithms to determine the actual length, width, and height values of the object, that is, the object specifications.

[0032] S22. Obtain the maximum arm reach length of the robot terminal, determine the semi-perimeter based on the object specifications. When the maximum arm reach length is greater than the semi-perimeter, determine the center point of the opposite object side as the grasping point.

[0033] Specifically, after determining the object specifications, the cloud server can obtain the key parameter of the maximum arm reach length corresponding to its robotic arm from the configuration parameters of the robot terminal. The maximum arm reach length determines the maximum range that the robot can reach when grasping an object. Then, the semi-perimeter of the object can be calculated based on the object specifications. For an object with a regular shape, such as a cuboid, the semi-perimeter can be obtained by the sum of the length and width. When the maximum arm reach length of the robot terminal is greater than the semi-perimeter of the object, it means that the robot can easily surround one side of the object for grasping. At this time, the cloud server can determine the center points of the two opposite object sides as the grasping points. For example, when the maximum arm reach length of the robot terminal is greater than the semi-perimeter and the object is a regular cuboid, for the four sides of the cuboid, when looking at the cuboid object from the front view, the four sides are the left side, the right side, the front side, and the back side respectively. Among them, the left side and the right side are two opposite object sides, and the front side and the back side are two opposite object sides. Therefore, when the robot terminal grasps the object, the center points of the left side and the right side can be determined as the grasping points, or the center points of the front side and the back side can be determined as the grasping points. Selecting the center points of the opposite object sides as the grasping points is because this point can enable the robot to better balance the weight distribution of the object when grasping the object and reduce the risk of the object tilting or falling during the handling process. Among them, the maximum arm reach length refers to the total length that the two robotic arms of the robot terminal can reach when fully extended.

[0034] S23. Trigger multi-robot collaboration when the maximum arm reach length is less than the semi-perimeter, determine the respective handling surfaces of multiple robot terminals, and determine the points at the grasping offset distance from the center point in the handling surface as the grasping points corresponding to each robot terminal.

[0035] Specifically, when the maximum arm span length is less than half of the perimeter, it indicates that a single robot cannot independently and effectively grasp an object. At this time, the cloud server triggers the multi-robot collaboration mode, schedules multiple robot terminals to cooperate to complete the handling task, and the cloud server can allocate corresponding handling surfaces to each participating robot terminal according to the specifications and shape of the object. For example, for a cuboid object, four of its six sides can be assigned to four robot terminals, and each robot is responsible for a corresponding side. For each handling surface, the cloud server can calculate its center point and determine a grasping offset distance according to the specifications of the object. The grasping offset distance is to ensure that the robot terminal can keep the object balanced during grasping and avoid collisions with other robot terminals. The point at a distance equal to the grasping offset distance from the center point of each handling surface is determined as the grasping point corresponding to each robot terminal. Among them, multi-robot collaboration means that multiple robot terminals cooperate to complete the handling of an object, the handling surface refers to the side of the object that each robot terminal is responsible for during the process of multiple robot terminals cooperating to complete the handling of the object, and the grasping offset distance refers to the distance offset from the center point of the handling surface when determining the grasping point.

[0036] Based on the above embodiments, the specific implementation manner of step S23 can be as follows: S231, determine the half-arm span length of the robot terminal and obtain the grasping range corresponding to the half-arm span length on each side of the object.

[0037] In the multi-robot collaborative handling task, understanding the grasping ability of the robot terminal is the basis. The half-arm span length is the length of a single robotic arm, which directly limits the range that the robot can reach on the side of the object. Only by first clarifying the half-arm span length can we further evaluate the grasping feasibility of the robot for each side of the object, providing a basis for subsequent judgment on whether multi-robot collaboration is needed and how to allocate tasks.

[0038] Specifically, the cloud server can obtain the length data of a single robotic arm from the device information database of the robot. This data is usually accurately recorded during the production and configuration of the robot and will be uploaded to the cloud server for unified management. After obtaining the semi-extended arm length, it can combine the three-dimensional spatial information of the object and the position and posture of the robot around the object, and determine the grasping range corresponding to the semi-extended arm length on each side of the object through geometric calculations and spatial modeling. For example, taking the shoulder joint of the robot as the origin and drawing a circle with the semi-extended arm length as the radius, the area where the circle intersects with the side of the object is the possible grasping range. By determining the semi-extended arm length and the corresponding grasping range, the cloud server can clearly understand the ability boundary of each robot in the current object handling task, which helps to predict in advance whether a single robot can complete the grasping of a certain side of the object, provides accurate data support for subsequent multi-robot cooperation decisions, avoids the situation where the robot cannot reach the object during the handling process, and improves the planning efficiency and accuracy of the handling task.

[0039] Among them, the semi-extended arm length refers to the length corresponding to a single robotic arm at the end of the robot, and the grasping range refers to the range that the robotic arm at the end of the robot can cover on each side of the object.

[0040] S232. When there is a grasping range smaller than the range threshold of the handling surface, determine that the number of robot terminals for multi-robot cooperation is the preset number at the four corners, and sequentially determine the adjacent object sides as the handling surfaces corresponding to the respective robot terminals.

[0041] Specifically, the cloud server can compare the grasping range of each handling surface with the preset range threshold corresponding to the handling surface. The setting of the range threshold is based on a large number of experiments and practical application experiences. When it is found that there is a grasping range smaller than the range threshold of the handling surface, when the grasping range of a certain handling surface is smaller than the preset range threshold, it means that a single robot cannot effectively cover this side for grasping operations, which may lead to unstable grasping or inability to complete the grasping task. At this time, robot terminals with a preset number at the four corners, usually 4 for example, can be used for cooperation. The cloud server will automatically determine that 4 robots are needed for cooperation. Then, in a clockwise or counterclockwise order, the adjacent object sides can be sequentially assigned to the 4 robot terminals, which can provide stable support and grasping force from the four corners of the object to ensure the balance and stability of the object during the handling process. The reason for choosing to assign adjacent object sides to the robots is that the layout of the adjacent sides makes the cooperation between the robots closer and more efficient, and can better coordinate actions to jointly complete the handling task. For example, see Figure 2 , which is a schematic diagram for determining the handling surface corresponding to the robot terminal provided by the embodiment of the present invention. When the number of robot terminals for multi-robot cooperation is 4, as Figure 2As shown in the figure, for the four sides of a regular object, they are respectively object side 1, object side 2, object side 3 opposite to object side 1, and object side 4 opposite to object side 2. Among them, object side 1 and object side 2 are adjacent object sides, object side 2 and object side 3 are adjacent object sides, object side 3 and object side 4 are adjacent object sides, and object side 4 and object side 1 are adjacent object sides. Then, object side 1 and object side 2 can be determined as the handling surfaces corresponding to robot terminal 1, object side 2 and object side 3 can be determined as the handling surfaces corresponding to robot terminal 2, object side 3 and object side 4 can be determined as the handling surfaces corresponding to robot terminal 3, and object side 4 and object side 1 can be determined as the handling surfaces corresponding to robot terminal 4.

[0042] Among them, the range threshold refers to a preset threshold used to measure whether the robot terminal can effectively cover the object side and perform a grasping operation. The number of terminals refers to the number of robot terminals during multi-robot cooperation, and the preset number of four corners is 4.

[0043] S233, when there is no grasping range smaller than the range threshold of the handling surface, determine the number of terminals for multi-robot cooperation as the preset number of diagonals, and determine the object sides that are diagonal and adjacent as the handling surfaces corresponding to the respective robot terminals.

[0044] Specifically, after the cloud server completes the comparison of the grasping ranges of all object sides with the range threshold, if it is found that there is no situation where the grasping range is smaller than the threshold, it means that a single robot has sufficient ability to cover each side for grasping. At this time, the robots with the preset number of diagonals can be used for cooperation. For example, it can be determined to use 2 robots for cooperation. Then, the object sides that are diagonal and adjacent can be selected and determined as the handling surfaces corresponding to the respective robot terminals. This can reduce the number of robots used and improve resource utilization efficiency while ensuring the handling effect. The reason for selecting the object sides that are diagonal and adjacent and allocating them to the robots is that such a layout can enable the two robots to form a stable diagonal pulling force, better control the balance and movement direction of the object, and at the same time avoid mutual collision and interference between the robots. For example, when the number of terminals for multi-robot cooperation is 2, for the four sides of a regular object, they are respectively object side 1, object side 2, object side 3, and object side 4. Among them, object side 1 and object side 2 are adjacent object sides, object side 3 and object side 4 are adjacent object sides, and the angle formed by object side 1 and object side 2 is diagonal to the angle formed by object side 1 and object side 1. Therefore, object side 1 and object side 2 can be determined as the handling surfaces corresponding to robot terminal 1, and object side 3 and object side 4 can be determined as the handling surfaces corresponding to robot terminal 2. Among them, the preset number of diagonals is 2.

[0045] S234. Determine the point at the grasping offset distance from the center point on the handling surface as the grasping point corresponding to each robot terminal.

[0046] Specifically, after determining the handling surface corresponding to each robot terminal, on the handling surface, the offset direction corresponding to each robot terminal can be determined. On the offset direction corresponding to each robot terminal, the point at the grasping offset distance from the center point is determined as the grasping point corresponding to the corresponding robot terminal. When determining the offset direction corresponding to each robot terminal, it can be determined according to the handling surface corresponding to each robot terminal. For example, when the handling surfaces corresponding to robot terminal 1 are object side surface 1 and object side surface 2 respectively, there is a common edge between object side surface 1 and object side surface 2. The midpoint of this common edge can be obtained, and the direction from the center point of object side surface 1 to the midpoint of the common edge can be obtained. This direction can be determined as the offset direction corresponding to the robot terminal on object side surface 1. The direction from the center point of object side surface 2 to the midpoint of the common edge can be obtained, and this direction can be determined as the offset direction corresponding to object side surface 2.

[0047] In some embodiments, the grasping offset distance can be preset. In some other embodiments, the grasping offset distance can be determined through the following steps: S2341. Calculate the volume of the object according to the object specifications, and obtain the offset coefficient based on the ratio of the reference volume and the object volume.

[0048] Specifically, according to the object specifications, the volume corresponding to the object, that is, the object volume, can be calculated. For example, for a regular object, after obtaining the length, width, and height of the object, by multiplying the length, width, and height, the object volume corresponding to the object can be obtained. By calculating the ratio of the reference volume and the object volume, the corresponding offset coefficient can be obtained.

[0049] Among them, the object volume refers to the volume calculated according to the object specifications, the reference volume refers to a preset reference volume value used for comparison with the object volume, and the offset coefficient refers to the ratio between the reference volume and the object volume, which can be used to represent the degree of deviation of the object volume relative to the reference volume.

[0050] S2342. Perform weighted processing on the product of the offset coefficient and the reference offset distance to obtain the grasping offset distance.

[0051] Specifically, after multiplying the offset coefficient by the preset reference offset distance, perform weighted processing on the obtained value to obtain the corresponding grasping offset distance. Among them, the reference offset distance refers to the offset distance under preset standard conditions.

[0052] Through the above embodiments, the stability and balance of the robot terminal during object grasping can be ensured, and the object is prevented from tilting or falling during transportation.

[0053] S3. The cloud server determines the constrained section according to the environmental data, calculates the trajectory offset value according to the object specifications and section parameters, and generates a control instruction based on the grasping point and the trajectory offset value and sends it to the robot terminal.

[0054] Specifically, after obtaining the object specifications, during the process of the robot terminal transporting the object, the cloud server can comprehensively and meticulously analyze the surrounding environment based on the environmental data real-time uploaded by the robot terminal, and accurately determine the constrained section. The constrained section covers various areas that require the robot to be cautious, such as sections with obstacles ahead or sections that need to turn. The turning angles of these sections will affect the movement of the robot. When it is determined that there is a constrained section, that is, when turning or there are obstacles ahead, the cloud server can start the trajectory offset planning. In this process, the object specifications are a key consideration factor. Different specifications of objects may be related to the flexibility and stability of the robot during transportation. For example, larger objects require a larger turning angle when turning to avoid colliding with surrounding obstacles, and during turning, a smoother operation is required to prevent the object from falling or the robot from losing balance due to inertia. At the same time, the section parameters also play an important role in the trajectory offset planning. Here, the section parameters mainly refer to the turning angles at the corners. Different turning angles require the robot to perform turning operations at different speeds and angles. The cloud server can integrate the object specifications and section parameters and use advanced path planning algorithms to perform complex and accurate calculations to calculate the trajectory offset value. The trajectory offset value details the key information such as the direction and angle by which the robot needs to offset from the original movement trajectory when facing the constrained section, so as to plan a safe and efficient transportation path.

[0055] After obtaining the trajectory offset value, the cloud server integrates it with the previously calculated grasping point information. The grasping point determines how the robot stably grasps the object, while the trajectory offset value determines the movement path of the robot during transportation. By combining these two, the cloud server can generate a set of detailed and accurate control instructions, and the cloud server can quickly and accurately send these control instructions to the robot terminal through the communication module to guide the robot to efficiently and safely complete the object transportation task in a complex working environment.

[0056] Among them, the constrained section refers to the section that the robot needs to treat with caution during the process of carrying an object, due to the existence of obstacles or the need to turn. The section parameter refers to the turning angle at the corner. The trajectory offset value refers to the direction and angle by which the robot needs to offset from the original travel trajectory when facing the constrained section. The control instruction refers to a set of detailed and precise instructions generated by the cloud server according to the grasping point information and the trajectory offset value to guide the robot's operation.

[0057] In some embodiments, the step of "the cloud server determines the constrained section according to the environmental data and calculates the trajectory offset value according to the object specification and the section parameter" in step S3 includes the following steps: S31, the cloud server identifies the passable area and the impassable area according to the pixel classification result in the environmental data, and the environmental data includes the image data corresponding to the front view.

[0058] Specifically, when the sensor module of the robot terminal collects environmental data, it can collect the image data of the front view. The cloud server can perform corresponding preprocessing on the front view image data in the uploaded environmental data. For example, it can use a filtering algorithm to remove the noise interference in the image. After the preprocessing is completed, the preprocessed image data can be input into a pre-trained semantic segmentation model for inference. This model is trained based on a large amount of image data, which covers various different working environment scenarios, including warehouses with different layouts, various types of obstacles, and complex passage conditions, etc. Through deep learning algorithms, the model has learned the characteristic patterns of different semantic categories in the image. When the input image enters the model, the model will analyze and judge each pixel in the image, and output the classification result of each pixel, that is, clarify which semantic category each pixel belongs to. For example, the model can identify that the pixel belongs to categories such as the ground, the passage, the obstacle, or the wall.

[0059] According to the pixel classification result output by the semantic segmentation model, the cloud server can further mark the image area. For the areas that can allow the robot to pass safely, such as the ground and the passage, it is marked as the passable area. These areas provide a safe path for the robot to travel, and the robot can move freely within these areas to perform the carrying task. For the areas that will block the robot's progress, such as obstacles and walls, the cloud server marks them as the impassable area. These areas clarify the range that the robot needs to avoid during the travel process, preventing the robot from colliding with obstacles and ensuring the safe progress of the carrying operation. Through this precise area marking, the cloud server provides a clear and accurate environmental information basis for subsequent determination of the constrained section and planning of the robot's travel trajectory, enabling the robot to complete the carrying task safely and efficiently in a complex working environment.

[0060] Among them, the pixel classification result refers to the category to which each pixel in the image belongs after analysis and judgment. The passable area refers to the area such as the ground and the passage that allows the robot to pass safely according to the pixel classification result. The non-passable area refers to the area such as obstacles and walls that will block the robot's progress according to the pixel classification result.

[0061] S32. Obtain the path centerline of the passable area. When the angle between the path centerline and the current driving direction of the robot terminal is greater than the threshold, retrieve the area map.

[0062] After identifying the passable area, the cloud server can obtain the path centerline of the passable area through the skeleton extraction algorithm. The path centerline represents the ideal driving path of the robot in the passable area. The cloud server can compare the path centerline with the current driving direction of the robot terminal. When the angle between the path centerline and the current driving direction of the robot terminal exceeds the set threshold, it clearly indicates that if the robot continues to drive in the current direction, it will not be able to move forward along the ideal path centerline and needs to perform a turning operation. Once it is determined that the robot needs to turn, the cloud server will quickly retrieve the area map. The area map contains comprehensive information about the working environment, with detailed distributions of passable areas and non-passable areas. By retrieving the area map, the cloud server can obtain richer information, providing sufficient data support for more accurately determining the constraint sections and planning the turning path of the robot in the future, so as to ensure that the robot can smoothly adjust its traveling direction in a complex environment and efficiently and safely complete the handling task.

[0063] Among them, the path centerline refers to the ideal driving path of the robot terminal, the current driving direction refers to the current driving direction of the robot terminal, the threshold refers to a pre-set angle value used to judge whether the angle between the path centerline and the current driving direction of the robot is too large, and the area map refers to the map of the working environment where the robot terminal is located.

[0064] S33. Determine the positioning point of the robot terminal in the area map, and determine the section where the positioning point is located and the robot terminal has not passed through as the constraint section.

[0065] Specifically, the cloud server can use the positioning system information of the robot terminal to determine its positioning point in the area map. The positioning point can accurately reflect the current position of the robot terminal in the entire working environment. Then, the cloud server uses the positioning point as a clue to find the section where the positioning point is located and the robot terminal has not passed through. These sections can be determined as constraint sections. Constraint sections are usually areas that the robot needs to focus on during subsequent travel, possibly because there are obstacles ahead or it is about to enter a complex section that requires turning.

[0066] After the cloud server determines the restricted section according to the environmental data, the following embodiments are also included in this solution: A1. Obtain the width of the target item corresponding to the direction perpendicular to the current driving direction, and obtain the width of the restricted section.

[0067] Specifically, after determining the restricted section, when the robot terminal transports the target item through the restricted section, it is necessary to determine whether the robot terminal can smoothly pass through the restricted section when transporting the target item. At this time, the width of the target item corresponding to the direction perpendicular to the current driving direction can be obtained, and the cloud server can obtain the width of the restricted section from the area map. By comparing the width of the target item in the direction perpendicular to the current driving direction with the width of the section, it can be intuitively judged whether the robot terminal can smoothly pass through the restricted section with the current handling posture. If the item width is less than the section width, the robot terminal can usually pass smoothly. If the item width is greater than the section width, it may not be able to pass smoothly. At this time, it may be necessary to make corresponding angular adjustments to the target item. Among them, the target item refers to the item transported by the robot terminal, the item width refers to the width of the target item in the direction perpendicular to the current driving direction, and the section width refers to the width of the restricted section.

[0068] A2. When the item width is greater than or equal to the section width, generate a reference model corresponding to the item specification, adjust the reference model according to the preset rotation angle, and obtain the reference width corresponding to the reference model.

[0069] Specifically, when the width of the item is greater than or equal to the width of the section, the robot terminal may not be able to smoothly carry the item through the restricted section. When the target item is an item that can be angle-adjusted, for example, when the target item is a wooden product, the robot terminal can make corresponding angle adjustments to the target item being carried. If the target item is an item that cannot be angle-adjusted, for example, when the target item is a glass product, a new carrying path can be replaced at this time. When making angle adjustments to the target item that can be angle-adjusted, the robot terminal can place the target item on the ground, and then can first construct a reference model corresponding to the item specifications of the target item, and the initial reference model is consistent with the placement posture of the target item. After constructing the reference model, multiple rotation angles can be preset, for example, the rotation angles can be 45°, 90°, etc. According to the preset rotation angle and the preset rotation direction, the reference model is angle-adjusted. Each time an adjustment is made, the width of the reference model in the direction perpendicular to the current driving direction of the target item is obtained, that is, the reference width. For example, the preset rotation direction can be the clockwise direction. When making the first angle adjustment to the reference model, the reference model can be rotated 45 degrees in the clockwise direction, and the reference width of the rotated reference model in the direction perpendicular to the current driving direction of the target item is obtained. Among them, the reference model refers to a virtual model corresponding to the target item, the preset rotation angle refers to the angle preset for rotating the reference model, and the reference width refers to the width of the reference model in the direction perpendicular to the current driving direction of the target item under the corresponding preset rotation angle.

[0070] A3. Determine the rotation angle with the reference width less than the section width as the target angle, and send the target angle to the robot terminal for angle adjustment of the item.

[0071] Specifically, when the reference width is less than the section width, determine the rotation angle corresponding to the current reference width of the reference model as the target angle. For example, when the reference model is rotated 90° clockwise, its corresponding reference width is less than the section width, then 90° can be determined as the target angle. After determining the target angle, the cloud server can send the target angle information to the robot terminal in a specific data format through the reliable communication link established with the robot terminal. After receiving the target angle information, the robot terminal can make corresponding angle adjustments to the target item. By making angle adjustments to the target item, the robot terminal can carry the item through the restricted section at an appropriate angle, ensuring the safe and efficient completion of the carrying task.

[0072] If the reference width is always greater than the section width after rotating the reference model for one week, then it can be considered that the target item may not be able to pass through the restricted section, and a new path can be replaced at this time. Among them, the target angle refers to the rotation angle of the reference model when the reference width is less than the section width.

[0073] S34. Obtain the turning angle of the constrained section, and calculate the turning speed at which the robot terminal adjusts from the current driving direction to the direction corresponding to the turning angle according to the item specifications. The section parameters include the turning angle, and the trajectory offset value includes the turning speed.

[0074] For a determined constrained section, the cloud server first needs to obtain its turning angle, which is one of the key parameters for calculating the trajectory offset value. The turning angle can be obtained by reading the section information in the area map or by analyzing the geometric shape of the constrained section using image recognition technology. Then, according to the previously determined item specifications, the cloud server can calculate the turning speed at which the robot terminal adjusts from the current driving direction to the direction corresponding to the turning angle. Larger objects require a slower speed when turning to ensure sufficient time and space to complete the turn and avoid colliding with surrounding obstacles, while lighter and smaller objects can appropriately increase the turning speed. By comprehensively considering the item specifications and the turning angle and applying specific physical models and algorithms, an appropriate turning speed can be calculated.

[0075] Among them, the turning angle refers to the angle that the robot terminal needs to change from the current straight driving direction to the target turning direction during driving, and the turning speed refers to the speed of the robot terminal during turning.

[0076] In some embodiments, "obtain the turning angle of the constrained section" in step S34 includes the following steps: S341. Obtain the action trajectory of the robot terminal in the area map within a preset time period, and determine the tangent direction of the action trajectory end point as the driving direction of the robot terminal.

[0077] Specifically, the cloud server can extract the action trajectory information of the robot terminal in the area map within a preset time period. For example, it can obtain the action trajectory in the past 10 seconds. After obtaining the action trajectory of the robot terminal, the cloud server can analyze the trajectory curve at the end point of the trajectory and determine the tangent direction corresponding to the end point of the trajectory, and can determine this tangent direction as the driving direction of the robot terminal.

[0078] Among them, the preset time period refers to the preset time period, the action trajectory refers to the trajectory that the robot terminal moves within the preset time period, and the driving direction refers to the corresponding direction when the robot terminal moves.

[0079] S342. Determine the center line of the constrained section. When the tangent directions of all path points on the center line are the same, determine the included angle between the center line and the driving direction as the turning angle, and determine the direction from the driving direction to the center line as the turning direction.

[0080] Specifically, after determining the driving direction of the robot terminal, the cloud server can obtain the center line of the constrained section through image processing techniques such as the skeleton extraction algorithm. For the constrained section, whether it is a straight channel, a bend that requires turning, or a section with obstacles that needs to be bypassed, its center line represents the most suitable path for the robot terminal to pass through. When analyzing the center line, if the tangent directions of all path points on the center line are the same, it can be considered that the constrained section is relatively regular and may be a straight channel. At this time, the cloud server can calculate the angle between the center line and the driving direction of the robot terminal through geometric calculation methods such as trigonometric functions. This angle is the turning angle that the robot terminal needs to turn when passing through the corresponding constrained section, that is, the turning angle. And by comparing the spatial position relationship between the center line and the driving direction, the direction from the driving direction to the center line can be determined as the turning direction of the robot terminal.

[0081] See Figure 3 , which is a schematic diagram for determining the turning direction provided by an embodiment of the present invention. As shown in Figure 3 , according to the tangent direction of the end point of the action trajectory of the robot terminal within a preset time period, the driving direction of the robot terminal can be determined, and the center line of the constrained section can be obtained. As can be seen from Figure 3 , there is an angle between the driving direction and the center line of the constrained section, and this angle can be determined as the turning angle, and the direction from the driving direction to the center line can be determined as the turning direction.

[0082] Among them, the center line refers to a representative path of the constrained section, representing the most suitable path for the robot terminal to pass through, and the turning direction refers to the direction that the robot terminal needs to take when turning from the current driving direction to the center line direction.

[0083] S343. When the tangent directions of the path points on the center line are inconsistent, determine the path point on the center line that is closest to the positioning point of the robot terminal as the starting point.

[0084] Specifically, when analyzing the center line of a constrained section and finding that the tangent directions of the path points are inconsistent, it can be considered that there may be multiple path segments that need to turn on this constrained section. In this case, determining a suitable starting point is crucial for accurately calculating the turning angle and turning direction. The cloud server can obtain the positioning point of the robot terminal in the area map, which is fed back to the cloud server in real time through the positioning system built into the robot terminal, such as GPS. Then, the cloud server can search for the path point on the center line of the constrained section that is closest to the positioning point and determine it as the starting point. Selecting the closest path point as the starting point can best fit the current actual position and movement trend of the robot, making the subsequent calculation of the turning angle and direction more accurate and effective. Herein, the starting point refers to the path point on the center line that is closest to the robot terminal.

[0085] S344. Taking the starting point as the traversal starting point, obtain the path sub-segment composed of the path points with the same tangent direction as the starting point, determine the included angle between the path sub-segment and the driving direction as the turning angle, and determine the direction from the driving direction to the path sub-segment as the turning direction.

[0086] After determining the starting point, the cloud server takes the starting point as the traversal starting point and searches for path points along the center line of the constrained section. Using the method for judging the consistency of the curve tangent direction, it checks the path points on the center line one by one, and can filter out the path points with the same tangent direction as the starting point. These path points together form a path sub-segment, which represents a section of the path with the same turning trend starting from the current position of the robot. After obtaining the path sub-segment, the cloud server can calculate the included angle between the center line of the path sub-segment and the driving direction of the robot terminal, determine this included angle as the turning angle, and can determine the direction from the driving direction to the path sub-segment as the turning direction. Herein, the traversal starting point refers to the starting point for traversing the path points with the same tangent direction, and the path sub-segment refers to the path segment on the constrained section with the same turning trend.

[0087] In some embodiments, the step of "calculating the turning speed of the robot terminal from the current driving direction to the direction corresponding to the turning angle according to the item specification" in step S34 includes the following steps: S345. Calculate the volume of the object based on the item specification, and perform weighted processing on the reciprocal of the object volume to obtain the first adjustment coefficient.

[0088] Specifically, according to the item specifications, the corresponding volume of the object can be calculated. For example, when the obtained item specifications are the length, width, and height of the item, the volume of the object can be obtained by multiplying the length, width, and height. After obtaining the volume of the object, the reciprocal of the object volume is weighted to obtain the first adjustment coefficient. The weighting process takes into account various factors, such as the stability requirements of different types of objects when turning and the carrying capacity of the robot. For an object with a larger volume, its reciprocal volume is smaller, and after weighting, the first adjustment coefficient is relatively smaller, which means that when calculating the turning speed later, the turning speed will be reduced to adapt to the need for more time and space to complete the turn during turning and avoid colliding with surrounding obstacles. Among them, the first adjustment coefficient refers to the coefficient for adjusting the speed according to the object volume.

[0089] S346, obtain the item attribute. When the item attribute is a vulnerable attribute, obtain the preset coefficient corresponding to the risk level of the target item as the second adjustment coefficient.

[0090] Specifically, the cloud server can obtain the attribute information of the item being carried by the robot terminal. The item attribute covers important features such as the material of the item and whether it is vulnerable. When the item attribute is a vulnerable attribute, the cloud server can further obtain the preset coefficient corresponding to the risk level of the target item as the second adjustment coefficient. The risk level of the target item is usually divided according to factors such as the fragility of its material and its value. For example, for glass products, the risk level may be relatively high, while for plastic products, the risk level is relatively low. Each risk level has a corresponding preset coefficient, which is obtained through a large number of experiments and practical experiences. For example, assuming that the risk levels are divided into high, medium, and low levels, and the corresponding preset coefficients are 0.5, 0.7, and 0.9 respectively. When it is determined that the target item is a vulnerable item with a high risk level, the corresponding second adjustment coefficient is 0.5. This coefficient is used to further reduce the turning speed when calculating the turning speed to ensure the safety of the vulnerable item during the turning process.

[0091] Among them, the item attribute refers to the characteristics inherent in the item itself, the vulnerable attribute refers to the characteristic that the item is easily damaged when subjected to external forces, the target item refers to the item being carried by the robot terminal, the risk level refers to the risk classification level of the target item based on its fragility, which can be used to evaluate the possible loss risk of the item during handling, the preset coefficient refers to the value preset for each risk level and used for adjustment when calculating the turning speed, and the second adjustment coefficient refers to the coefficient for adjusting the speed determined according to the item attribute.

[0092] S347. The first adjustment coefficient and the second adjustment coefficient are comprehensively calculated to obtain the total adjustment coefficient, and the turning speed is obtained by multiplying the total adjustment coefficient by the reference speed.

[0093] Specifically, after obtaining the first adjustment coefficient and the second adjustment coefficient, the cloud server comprehensively calculates the two to obtain the total adjustment coefficient. The calculation method of the total adjustment coefficient can select a suitable algorithm according to the actual situation. For example, the corresponding weighting factors can be determined according to the degree of emphasis on the object volume and the fragile property of the item, and the first adjustment coefficient and the second adjustment coefficient are simply weighted and summed according to the weighting factors corresponding to the object volume and the fragile property of the item to obtain the corresponding total adjustment coefficient. Then, the turning speed can be obtained by multiplying the total adjustment coefficient by the preset reference speed. Through such a calculation method, factors such as the item specifications and the turning angle are comprehensively considered, and a suitable speed is determined for the turning operation of the robot in the restricted section.

[0094] Among them, the total adjustment coefficient refers to the coefficient obtained by comprehensively calculating the first adjustment coefficient and the second adjustment coefficient, and the reference speed refers to the preset driving speed of the robot in the ideal situation without considering the object specifications and the section conditions.

[0095] Through the above implementation manner, the risk of the object falling or being damaged during handling can be effectively reduced, and at the same time, the passing efficiency of the robot terminal in the restricted section can be improved.

[0096] S4. Obtain the real-time state data of the robot terminal when executing the control instruction, and the cloud server generates action adjustment information based on the real-time state data to update the instruction of the robot terminal.

[0097] After receiving the control instructions sent by the cloud server, the robot terminal can perform corresponding turning operations during the object handling process according to the control instructions. And during the process of handling the object according to the control instructions, its sensor module can continuously and real-time collect the status data during the handling process, that is, the real-time status data, and can continuously send its own real-time status data to the cloud server. The real-time status data includes the vibration data of the robot terminal. The vibration data is an important indicator for evaluating the stability of the robot. When the vibration value is small, it indicates that the robot is running smoothly and the object handling process is stable. While when the vibration value increases, it may mean that the robot encounters unstable factors, such as uneven ground or too fast movement speed, etc. These may all cause the object to shake during the handling process, thus affecting the smooth completion of the task. The cloud server can analyze the received real-time status data, especially the vibration data. When detecting abnormal vibration data, the cloud server can generate corresponding adjustment information, that is, action adjustment information. The action adjustment information can correspondingly adjust the speed of the robot terminal to slow down or eliminate the vibration, so as to maintain the stability of the object during the handling process. After receiving these action adjustment information, the robot terminal will immediately execute the corresponding instruction update and adjust its own movement speed. Through such a real-time status monitoring and instruction update mechanism, the cloud server can ensure that the robot always maintains the best state when executing the control instructions, and can quickly make adjustments even when encountering unstable factors, thus greatly improving the stability of object handling.

[0098] Among them, the real-time status data refers to the data continuously and real-time collected by the sensor module of the robot terminal during the execution of the control instructions and sent to the cloud server. The action adjustment information refers to the instruction information generated by the cloud server based on the analysis of the received real-time status data for adjusting the motion state of the robot terminal.

[0099] On the basis of the above embodiments, the specific implementation manner of step S4 can be: S41, obtain the vibration value based on the sensor module of the robot terminal. The real-time status data includes the vibration value.

[0100] Specifically, the sensor module equipped on the robot terminal can sense the vibration generated by the robot terminal during the handling process due to various reasons, such as uneven ground, too fast movement speed, etc., and convert it into a quantifiable value, that is, the vibration value. The vibration value can be used as a part of the real-time status data and continuously sent to the cloud server for analysis. The larger the vibration value, the more unstable the robot terminal can be considered during the handling process, and it may cause the object to fall. Among them, the vibration value refers to the quantifiable value collected by the vibration sensor of the robot terminal, indicating the vibration intensity of the robot.

[0101] S42. When the vibration value is greater than the vibration threshold, obtain the vibration difference between the vibration value and the reference vibration value, and calculate the weighted value of the ratio of the vibration difference to the reference difference to obtain the speed adjustment value.

[0102] Specifically, after receiving the real-time status data sent by the robot terminal, the cloud server first detects whether the vibration value exceeds the preset vibration threshold. The vibration threshold is a preset limit used to determine whether the current vibration level of the robot terminal is within an acceptable range. If the vibration value exceeds the vibration threshold, it can be considered that the robot terminal may be relatively unstable during the process of carrying an object and corresponding adjustments are needed. At this time, the cloud server will calculate the difference between the vibration value and the reference vibration value, that is, the vibration difference. The reference vibration value is a preset value representing the vibration level that the robot terminal should have in an ideal state, and the vibration difference can reflect the degree of deviation between the current vibration state and the ideal state. Then, after calculating the ratio of the vibration difference to the reference difference and performing weighted calculation, the speed adjustment value can be obtained. According to the speed adjustment value, the speed of the robot terminal can be adjusted accordingly. Among them, the vibration threshold refers to the preset limit value used to determine whether the vibration level exceeds the standard, the reference vibration value refers to the preset value representing the ideal vibration state, the vibration difference refers to the difference between the current vibration value and the reference vibration value, the reference difference refers to the vibration difference in the preset standard state, and the speed adjustment value refers to the value that can be used to adjust the speed of the robot terminal.

[0103] S43. Obtain the action adjustment information by subtracting the speed adjustment value from the current speed of the robot terminal, and update the instruction of the speed of the robot terminal based on the action adjustment information.

[0104] Specifically, the cloud server can obtain the current speed of the robot terminal, subtract the speed adjustment value from the current speed to obtain the action adjustment information, and update the speed of the robot terminal accordingly according to the action adjustment information.

[0105] Through the above implementation manner, the stability of the robot terminal during object handling can be improved.

[0106] See Figure 4 , which is a schematic structural diagram of a system for controlling a robot by the cloud provided by an embodiment of the present invention. The data processing system of the system for controlling a robot based on the cloud includes: An upload module, configured to enable the robot terminal to obtain environmental data and object status information based on the sensor module, and upload the environmental data and object status information to the cloud server based on the communication module; A determination module, configured to enable the cloud server to determine the object specification according to the object status information and calculate the grasping point according to the object specification; A calculation module, configured to enable a cloud server to determine a restricted section according to environmental data, calculate a trajectory offset value according to an object specification and section parameters, and generate a control instruction based on a grasping point and the trajectory offset value and send the control instruction to a robot terminal; A generation module, configured to obtain real-time status data of the robot terminal when executing the control instruction, and enable the cloud server to generate action adjustment information based on the real-time status data to update the instruction for the robot terminal.

[0107] Figure 4 The device of the illustrated embodiment can correspondingly be used to execute Figure 1 the steps in the method illustrated embodiment, and the implementation principle and technical effects are similar, which will not be elaborated here.

[0108] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for controlling a robot via the cloud, characterized in that: include: The robot terminal obtains environmental data and object status information based on the sensor module, and uploads the environmental data and object status information to the cloud server based on the communication module; The cloud server determines the object specifications based on the object status information and calculates the grasping points based on the object specifications; The cloud server determines the constrained sections based on the environmental data, calculates the trajectory deviation value based on the object specifications and section parameters, and generates control instructions based on the grasping points and trajectory deviation values ​​and sends them to the robot terminal; The real-time status data of the robot terminal when executing the control instructions is obtained, and the cloud server generates action adjustment information based on the real-time status data to update the instructions for the robot terminal.

2. The method according to claim 1, characterized in that The cloud server determines the object specifications based on the object status information and calculates the grasping points based on the object specifications, including: The cloud server determines the object specifications based on the image data of each viewing angle, and the object status information includes the image data of each viewing angle; Get the maximum arm span length of the robot terminal, determine the semi-circumference according to the object specifications, and when the maximum arm span length is greater than the semi-circumference, determine the center point of the opposite side of the object as the grasping point; When the maximum arm length is less than half the circumference, multi-machine collaboration is triggered, and the transport surfaces corresponding to the multiple robot terminals are determined, and the points on the transport surface at the center point grabbing offset distance are determined as the grabbing points corresponding to each robot terminal.

3. The method according to claim 2, characterized in that When the maximum arm length is less than half the circumference, multi-machine collaboration is triggered to determine the handling surfaces corresponding to the multiple robot terminals, and determine the points on the handling surface at the center point grabbing offset distance as the grabbing points corresponding to each robot terminal, including: Determine the half-extended arm length of the robot terminal, and obtain the grasping range corresponding to the half-extended arm length on the side of each object; When there is a grasping range that is smaller than the range threshold of the handling surface, the number of terminals for multi-machine collaboration is determined to be the preset number at the four corners, and the sides of adjacent objects are determined in turn as the handling surfaces corresponding to the corresponding robot terminals; When there is no grasping range smaller than the range threshold of the transport surface, the number of terminals of the multi-machine collaboration is determined to be the preset number of diagonal objects, and the side faces of the objects that are diagonally adjacent are determined to be the transport surfaces corresponding to the corresponding robot terminals; The point located at the center point grab offset distance in the transport surface is determined as the grab point corresponding to each robot terminal.

4. The method according to claim 2, characterized in that: The following steps are used to determine the grab offset distance, including: The object volume is calculated according to the object specifications, and the offset coefficient is obtained based on the ratio of the reference volume and the object volume; The offset coefficient and the reference offset distance are multiplied and weighted to obtain the grab offset distance.

5. The method according to claim 1, characterized in that The cloud server determines the constraint section based on the environmental data and calculates the trajectory deviation value based on the object specifications and section parameters, including: The cloud server identifies the passable area and the impassable area according to the pixel classification results in the environmental data, wherein the environmental data includes image data corresponding to the positive viewing angle; Obtain the path centerline of the passable area. When the angle between the path centerline and the current driving direction of the robot terminal is greater than a threshold, retrieve the area map. Determine the positioning point of the robot terminal in the regional map, and determine the road section where the positioning point is located and the robot terminal has not passed as the constrained road section; Get the turning angle of the constrained section, and calculate the turning speed of the robot terminal from the current driving direction to the direction corresponding to the turning angle according to the item specifications. The section parameters include the turning angle, and the trajectory offset value includes the turning speed.

6. The method according to claim 5, characterized in that Get the turning angle of the constraint segment, including: Obtain the movement trajectory of the robot terminal in the regional map within a preset time period, and determine the tangent direction of the end point of the movement trajectory as the driving direction of the robot terminal; Determine the center line of the constrained section, and when the tangent directions of all path points on the center line are consistent, determine the angle between the center line and the driving direction as the turning angle, and determine the direction from the driving direction to the center line as the turning direction; When the tangent directions of the path points on the center line are inconsistent, the path point on the center line closest to the positioning point of the robot terminal is determined as the starting point; Taking the starting point as the traversal starting point, obtain the path subsegment composed of path points with the same tangent direction as the starting point, determine the angle between the path subsegment and the driving direction as the turning angle, and determine the direction from the driving direction to the path subsegment as the turning direction.

7. The method according to claim 6, characterized in that Calculate the turning speed of the robot terminal from the current driving direction to the direction corresponding to the turning angle according to the item specifications, including: The volume of the object is calculated based on the item specifications, and the inverse of the volume of the object is weighted to obtain a first adjustment coefficient; Acquire the attribute of the item. When the attribute of the item is a fragile attribute, acquire a preset coefficient corresponding to the risk level of the target item as the second adjustment coefficient; The first adjustment coefficient and the second adjustment coefficient are comprehensively calculated to obtain a total adjustment coefficient, and the turning speed is obtained according to the product of the total adjustment coefficient and the reference speed.

8. The method according to claim 5, characterized in that After the cloud server determines the restricted road section based on the environmental data, it also includes: Obtain the object width corresponding to the target object in the direction perpendicular to the current driving direction, and obtain the section width of the constraint section; When the width of the object is greater than or equal to the width of the road section, a reference model corresponding to the specification of the object is generated, the reference model is adjusted according to a preset rotation angle, and a reference width corresponding to the reference model is obtained; The rotation angle at which the reference width is smaller than the road width is determined as the target angle, and the target angle is sent to the robot terminal to adjust the angle of the object.

9. The method according to claim 1, characterized in that: The real-time status data of the robot terminal when executing the control command is obtained. The cloud server generates action adjustment information based on the real-time status data to update the command of the robot terminal, including: The sensor module based on the robot terminal obtains the vibration value, and the real-time status data includes the vibration value; When the vibration value is greater than the vibration threshold, the vibration difference between the vibration value and the reference vibration value is obtained, and the speed adjustment value is obtained by weighted calculation of the ratio of the vibration difference to the reference difference; The motion adjustment information is obtained by subtracting the speed adjustment value from the current speed of the robot terminal, and the speed of the robot terminal is updated based on the motion adjustment information.

10. A cloud-based robot control system, characterized in that: include: An upload module is used for the robot terminal to obtain environmental data and object status information based on the sensor module, and upload the environmental data and object status information to the cloud server based on the communication module; A determination module, used for the cloud server to determine the object specifications according to the object state information and calculate the grasping point according to the object specifications; A calculation module is used for the cloud server to determine the constraint section according to the environmental data, calculate the trajectory deviation value according to the object specifications and the section parameters, and generate a control command according to the grasping point and the trajectory deviation value and send it to the robot terminal; The generation module is used to obtain the real-time status data of the robot terminal when executing the control instructions. The cloud server generates action adjustment information based on the real-time status data to update the instructions of the robot terminal.

Images