Operation method and device of object control system and related equipment

Abstract

The embodiment of the invention provides an operation method of an object control system, which is applied to the technical field of control and is used for managing the object control system. The object control system comprises a mechanical arm device, a position detection device and an object conveying device. Based on the initial parameters of the mechanical arm device, the position detection device and the object transmission device, the installation position of each device is calculated, the target control position and the actual control position of a control object of the mechanical arm device are determined, and the operation result of the mechanical arm device for controlling the target object at the actual control position is obtained; if the operation result meets the adjustment condition, the adjustment position of any equipment is determined based on the target control position, the actual control position and the installation position of any equipment included in the object control system. Therefore, the adjustment position of any device included in the object control system can be automatically determined, the installation position of the device can be conveniently adjusted, and the success rate of object control is improved. The embodiment of the invention further provides a corresponding device and related equipment.

Description

[0001] This application claims the priority of the Chinese patent application filed with the State Intellectual Property Office on July 28, 2023, with application number 202310943417.0 and invention name “A position adjustment method, device and system”, all contents of which are incorporated by reference in this application. Technical Field

[0002] The present application relates to the field of control technology, and in particular to an operation method, device and related equipment of an object control system. Background Art

[0003] With the development of robotics technology, robots can undertake a variety of tasks. For example, a robot arm is a mechanical device that can move and control objects. One end of the robot arm is usually fixed to a base, and the other end has a device for controlling objects, such as a gripper. The robot arm grabs objects that appear at a specific location and moves the objects to other locations to achieve the transfer of objects.

[0004] During the operation of the robotic arm, due to the influence of the force or the change of the working environment, the accuracy of the robotic arm in controlling the object may decrease, affecting the normal execution of the task of controlling the object. Summary of the invention

[0005] The embodiments of the present application provide an operation method, device and related equipment of an object control system, which are intended to determine the adjustment position of any device included in the object control system, realize closed-loop control of the object control system, and thereby reduce the error of the object control system in controlling the object.

[0006] In a first aspect, the present application provides an operation method of an object control system. The method can be applied to a management device. The management device can belong to an object control system, and can also be deployed on a cloud platform to provide a cloud service for managing the object control system. The method is used to manage the object control system. The object control system includes a manipulator device, a position detection device, and an object transmission device, the object transmission device is used to transmit a target object, and the manipulator device is used to control the target object during the process of the object transmission device transmitting the target object. The position detection device is used to detect the position of the target object. Based on the initial parameters of the manipulator device, the position detection device, and the object transmission device, the installation positions of the manipulator device, the position detection device, and the object transmission device are calculated. In this way, according to the initial parameters of the manipulator device, the position detection device, and the object transmission device, a more suitable installation position of the manipulator device, the position detection device, and the object transmission device can be determined, so as to automatically determine the installation position of the device and reduce the cost of manual calculation and manual debugging of the installation position. Determine the target control position and the actual control position of the manipulator device. The target control position is the target position of the manipulator device controlling the target object. The actual control position is the actual position of the manipulator device controlling the target object. Determine the operation result of the robot arm device controlling the target object at the actual control position. If the operation result meets the adjustment condition, determine the adjustment position of any device included in the object control system based on the target control position, the actual control position and the installation position of any device included in the object control system. The adjustment position is used to indicate the adjustment of the installation position of any device included in the object control system. Based on the operation result, it can be determined whether it is necessary to adjust the installation position of any device included in the object control system to achieve closed-loop control of the object control system. In the case where it is necessary to adjust the installation position of any device included in the object control system, the adjustment position of any device included in the object control system is automatically determined according to the target control position, the actual control position and the installation position of any device included in the object control system, so as to adjust the installation position of the device, thereby improving the accuracy of the object control system in controlling the object and reducing the cost of manually debugging the installation position of the device.

[0007] In a possible implementation, the adjustment range of the relative position between the manipulator device, the position detection device and the object transmission device is first determined based on the initial parameters of the manipulator device, the position detection device and the object transmission device. Within the adjustment range, the adjustment position of any device among the manipulator device, the position detection device and the object transmission device is determined so that the difference between the target control position and the actual control position meets the condition. The adjustment range is used to determine the range of the relative position between the devices when the object control system can operate normally. In this way, on the basis of satisfying the object control system to control the object, the difference between the target control position and the actual control position can be reduced to determine a more accurate adjustment position.

[0008] In a possible implementation, the position detection device is a camera, and the object transmission device is a conveyor belt. The initial parameters of the robotic arm device, the position detection device, and the object transmission device include: the field of view angle of the camera, the movement speed of the conveyor belt, the width of the conveyor belt, the movement speed of the robotic arm device, the movement range of the robotic arm device, the first duration, and the second duration. The first duration is the duration from the moment when the target object appears within the detection range of the camera to the moment when the target object disappears from the detection range of the camera. The second duration is the duration from the moment when the target object disappears from the detection range of the camera to the moment when the robotic arm device performs a control operation on the target object.

[0009] In a possible implementation, the minimum relative position between the camera and the conveyor belt is determined based on the field of view of the camera and the width of the conveyor belt. The minimum relative position between the camera and the robotic arm device is determined based on the movement speed of the conveyor belt, the first duration, and the second duration. The first maximum relative position between the conveyor belt and the robotic arm device is determined based on the movement range of the robotic arm device. The second maximum relative position between the conveyor belt and the robotic arm device is determined based on the movement speed of the robotic arm device, the first duration, and the second duration.

[0010] In a possible implementation, the operation result is a position deviation between a target control position and an actual control position. The adjustment condition is that the position deviation is greater than or equal to a position deviation threshold. The position deviation threshold is determined based on the control range of the manipulator device. In this way, when the position deviation between the target control position and the actual control position is greater than or equal to the position deviation threshold, the adjustment position of any device included in the object control system can be triggered to be determined, so that the adjustment position of any device included in the object control system can be determined more timely, which is convenient for timely adjustment of the installation position of the device.

[0011] In a possible implementation, the adjustment condition further includes the number of control operations with a position deviation greater than or equal to a position deviation threshold, which is greater than or equal to the number threshold. This can avoid frequently determining the adjustment position of any device included in the object control system, reducing calculation costs.

[0012] In a possible implementation, the object control system includes a position control device. The position control device is used to control the installation position of the manipulator device, the position detection device, and the object transmission device. The method also includes: sending an adjustment instruction to the position control device of the object control system. The adjustment instruction is generated based on the adjustment position. The adjustment instruction is used to instruct the position control device to adjust the installation position of any device included in the object control system. In this way, the installation position of the device included in the object control system can be automatically adjusted, thereby improving the adjustment efficiency of the installation position of the device.

[0013] In a possible implementation, the object control system includes a display device. The method further includes: sending first adjustment information to the display device. The first adjustment information is generated based on the adjustment position so that the user can adjust the installation position of the device of the object control system based on the first adjustment information displayed by the display device.

[0014] In a possible implementation, the method further includes: acquiring a real-time installation position of a device included in the object control system, and sending second adjustment information to a display device. The second adjustment information is generated based on the adjustment position and the real-time installation position, so as to guide a user in real time to adjust the installation position of the object control system based on the second adjustment information displayed by the display device.

[0015] In a possible implementation, the method further includes: if it is determined that the target control position is not within the safe position range, sending a safety prompt message to the display device. The safe position range is determined based on the operating range of the robotic arm device. In this way, the operating safety of the object control system can be detected, and the user can be promptly reminded of the operating safety problems of the object control system.

[0016] In a second aspect, the present application provides an operating device for an object control system. The device is used to manage the object control system. The object control system includes a robotic arm device, a position detection device, and an object transmission device. The object transmission device is used to transmit a target object. The robotic arm device is used to control the target object during the process of the object transmission device transmitting the target object. The position detection device is used to detect the position of the target object. The device includes: a calculation module, which is used to calculate the installation positions of the robotic arm device, the position detection device and the object transmission device based on the initial parameters of the robotic arm device, the position detection device and the object transmission device; a first determination module, which is used to determine the target control position of the robotic arm device, and the target control position is the target position of the robotic arm device controlling the target object; an acquisition module, which is used to acquire the actual control position of the robotic arm device, and the actual control position is the actual position of the robotic arm device controlling the target object; a second determination module, which is used to determine the operation result of the robotic arm device controlling the target object at the actual control position, and if the operation result meets the adjustment condition, the adjustment position of any device included in the object control system is determined based on the target control position, the actual control position and the installation position of any device included in the object control system, and the adjustment position is used to indicate the adjustment of the installation position of any device included in the object control system.

[0017] In a possible implementation, the second determination module is used to determine the adjustment position of any device included in the object control system based on the target control position, the actual control position and the installation position of any device included in the object control system, including: the second determination module is used to determine the adjustment range of the relative position between the robotic arm device, the position detection device and the object transmission device based on the initial parameters of the robotic arm device, the position detection device and the object transmission device; determine the adjustment position of any device among the robotic arm device, the position detection device and the object transmission device, so that the relative position between the robotic arm device, the position detection device and the object transmission device after adjustment based on the adjustment position is within the adjustment position range, and the difference between the target control position and the actual control position satisfies a condition.

[0018] In one possible implementation, the position detection device is a camera, the object transmission device is a conveyor belt, and the initial parameters of the robotic arm device, the position detection device, and the object transmission device include: the field of view angle of the camera, the movement speed of the conveyor belt, the width of the conveyor belt, the movement speed of the robotic arm device, the movement range of the robotic arm device, a first time length and a second time length, wherein the first time length is the time length from the moment when the target object appears within the detection range of the camera to the moment when the target object disappears from the detection range of the camera, and the second time length is the time length from the moment when the target object disappears from the detection range of the camera to the moment when the robotic arm device performs a control operation on the target object.

[0019] In a possible implementation, the second determination module is used to determine the adjustment position of any device included in the object control system based on the target control position, the actual control position and the installation position of any device included in the object control system, including: the second determination module is used to determine the minimum relative position between the camera and the conveyor belt based on the field of view of the camera and the width of the conveyor belt; based on the movement speed of the conveyor belt, the first time length and the second time length, determine the minimum relative position between the camera and the robotic arm device; based on the moving range of the robotic arm device, determine the first maximum relative position between the conveyor belt and the robotic arm device; based on the moving speed of the robotic arm device, the first time length and the second time length, determine the second maximum relative position between the conveyor belt and the robotic arm device.

[0020] In a possible implementation, the operation result is a position deviation between the target control position and the actual control position, the adjustment condition is that the position deviation is greater than or equal to a position deviation threshold, and the position deviation threshold is determined based on a control range of the robotic arm device.

[0021] In a possible implementation, the adjustment condition further includes the number of control operations of the position deviation being greater than or equal to the position deviation threshold, which is greater than or equal to the number threshold.

[0022] In a third aspect, the present application provides a computing device cluster, the computing device cluster includes at least one computing device, each computing device includes a processor and a memory; the memory is used to store instructions, and when the computing device cluster is running, the processor in each computing device executes the instructions stored in the memory, so that the computing device cluster executes the operation method of the object control system in the above-mentioned first aspect or any possible implementation of the first aspect. It should be noted that the memory can be integrated into the processor or can be independent of the processor. Each computing device may also include a bus. The processor is connected to the memory via a bus. The memory may include a readable memory and a random access memory.

[0023] In a fourth aspect, the present application provides a computer-readable storage medium, which stores instructions. When the computer-readable storage medium is run on a computing device cluster (the computing device cluster includes at least one computing device), the computing device cluster executes the method for operating the object control system described in the first aspect or any one of the implementations of the first aspect.

[0024] In a fifth aspect, the present application provides a computer program product comprising instructions, which, when executed on a computing device cluster (the computing device cluster includes at least one computing device), enables the computing device cluster to execute the method for operating the object control system described in the first aspect or any one of the implementations of the first aspect.

[0025] Based on the implementations provided in the above aspects, this application can also be further combined to provide more implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1a A schematic diagram of an application scenario provided for an embodiment of the present application;

[0027] Figure 1b A schematic diagram of an application scenario provided for an embodiment of the present application;

[0028] Figure 2 A schematic diagram of a flow chart of an operating method of an object control system provided in an embodiment of the present application;

[0029] Figure 3 A schematic diagram of the structure of an object control system provided in an embodiment of the present application;

[0030] Figure 4A schematic diagram of the relative positions of devices included in an object control system provided in an embodiment of the present application;

[0031] Figure 5 A schematic diagram of the structure of an operating device of an object control system provided in an embodiment of the present application;

[0032] Figure 6 A schematic diagram of the structure of a computing device provided in an embodiment of the present application;

[0033] Figure 7 A schematic diagram of the structure of a computing device cluster provided in an embodiment of the present application;

[0034] Figure 8 A schematic diagram of the structure of another computing device cluster provided in an embodiment of the present application. DETAILED DESCRIPTION

[0035] The scheme in the embodiments provided in this application will be described below in conjunction with the drawings in this application.

[0036] The terms "first", "second", etc. in the specification, claims and drawings of this application are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances. This is just a way of distinguishing objects with the same attributes when describing the embodiments of this application.

[0037] Robotic arm devices are widely used in business scenarios for handling object control tasks. Robotic arm devices use the collected position information of objects to control objects at specific positions where the objects move. However, during the operation of the robotic arm device, it may be affected by external forces, causing the position of the robotic arm device to change. For example, the robotic arm device may collide with other devices during the process of controlling the object, causing the position of the robotic arm device to move. For another example, the movement of the robotic arm device itself may cause its position to shift. In addition, the movement process of the object may also change. Such factors will cause the robotic arm device to fail to predict the specific position of the controlled object, and then cause the robotic arm device to fail to successfully control the object at a specific position. At present, there is a lack of methods for processing the control errors of robotic arm devices, which affects the success rate of the robotic arm device in performing object control tasks.

[0038] Based on this, an embodiment of the present application provides an operation method of an object control system. The method is used to manage the object control system. The object control system includes a mechanical arm device, a position detection device and an object transmission device. The object transmission device is used to transmit the object. The mechanical arm device is used to control the object during the object transmission device transmitting the object. The position detection device is used to detect the position of the object. Based on the initial parameters of the mechanical arm device, the position detection device and the object transmission device, the installation positions of the mechanical arm device, the position detection device and the object transmission device are calculated, the target control position and the actual control position of the control object of the mechanical arm device are determined, and the operation result of the mechanical arm device controlling the target object at the actual control position is obtained. If the operation result meets the adjustment condition, the adjustment position of any device included in the object control system is determined based on the target control position, the actual control position and the installation position of any device included in the object control system, and the adjustment position is used to indicate the adjustment of the installation position of any device included in the object control system. In this way, the adjustment position of any device included in the object control system can be automatically determined, and then it is convenient to use the adjustment position to adjust the installation position of the device included in the object control system, improve the success rate of the subsequent execution of the object control task by the object control system, and realize the closed-loop control of the object control system.

[0039] The embodiments of the present application do not limit the application scenarios of the operation method of the object control system. As some examples, Figure 1a and Figure 1b A schematic diagram of an application scenario provided for an embodiment of the present application.

[0040] See also Figure 1a and Figure 1b As shown, the operation method of the object control system provided in the embodiment of the present application can be applied to the management device 100. The management device 100 is used to manage the object control system 101. Figure 1a As shown, the management device 100 can be a device belonging to the object control system 101. Figure 1b As shown, the management device 100 can be a device deployed on the cloud, which is not limited in the embodiments of the present application. The management device 100 is connected to the object control system 101. The object control system 101 includes at least a mechanical arm device 1011, a position detection device 1012, and an object transmission device 1013. Among them, the mechanical arm device 1011 is a mechanical device with control functions such as object grasping, moving and placing. The position detection device 1012 is used to detect the position of an object. The position detection device 1012 is, for example, an image acquisition device, such as a camera. The position detection device 1012 is, for example, a position sensing device, such as a sensor for detecting the position of an object. The object transmission device 1013 is used to carry objects and transmit objects. The object transmission device 1013 is, for example, a conveyor belt.

[0041] The management device 100 calculates the installation positions of the manipulator device 1011, the position detection device 1012 and the object transmission device 1013 based on the initial parameters of the manipulator device 1011, the position detection device 1012 and the object transmission device 1013. The management device 100 determines the target control position of the manipulator device 1011 controlling the target object and the actual control position of the manipulator device 1011 controlling the target object. The management device 100 determines the operation result of the manipulator device controlling the target object at the actual control position. If the operation result satisfies the adjustment condition, the adjustment position of any device included in the object control system is determined based on the target control position, the actual control position and the installation position of any device included in the object control system.

[0042] In this way, closed-loop control of the object by the object control system can be achieved. When the adjustment conditions are met, the adjustment position of any device included in the object control system can be determined, which facilitates adjustment of the installation position of the device included in the object control system based on the adjustment position, thereby improving the success rate of the object control system's subsequent control of the object.

[0043] It should be noted that the above Figure 1a and Figure 1b The application scenarios shown are only examples, and the application scenarios of the operation method of the object control system provided by this application are not limited to Figure 1a and Figure 1b Limitations of the application scenarios shown.

[0044] Next, various non-limiting specific implementations of the operation method of the object control system provided by the present application are described in detail. Figure 2 As shown in the figure, this figure is a schematic diagram of a flow chart of an operation method of an object control system provided in an embodiment of the present application. The method can be applied to, for example, the above Figure 1a and Figure 1b The management device 100 in the illustrated scenario. The method includes S201-S204.

[0045] S201: Calculate installation positions of the robot arm device, the position detection device, and the object transmission device based on initial parameters of the robot arm device, the position detection device, and the object transmission device.

[0046] When the object control system is initially installed, the initial installation position of each device included in the object control system can be determined based on the initial parameters of each device included in the object control system.

[0047] The initial parameters of each device included in the object control system are, for example, parameters related to the operation of each device that are pre-set by each device. As an example, the position detection device is a camera, and the object transmission device is a conveyor belt. Figure 3The initial parameters of each device include:

[0048] The camera's field of view, the conveyor belt's speed, the conveyor belt's width, the robot's speed, the robot's range, the first duration, and the second duration. The first duration is the duration from the moment the target object appears within the camera's detection range to the moment the target object leaves the detection range. The second duration is the duration from the moment the target object leaves the camera's detection range to the moment the robot performs a control operation.

[0049] The first duration can be calculated based on the detection range of the camera and the moving speed of the conveyor belt. The field of view of the camera, the moving speed of the conveyor belt, the width of the conveyor belt, the moving speed of the robotic arm device, and the moving range of the robotic arm device can be determined based on the device parameters of the camera, the device parameters of the conveyor belt, and the device parameters of the robotic arm device, respectively. The second duration can be a preset value.

[0050] In one possible implementation, a pre-trained artificial intelligence model is used to determine the installation location of the equipment included in the object control system. The artificial intelligence model can be trained based on training data. The training data, for example, includes initial parameters and labels of a manipulator device, a position detection device, and an object transfer device. The labels are the installation locations of the manipulator device, the position detection device, and the object transfer device. The labels can be installation locations generated based on the equipment installation process of the historical object control system, or experience installation locations of manually installed object control systems.

[0051] The initial parameters of the robotic arm device, the position detection device and the object transmission device are input into the trained artificial intelligence model to obtain the initial relative position of the object control system output by the artificial intelligence model.

[0052] In another possible implementation, based on the initial parameters of the robot device, the position detection device and the object transmission device, the adjustment range of the relative position between the robot device, the position detection device and the object transmission device is determined. Within the adjustment range, the installation positions of the robot device, the position detection device and the object transmission device are determined.

[0053] Taking the above initial parameters as an example, the embodiment of the present application provides a possible implementation method for determining the adjustment range of the relative position between the robot arm device, the position detection device and the object transmission device based on the initial parameters of the robot arm device, the position detection device and the object transmission device, including the following four steps:

[0054] A1: Determine the minimum relative position between the camera and the conveyor belt based on the camera's field of view and the width of the conveyor belt.

[0055] The camera's field of view determines the camera's detection range of the target object. The distance between the camera and the conveyor belt affects the camera's detection range of the conveyor belt. The detection range included in the camera's field of view must at least include the entire width of the conveyor belt to ensure that the camera can detect the target object. According to the camera's field of view and the width of the conveyor belt, determine the minimum relative position between the camera and the conveyor belt.

[0056] As an example, see formula (1):

[0057]

[0058] fov is the field of view of the camera. W is the width of the conveyor belt. Figure 4 , Figure 4 A schematic diagram of the relative positions of devices included in an object control system provided in an embodiment of the present application. cc is the relative position between the camera and the conveyor belt.

[0059] A2: Determine the minimum relative position between the camera and the robotic arm device based on the movement speed of the conveyor belt, the first duration, and the second duration.

[0060] Based on the moving speed of the conveyor belt, the first duration, and the second duration, it is possible to determine the distance that the target object is transported by the conveyor belt from the moment when the camera detects the target object to the moment when the robotic arm device performs the control operation. The distance between the camera and the robotic arm device needs to be greater than or equal to the distance determined based on the moving speed of the conveyor belt, the first duration, and the second duration, to ensure that the robotic arm device has enough time to perform the control operation after the camera detects the target object.

[0061] As an example, see formula (2):

[0062] vc(t0+t)≤T cr (2)

[0063] T cr is the relative position between the camera and the robotic arm device. vc is the moving speed of the conveyor belt. t0 is the duration for which the camera can detect the object, i.e., the first duration. t is the duration between the moment when the target object leaves the detection range of the camera and the moment when the robotic arm device performs the control operation, i.e., the second duration.

[0064] A3: Based on the moving range of the robotic arm device, determine the first maximum relative position between the conveyor belt and the robotic arm device.

[0065] The relative position between the conveyor belt and the robot arm device needs to be within the moving range of the robot arm device, so as to ensure that the robot arm device can control the target object transported by the conveyor belt.

[0066] As an example, see formula (3):

[0067] T rc <c(3)

[0068] T rc is the relative position between the conveyor belt and the robot device. C is the travel range of the robot device. It should be noted that the travel range of the robot device is the travel range in the direction from the robot device to the conveyor belt, and the relative position between the conveyor belt and the robot device is also the relative position in this direction. A specific direction is, for example, a direction horizontal to the transmission direction of the conveyor belt.

[0069] A4: Determine the second maximum relative position between the conveyor belt and the robotic arm device based on the moving speed of the robotic arm device, the first time duration, and the second time duration.

[0070] The relative position between the robot device and the conveyor belt cannot be too large to prevent the robot device from being unable to move to the target control position in time. Based on the movement speed of the robot device, the first time duration and the second time duration, the maximum movement distance of the robot device within the first time duration and the second time duration can be determined. Then, based on the movement distance, the second maximum relative position between the conveyor belt and the robot device can be determined.

[0071] As an example, see formula (4):

[0072]

[0073] vr is the speed of the robot arm device in the direction from the conveyor belt.

[0074] The installation positions of the robot arm device, the position detection device and the object transmission device are determined according to the adjustment range of the relative positions among the robot arm device, the position detection device and the object transmission device.

[0075] S202: Determine a target control position of the robotic arm device.

[0076] The target control position is the target position of the target object controlled by the robot device. The target object is the object that needs to be controlled when the robot device performs the control operation.

[0077] The embodiment of the present application does not limit the implementation method of determining the target control position of the manipulator device. In one possible implementation method, the target control position is a preset position of the manipulator device to control the object. The target control position is determined based on the preset device parameters of the manipulator device. In another possible implementation method, the target control position is a position determined based on the position of the target object. The movement data of the target object during the transmission process and the position of the target object are obtained, and the position of the object that can be controlled by the manipulator device is determined by calculation, that is, the target control position. The movement data, for example, includes data such as the moving speed of the target object, the distance between the target object and the manipulator device. The movement data can be detected by a position detection device, or determined based on system parameters of the object transmission device, such as the transmission parameters of the object transmission device. The position of the target object can be detected by a position detection device. As an example, the position detection device is an image acquisition device, such as a camera. The position detection device detects the position of the target object before the target object moves to the control range of the manipulator device. For example, when it is detected that the target object enters the detection range of the position detection device, an image of the target object is collected and uploaded to the management device. After acquiring the image, the management device performs image recognition on the image, determines the position of the target object on the object transmission device, and calculates the target control position according to the moving speed of the object transmission device and the relative position between the camera and the robotic arm device. As another example, the management device acquires a user-triggered instruction to determine the target control position. In response to the instruction to determine the target control position, the management device calculates the target control position based on the position of the target object generated by the position detection device.

[0078] S203: Acquire the actual control position of the robotic arm device.

[0079] The robot arm device performs a control operation on the target object based on the target control position. The control operation is, for example, a grabbing operation, a moving operation, a placing operation, etc. For example, the robot arm device includes a gripper device. The robot arm device first moves based on the target control position, and then uses the gripper device to perform a grabbing operation on the target object.

[0080] The actual position of the target object controlled by the robotic arm device is the actual control position.

[0081] The embodiments of the present application do not limit the method for obtaining the actual control position. As an example, the position detection device can also collect the actual position of the robotic arm device. The position detection device detects the actual position of the robotic arm device in real time. The management device determines the moment when the robotic arm device performs the control operation and the position at which the robotic arm device is located based on the actual position of the robotic arm device in real time, that is, the actual position of the target object controlled by the robotic arm device. As another example, the robotic arm device includes a position sensor. The position sensor is used to detect the actual position of the target object controlled by the robotic arm device when the robotic arm device performs the control operation.

[0082] S204: Determine the operation result of the robotic arm device controlling the target object at the actual control position. If the operation result meets the adjustment conditions, determine the adjustment position of any device included in the object control system based on the target control position, the actual control position and the installation position of any device included in the object control system.

[0083] The operation result is the result of the robot arm device performing the control operation at the actual control position to control the target object. The operation result is used to indicate whether the robot arm device successfully controls the target object at the actual control position.

[0084] The embodiments of the present application do not limit the specific content of the operation results.

[0085] In a possible implementation, the operation result is a control result of whether the robotic arm device successfully controls the target object. As an example, the operation result is generated by the robotic arm device or by the position detection device. For example, the robotic arm device includes a force control sensor. Based on the force control sensor, it can be determined whether the robotic arm device successfully controls the target object. For another example, the position detection device can capture an image of the target object, and based on the image of the target object, it can be determined whether the robotic arm device successfully controls the target object. The management device obtains the operation result generated and sent by the robotic arm device or the position detection device.

[0086] In another possible implementation, the operation result is a position deviation between the target control position and the actual control position. As an example, the operation result is determined by the management device based on the acquired target control position and the actual control position. In a possible implementation, both the target control position and the actual control position are represented by coordinates. If the target control position and the actual control position are represented by coordinates of different coordinate systems, the coordinates of the target control position and the coordinates of the actual control position are first converted to coordinates in the same coordinate system. The difference between the coordinates of the target control position and the coordinates of the actual control position is calculated to obtain the position deviation between the target control position and the actual control position.

[0087] The position deviation between the target control position and the actual control position can reflect the accuracy of the robot device controlling the target object. For example, when the position deviation is greater than the deviation threshold, it means that when the robot device performs a control operation, the target control position is far away from the actual control position, and the probability of the robot device successfully controlling the target object is low. When the position deviation is less than or equal to the deviation threshold, it means that when the robot device performs a control operation, the target control position is close to the actual control position, or the same, and the probability of the robot device successfully controlling the target object is high.

[0088] Determine whether the operation result satisfies the adjustment condition. The adjustment condition is a pre-set condition that requires adjustment of the installation position of any device included in the object control system. Each device included in the object control system, that is, the robot arm device, the position detection device, and the object transmission device, has a pre-set installation position. The installation position affects the operation effect of the object control system, that is, the operation result of controlling the object, and affects the accuracy of the object control system in controlling the object.

[0089] The embodiments of the present application do not limit the specific content of the adjustment conditions.

[0090] As an example, the adjustment condition is that the position deviation is greater than or equal to the position deviation threshold. The position deviation threshold is determined based on the control range of the robotic arm device. For example, the control operation is a grasping operation, and the control range of the robotic arm device is, for example, the grasping range of a grasping device included in the robotic arm device. The grasping device included in the robotic arm device is, for example, a gripper. The position deviation threshold is a pre-set threshold for determining whether the position adjustment of the object control system is required. If the position deviation is greater than or equal to the position deviation threshold, it means that the accuracy of the robotic arm device controlling the object may not meet the requirements for controlling the object, and the installation position of any device included in the object control system needs to be adjusted. The position deviation threshold can be set based on the need to adjust the installation position of any device of the object control system.

[0091] In another possible implementation, the adjustment condition further includes the number of operation results of the position deviation being greater than or equal to the position deviation threshold, which is greater than or equal to the number threshold.

[0092] During the operation of the object control system, there may be many uncontrollable factors that cause the target control position of the robot arm device to deviate from the actual control position. The number threshold is set in advance. The number threshold is a threshold used to determine whether it is necessary to adjust the installation position of any device included in the object control system. The number threshold can be set based on the need to adjust the installation position of any device of the object control system.

[0093] If the number of operation results of the position deviation greater than or equal to the position deviation threshold is greater than or equal to the number threshold, it means that the control deviation has occurred multiple times, the object control system may have a fault, and the object control system needs to adjust the installation position of the equipment. In this way, it is possible to avoid too frequent adjustment of the installation position of the equipment included in the object control system while ensuring that the fault can be handled, thereby reducing the adjustment cost.

[0094] It should be noted that the embodiment of the present application does not limit the method of triggering the judgment of whether the operation result meets the adjustment condition. In one possible implementation, the operation result is determined in real time, and it is judged whether the operation result meets the adjustment condition. In this way, real-time detection of the object control system can be achieved, and existing faults can be discovered in time. In another possible implementation, a detection time period is pre-set. Within this time period, the operation result is determined, and it is judged whether the operation result meets the adjustment condition.

[0095] If the operation result satisfies the adjustment condition, the adjustment position of any device included in the object control system is determined based on the target control position, the actual control position, and the installation position of any device included in the object control system. The adjustment position of any device included in the object control system is used to adjust the installation position of any device included in the object control system. Specifically, the adjustment position of any device included in the object control system is used to adjust the installation position of one or more devices among the robot arm device, the position detection device, and the object transmission device.

[0096] In one possible implementation, the adjusted position is the adjusted installation position of the device. In another possible implementation, the adjusted position is the relative position between the current installation position of the device that needs to adjust the installation position and the adjusted installation position.

[0097] The embodiments of the present application do not limit the implementation method of determining the adjustment position of any device included in the object control system based on the target control position, the actual control position, and the installation position of any device included in the object control system.

[0098] In one possible implementation, an artificial intelligence model for determining the adjustment position of any device included in the object control system is pre-trained. The artificial intelligence model can be trained based on training data. The training data includes, for example, a target control position, an actual control position, an installation position of any device included in the object control system, and a label. The label is the adjustment position of any device included in the object control system. The label can be data generated based on the adjustment process of the historical object control system, or empirical data of manually adjusting the object control system.

[0099] The target control position, the actual control position and the installation position of any device included in the object control system are input into the trained artificial intelligence model to obtain the adjustment position of any device included in the object control system output by the artificial intelligence model.

[0100] In another possible implementation, the adjustment range of the relative position between the manipulator device, the position detection device and the object transmission device is determined based on the initial parameters of the manipulator device, the position detection device and the object transmission device. The adjustment range is used to determine the range of the relative position between the devices when the object control system can operate normally. When the relative position between the manipulator device, the position detection device and the object transmission device after adjustment based on the adjustment position is within the adjustment range, and the difference between the target control position and the actual control position meets the conditions, the adjustment position of any device included in the object control system is determined.

[0101] As an example, take the above Figure 3 Taking the structure of the object control system shown as an example, the position detection device is a camera and the object transmission device is a conveyor belt.

[0102] The embodiment of the present application provides a possible implementation method for determining the adjustment range of the relative position between the robot arm device, the position detection device and the object transmission device based on the initial parameters of the robot arm device, the position detection device and the object transmission device, including the following four steps:

[0103] B1: Determine the minimum relative position between the camera and the conveyor belt based on the camera's field of view and the width of the conveyor belt.

[0104] The camera's field of view determines the camera's detection range of the target object. The distance between the camera and the conveyor belt affects the camera's detection range of the conveyor belt. The detection range included in the camera's field of view must at least include the entire width of the conveyor belt to ensure that the camera can detect the target object. According to the camera's field of view and the width of the conveyor belt, determine the minimum relative position between the camera and the conveyor belt.

[0105] As an example, see formula (5):

[0106]

[0107] fov is the field of view of the camera. W is the width of the conveyor belt. Figure 4 , Figure 4 A schematic diagram of the relative positions of devices included in an object control system provided in an embodiment of the present application. cc is the relative position between the camera and the conveyor belt that needs to be adjusted. cc is the relative position between the camera and the current installation position of the conveyor.

[0108] B2: Determine the minimum relative position between the camera and the robotic arm device based on the movement speed of the conveyor belt, the first duration, and the second duration.

[0109] Based on the moving speed of the conveyor belt, the first duration, and the second duration, it is possible to determine the distance that the target object is transported by the conveyor belt from the moment when the camera detects the target object to the moment when the robotic arm device performs the control operation. The distance between the camera and the robotic arm device needs to be greater than or equal to the distance determined based on the moving speed of the conveyor belt, the first duration, and the second duration, to ensure that the robotic arm device has enough time to perform the control operation after the camera detects the target object.

[0110] As an example, see formula (6):

[0111] vc(t0+t)≤T cr +Δ cr (6)

[0112] T cr is the relative position between the camera and the current installation position of the robotic arm device. Δ cr is the relative position between the camera and the robotic arm device that needs to be adjusted. vc is the speed of the conveyor belt. t0 is the duration for which the camera can detect the object, that is, the first duration. t is the duration between the moment when the target object leaves the detection range of the camera and the moment when the robotic arm device performs the control operation, that is, the second duration.

[0113] B3: Based on the moving range of the robotic arm device, determine the first maximum relative position between the conveyor belt and the robotic arm device.

[0114] The relative position between the conveyor belt and the robot arm device needs to be within the moving range of the robot arm device, so as to ensure that the robot arm device can control the target object transported by the conveyor belt.

[0115] As an example, see formula (7):

[0116] T rc +Δ rc <c(7)

[0117] T rc is the relative position between the conveyor belt and the current installation position of the robotic arm device. Δ rc is the relative position between the conveyor belt and the robot device that needs to be adjusted. C is the travel range of the robot device. It should be noted that the travel range of the robot device is the travel range in the direction from the robot device to the conveyor belt, and the relative position between the conveyor belt and the robot device is also the relative position in this direction. A specific direction is, for example, a direction horizontal to the transmission direction of the conveyor belt.

[0118] B4: Determine the second maximum relative position between the conveyor belt and the robotic arm device based on the moving speed of the robotic arm device, the first duration, and the second duration.

[0119] The relative position between the robot device and the conveyor belt cannot be too large to prevent the robot device from being unable to move to the target control position in time. Based on the movement speed of the robot device, the first time duration and the second time duration, the maximum movement distance of the robot device within the first time duration and the second time duration can be determined. Then, based on the movement distance, the second maximum relative position between the conveyor belt and the robot device can be determined.

[0120] As an example, see formula (8):

[0121]

[0122] vr is the speed of the robot arm device in the direction from the conveyor belt.

[0123] Based on the determined adjustment range, an adjustment position of any device included in the object control system is determined so that the difference between the target control position and the actual control position satisfies a condition.

[0124] The condition is, for example, less than a difference threshold. The difference threshold is a preset value. The difference threshold can be determined based on the maximum acceptable difference between the target control position and the actual control position. Another condition is, for example, that the difference between the target control position and the actual control position is the smallest.

[0125] In a possible implementation, the difference between the target control position and the actual control position is determined by formula (9).

[0126]

[0127] (x, y, z) is the coordinate of the target control position. (x0, y0, z0) is the coordinate of the actual control position. (x, y, z) and (x0, y0, z0) belong to the same coordinate system.

[0128] Based on the adjustment range of the above formulas (5)-(9) and the condition that the difference between the target control position and the actual control position is minimized, determine the Δ that minimizes the value of f(x, y, z) rc , Δ cr and Δ cc It should be noted that based on the Δ rc , Δ cr or Δ cc , can determine the equipment that needs to adjust the installation position.

[0129] It should be noted that in some possible implementation scenarios, the adjustment condition is that the number of operation results of the position deviation being greater than or equal to the position deviation threshold is greater than or equal to the number threshold.

[0130] In a possible implementation, a statistical position of a plurality of actual control positions whose position deviations are greater than or equal to a position deviation threshold is determined. The statistical position is obtained by calculating a statistical value of a plurality of actual control positions whose position deviations are greater than or equal to the position deviation threshold. The statistical value is, for example, an average value or a median value. The statistical position can more accurately reflect the actual control position and can reduce the adverse effects of extreme actual control positions on the accuracy of determining the adjustment position.

[0131] By using the statistical position as the actual control position for determining the adjustment position of any device included in the object control system, the obtained adjustment position of any device included in the object control system is more accurate.

[0132] Based on the above S201-S204, it can be known that when the operation result meets the adjustment condition, the adjustment position of any device included in the object control system is determined, so that the installation position of any device included in the object control system can be adjusted by using the adjustment position, thereby improving the success rate of the subsequent object control task of the object control system and realizing closed-loop control of the object control system. The operation method of the object control system provided in the embodiment of the present application can be applied to a variety of object control scenarios and has good versatility.

[0133] The determined adjustment position of any device included in the object control system is used to indicate the adjustment of the installation position of any device included in the object control system. Specifically, during the adjustment process, it may be that only the robot arm device is adjusted, or only the position detection device is adjusted, or only the object transmission device is adjusted; it may also be that the position of the robot arm device remains unchanged, and the relative position of the position detection device and the object transmission device is adjusted, or the position of the position detection device remains unchanged, and the relative position of the robot arm device and the object transmission device is adjusted, or the position of the object transmission device remains unchanged, and the relative position of the robot arm device and the position detection device is adjusted, or the relative position of the robot arm device, the position detection device and the object transmission device are adjusted. It may also be that the relative position among the robot arm device, the position detection device and the object transmission device is adjusted. The embodiments of the present application do not limit the implementation method of adjusting the installation position of the device based on the adjustment position of any device included in the object control system.

[0134] In a possible implementation, the object control system includes a position control device. The position control device is used to control the installation position of the device included in the object control system. The position control device includes, for example, a movable control between the manipulator device and the guide rail, a movable control between the position detection device and the guide rail, and a movable control between the object transmission device and the guide rail. The position control device is also, for example, a mobile base of the manipulator device, a mobile base of the position detection device, and a mobile base of the object transmission device.

[0135] The management device sends an adjustment instruction to the position control device of the object control system. The adjustment instruction is generated based on the adjustment position. As an example, the adjustment instruction includes the adjustment position. As another example, the adjustment instruction includes one or more information of the device identification, adjustment direction, and adjustment distance of the device to be adjusted. The device to be adjusted, the adjustment direction, and the adjustment distance can be determined based on the adjustment position. The device to be adjusted is any device included in the object control system whose installation position needs to be adjusted. The position control device adjusts the installation position of the device included in the object control system based on the adjustment instruction. In this way, the automatic position adjustment of the installation position of the device included in the object control system can be achieved, the adjustment efficiency can be improved, and the success rate and stability of the object control system in controlling the object can be improved.

[0136] In another possible implementation, the object control system includes a display device. The management device sends first adjustment information based on the adjustment position to the display device. The display device can display the adjustment position so that the user can adjust the relative position of the object control system based on the first adjustment information displayed by the display device. As an example, the first adjustment information includes the adjustment position. As another example, the first adjustment information includes one or more information of a device identification of the device to be adjusted, an adjustment direction, and an adjustment distance. The device to be adjusted, the adjustment direction, and the adjustment distance can be determined based on the adjustment position.

[0137] Furthermore, the management device also obtains the real-time relative position of the object control system. The management device sends second adjustment information to the display device so that the user can adjust the relative position of the object control system based on the second adjustment information displayed by the display device. The second adjustment information is determined based on the adjustment position and the real-time relative position. As an example, the second adjustment information includes one or more of the device identification, adjustment direction, and adjustment distance of the device to be adjusted. The device to be adjusted, the adjustment direction, and the adjustment distance can be determined based on the adjustment position and the real-time relative position. In this way, the adjustment information can be determined in a timely manner based on the adjustment of the relative position of the object control system, so as to guide the user to adjust the relative position of the object control system.

[0138] In addition, safety issues may arise during the operation of the object control system. For example, the robotic arm device may collide with other devices, or move to other areas where safety issues may arise. The embodiment of the present application also provides an operation method of the object control system, which, in addition to the above steps, also includes the following steps:

[0139] If it is determined that the target control position is not within the safe position range, a safety prompt message is sent to the display device.

[0140] The safe position range is a safe position range for the robot device to control an object. The safe position range can be determined based on the operating range of the robot device. As an example, based on the test result data of the robot device, the limit range of movement of the robot device is determined. The limit range is used as the safe position range. As another example, based on the system parameters of the robot device, the movement range of the robot device under normal working conditions is determined. The movement range under normal working conditions is used as the safe position range.

[0141] If it is determined that the target control position is not within the safe position range, it means that the robot arm device may exceed the safe range during operation and there is an operation risk. Send a safety prompt message to the display device. The safety prompt message is used to remind the object control system that there is an operation risk. Specifically, the safety prompt message includes the target control position of the robot arm device. This makes it easier for users who view the safety prompt message to identify the problem in a timely manner.

[0142] Furthermore, a stop operation instruction is sent to the object control system. In this way, when an operation risk of the object control system is detected, the object control system can be stopped in time to avoid more serious operation problems and improve the ability of the object control system to operate safely.

[0143] The present application also provides an operating device of an object control system, the device is used to manage the object control system, the object control system includes a mechanical arm device, a position detection device and an object transmission device, the object transmission device is used to transmit the target object, the mechanical arm device is used to control the target object during the process of the object transmission device transmitting the target object, and the position detection device is used to detect the position of the target object. Figure 5 As shown, the device comprises:

[0144] A calculation module 501 is used to calculate the installation positions of the robotic arm device, the position detection device and the object transmission device based on the initial parameters of the robotic arm device, the position detection device and the object transmission device;

[0145] A first determining module 502 is used to determine a target control position of the robotic arm device, where the target control position is a target position of the robotic arm device controlling the target object;

[0146] An acquisition module 503 is used to acquire an actual control position of the robotic arm device, where the actual control position is an actual position of the robotic arm device controlling the target object;

[0147] The second determination module 504 is used to determine the operation result of the robotic arm device controlling the target object at the actual control position. If the operation result meets the adjustment condition, the adjustment position of any device included in the object control system is determined based on the target control position, the actual control position and the installation position of any device included in the object control system. The adjustment position is used to indicate the adjustment of the installation position of any device included in the object control system.

[0148] In a possible implementation, the second determining module 504 is configured to determine the adjustment position of any device included in the object control system based on the target control position, the actual control position, and the installation position of any device included in the object control system, including:

[0149] The second determination module 504 is used to determine the adjustment range of the relative position between the robotic arm device, the position detection device and the object transmission device based on the initial parameters of the robotic arm device, the position detection device and the object transmission device; determine the adjustment position of any device among the robotic arm device, the position detection device and the object transmission device, so that the relative position between the robotic arm device, the position detection device and the object transmission device after adjustment based on the adjustment position is within the adjustment range, and the difference between the target control position and the actual control position satisfies a condition.

[0150] In a possible implementation, the position detection device is a camera, the object transmission device is a conveyor belt, and the initial parameters of the robotic arm device, the position detection device, and the object transmission device include:

[0151] The camera's field of view angle, the conveyor belt's movement speed, the conveyor belt's width, the robotic arm's movement speed, the robotic arm's movement range, a first duration and a second duration, wherein the first duration is the duration from the moment the target object appears within the camera's detection range to the moment the target object disappears from the camera's detection range, and the second duration is the duration from the moment the target object disappears from the camera's detection range to the moment the robotic arm's movement performs a control operation on the target object.

[0152] In a possible implementation, the second determining module 504 is configured to determine the adjustment position of any device included in the object control system based on the target control position, the actual control position, and the installation position of any device included in the object control system, including:

[0153] The second determination module 504 is used to determine the minimum relative position between the camera and the conveyor belt based on the field of view of the camera and the width of the conveyor belt; determine the minimum relative position between the camera and the robotic arm device based on the movement speed of the conveyor belt, the first time length and the second time length; determine the first maximum relative position between the conveyor belt and the robotic arm device based on the moving range of the robotic arm device; determine the second maximum relative position between the conveyor belt and the robotic arm device based on the moving speed of the robotic arm device, the first time length and the second time length.

[0154] In a possible implementation, the operation result is a position deviation between the target control position and the actual control position, the adjustment condition is that the position deviation is greater than or equal to a position deviation threshold, and the position deviation threshold is determined based on a control range of the robotic arm device.

[0155] In a possible implementation, the adjustment condition further includes the number of control operations of the position deviation being greater than or equal to the position deviation threshold, which is greater than or equal to the number threshold.

[0156] Among them, the calculation module 501, the first determination module 502, the acquisition module 503 and the second determination module 504 can all be implemented by software, or can be implemented by hardware. Exemplarily, the implementation of the calculation module 501 is introduced below by taking the calculation module 501 as an example. Similarly, the implementation of the first determination module 502, the acquisition module 503 and the second determination module 504 can refer to the implementation of the calculation module 501.

[0157] As an example of a software functional unit, the computing module 501 may include code running on a computing instance. Among them, the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the above-mentioned computing instance may be one or more. For example, the computing module 501 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region (region) or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including a data center or multiple data centers with similar geographical locations. Among them, usually a region may include multiple AZs.

[0158] Similarly, multiple hosts / virtual machines / containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.

[0159] As an example of a hardware functional unit, the computing module 501 may include at least one computing device, such as a server, etc. Alternatively, the computing module 501 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0160] The multiple computing devices included in the computing module 501 can be distributed in the same region or in different regions. The multiple computing devices included in the computing module 501 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the computing module 501 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0161] It should be noted that, in other embodiments, the computing module 501 can be used to execute any step in the operating method of the object control system, the first determining module 502 can be used to execute any step in the operating method of the object control system, the acquiring module 503 can be used to execute any step in the operating method of the object control system, and the second determining module 504 can be used to execute any step in the operating method of the object control system. The steps that the computing module 501, the first determining module 502, the acquiring module 503, and the second determining module 504 are responsible for implementing can be specified as needed. The computing module 501, the first determining module 502, the acquiring module 503, and the second determining module 504 respectively implement different steps in the operating method of the object control system to realize all the functions of the operating device of the object control system.

[0162] The present application also provides a computing device 600. Figure 6 As shown, computing device 600 includes: bus 602, processor 604, memory 606 and communication interface 608. Processor 604, memory 606 and communication interface 608 communicate through bus 602. Computing device 600 can be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in computing device 600.

[0163] The bus 602 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 The bus 602 may include a path for transmitting information between various components of the computing device 600 (eg, the memory 606, the processor 604, and the communication interface 608).

[0164] The processor 604 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0165] The memory 606 may include a volatile memory, such as a random access memory (RAM). The processor 604 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0166] The memory 606 stores executable program codes, and the processor 604 executes the executable program codes to respectively implement the functions of the aforementioned calculation module 501, the first determination module 502, the acquisition module 503, and the second determination module 504, thereby implementing the operation method of the object control system. That is, the memory 606 stores instructions for executing the operation method of the object control system.

[0167] The communication interface 608 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 600 and other devices or communication networks.

[0168] The embodiment of the present application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.

[0169] like Figure 7 As shown, the computing device cluster includes at least one computing device 600. The memory 606 in one or more computing devices 600 in the computing device cluster may store the same instructions for executing the operation method of the object control system.

[0170] In some possible implementations, the memory 606 of one or more computing devices 600 in the computing device cluster may also store partial instructions for executing the operating method of the object control system. In other words, the combination of one or more computing devices 600 may jointly execute instructions for executing the operating method of the object control system.

[0171] It should be noted that the memory 606 in different computing devices 600 in the computing device cluster may store different instructions, which are respectively used to execute part of the functions of the operating device of the object control system. That is, the instructions stored in the memory 606 in different computing devices 600 may implement the functions of one or more modules among the computing module 501, the first determining module 502, the obtaining module 503 and the second determining module 504.

[0172] In some possible implementations, one or more computing devices in the computing device cluster may be connected via a network, which may be a wide area network or a local area network. Figure 8 A possible implementation is shown. Figure 8 As shown, two computing devices 600A and 600B are connected via a network. Specifically, the network is connected via a communication interface in each computing device. In this type of possible implementation, the memory 606 in the computing device 600A stores instructions for executing the functions of the computing module 501. At the same time, the memory 606 in the computing device 600B stores instructions for executing the functions of the first determining module 502, the obtaining module 503, and the second determining module 504.

[0173] Figure 7 The connection method between the computing device clusters shown can be based on the needs of the operation method of the object control system provided in the present application, such as the need for computing data, and therefore consider handing over the functions implemented by the first determination module 502, the acquisition module 503 and the second determination module 504 to the computing device 600B for execution.

[0174] It should be understood that Figure 7 The functions of the computing device 600A shown in FIG. 6A may also be completed by multiple computing devices 600. Similarly, the functions of the computing device 600B may also be completed by multiple computing devices 600.

[0175] The embodiment of the present application also provides a computer program product including instructions. The computer program product may be a software or program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes the method for operating the object control system.

[0176] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to execute the operation method of the object control system.

[0177] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for operating an object control system, characterized in that: The method is used to manage the object control system, the object control system includes a mechanical arm device, a position detection device and an object transmission device, the object transmission device is used to transmit the target object, the mechanical arm device is used to control the target object during the process of the object transmission device transmitting the target object, the position detection device is used to detect the position of the target object, and the method includes: Calculating installation positions of the robotic arm device, the position detection device, and the object transmission device based on initial parameters of the robotic arm device, the position detection device, and the object transmission device; Determine a target control position of the robotic arm device, where the target control position is a target position of the robotic arm device controlling the target object; Acquire an actual control position of the robotic arm device, where the actual control position is an actual position of the robotic arm device controlling the target object; Determine the operation result of the robotic arm device controlling the target object at the actual control position. If the operation result meets the adjustment condition, determine the adjustment position of any device included in the object control system based on the target control position, the actual control position and the installation position of any device included in the object control system. The adjustment position is used to indicate the adjustment of the installation position of any device included in the object control system.

2. The method according to claim 1, characterized in that The determining the adjustment position of any device included in the object control system based on the target control position, the actual control position and the installation position of any device included in the object control system comprises: Determine the adjustment range of the relative positions between the robot arm device, the position detection device and the object transmission device based on the initial parameters of the robot arm device, the position detection device and the object transmission device; Determine the adjustment position of any device among the robotic arm device, the position detection device and the object transmission device, so that the relative positions among the robotic arm device, the position detection device and the object transmission device after adjustment based on the adjustment position are within the adjustment range, and the difference between the target control position and the actual control position satisfies the condition.

3. The method according to claim 2, characterized in that The position detection device is a camera, the object transmission device is a conveyor belt, and the initial parameters of the robotic arm device, the position detection device and the object transmission device include: The camera's field of view angle, the conveyor belt's movement speed, the conveyor belt's width, the robotic arm's movement speed, the robotic arm's movement range, a first duration and a second duration, wherein the first duration is the duration from the moment the target object appears within the camera's detection range to the moment the target object disappears from the camera's detection range, and the second duration is the duration from the moment the target object disappears from the camera's detection range to the moment the robotic arm's movement performs a control operation on the target object.

4. The method according to claim 3, characterized in that The determining the adjustment position of any device included in the object control system based on the target control position, the actual control position and the installation position of any device included in the object control system comprises: Determining a minimum relative position between the camera and the conveyor belt based on the field of view of the camera and the width of the conveyor belt; Determine a minimum relative position between the camera and the robotic arm device based on the movement speed of the conveyor belt, the first time duration, and the second time duration; Determining a first maximum relative position between the conveyor belt and the robotic arm device based on a moving range of the robotic arm device; Based on the moving speed of the robot arm device, the first time period, and the second time period, a second maximum relative position between the conveyor belt and the robot arm device is determined.

5. The method according to claim 1, characterized in that The operation result is a position deviation between the target control position and the actual control position, and the adjustment condition is that the position deviation is greater than or equal to a position deviation threshold, and the position deviation threshold is determined based on a control range of the robotic arm device.

6. The method according to claim 5, characterized in that The adjustment condition also includes the number of control operations of the position deviation being greater than or equal to the position deviation threshold, which is greater than or equal to the number threshold.

7. An operating device for an object control system, characterized in that: The device is used to manage the object control system, the object control system includes a mechanical arm device, a position detection device and an object transmission device, the object transmission device is used to transmit the target object, the mechanical arm device is used to control the target object during the process of the object transmission device transmitting the target object, the position detection device is used to detect the position of the target object, and the device includes: A calculation module, used to calculate the installation positions of the robotic arm device, the position detection device and the object transmission device based on the initial parameters of the robotic arm device, the position detection device and the object transmission device; A first determining module is used to determine a target control position of the robotic arm device, where the target control position is a target position of the robotic arm device controlling the target object; An acquisition module, used for acquiring an actual control position of the robotic arm device, where the actual control position is an actual position where the robotic arm device controls the target object; The second determination module is used to determine the operation result of the robotic arm device controlling the target object at the actual control position. If the operation result meets the adjustment condition, the adjustment position of any device included in the object control system is determined based on the target control position, the actual control position and the installation position of any device included in the object control system. The adjustment position is used to indicate the adjustment of the installation position of any device included in the object control system.

8. The device according to claim 7, characterized in that The second determination module, used to determine the adjustment position of any device included in the object control system based on the target control position, the actual control position and the installation position of any device included in the object control system, includes: The second determination module is used to determine the adjustment range of the relative position between the robotic arm device, the position detection device and the object transmission device based on the initial parameters of the robotic arm device, the position detection device and the object transmission device; determine the adjustment position of any device among the robotic arm device, the position detection device and the object transmission device, so that the relative position between the robotic arm device, the position detection device and the object transmission device after adjustment based on the adjustment position is within the adjustment range, and the difference between the target control position and the actual control position satisfies a condition.

9. The device according to claim 8, characterized in that The position detection device is a camera, the object transmission device is a conveyor belt, and the initial parameters of the robotic arm device, the position detection device and the object transmission device include: The camera's field of view angle, the conveyor belt's movement speed, the conveyor belt's width, the robotic arm's movement speed, the robotic arm's movement range, a first duration and a second duration, wherein the first duration is the duration from the moment the target object appears within the camera's detection range to the moment the target object disappears from the camera's detection range, and the second duration is the duration from the moment the target object disappears from the camera's detection range to the moment the robotic arm's movement performs a control operation on the target object.

10. The device according to claim 9, characterized in that The second determination module, used to determine the adjustment position of any device included in the object control system based on the target control position, the actual control position and the installation position of any device included in the object control system, includes: The second determination module is used to determine the minimum relative position between the camera and the conveyor belt based on the field of view of the camera and the width of the conveyor belt; determine the minimum relative position between the camera and the robotic arm device based on the movement speed of the conveyor belt, the first time length and the second time length; determine the first maximum relative position between the conveyor belt and the robotic arm device based on the moving range of the robotic arm device; determine the second maximum relative position between the conveyor belt and the robotic arm device based on the moving speed of the robotic arm device, the first time length and the second time length.

11. The device according to claim 7, characterized in that The operation result is a position deviation between the target control position and the actual control position, and the adjustment condition is that the position deviation is greater than or equal to a position deviation threshold, and the position deviation threshold is determined based on a control range of the robotic arm device.

12. The device according to claim 11, characterized in that The adjustment condition also includes the number of control operations of the position deviation being greater than or equal to the position deviation threshold, which is greater than or equal to the number threshold.

13. A computing device cluster, characterized in that: comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method according to any one of claims 1 to 6.

14. A computer-readable storage medium, characterized in that: The method comprises computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster performs the method according to any one of claims 1 to 6.

15. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device cluster, the computing device cluster executes the method according to any one of claims 1 to 6.

Images