Multi-mobile-machine cooperative control method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a multi-mobile-machine cooperative control method and device, electronic equipment and a storage medium, and the method comprises the steps: firstly, responding to a cooperative control request of at least two mobile machines, controlling a plurality of signal transmitters to transmit a calibration signal to each mobile machine, and obtaining the receiving time of each mobile machine for receiving the calibration signal, the mobile machine and the plurality of signal transmitters are arranged in the working area; then, acquiring a sensor position coordinate of each signal transmitter in an area coordinate system of the working area, and respectively calculating to obtain a real-time conversion relation between a machine coordinate system of the mobile machine and the area coordinate system based on the plurality of sensor position coordinates and the receiving time; and finally, the mobile machines are controlled to perform cooperative operation based on a plurality of real-time conversion relationships, so that the reliability and efficiency of cooperative production work are remarkably improved, and the flexible and intelligent production requirements are better met.

Description

Technical Field

[0001] The present application relates to the field of control technology, and in particular to a method, device, electronic device and storage medium for collaborative control of multiple mobile machines. Background Art

[0002] With the increasing demand for efficiency in industrial production and the development of advanced industrial technologies, a large number of automated, semi-autonomous, or autonomous machines, as well as intelligent machines, have emerged on production sites. These include mobile machine collaboration systems, advanced CNC manufacturing machines, fixed-arm robots, automated guided vehicles (AGVs), and mobile robots. Furthermore, with the advancement of artificial intelligence, the participation of humanoid robots in production activities is a growing trend. With so many automated devices operating simultaneously in the same space, ensuring close and efficient coordination between them is crucial for improving production efficiency and product quality.

[0003] Traditional industrial production systems typically employ a fixed-station processing model, where the relative positions of various machines are simple and fixed. Collaborative control of multiple machines requires only simple positional logic conversion of control signals based on predetermined relative positions to achieve collaborative tasks. However, with the addition of various automated equipment, particularly those with autonomous mobility and collaborative capabilities, the relative positions of various devices change dynamically, as does the process logic between them. This has made collaborative production complex and difficult, making it difficult to control multiple mobile machines in real time to coordinate their work on the same workstation when collaborating on production. This results in low reliability in collaborative work. Summary of the Invention

[0004] The embodiments of the present application provide a method, device, electronic device, and storage medium for collaborative control of multiple mobile machines, which can improve the reliability of collaborative control of multiple mobile machine devices.

[0005] To achieve the above objectives, a first aspect of an embodiment of the present application provides a method for cooperative control of multiple mobile machines, the method comprising:

[0006] In response to a cooperative control request from at least two mobile machines, controlling a plurality of signal transmitters to transmit a calibration signal to each of the mobile machines, and obtaining a reception time of the calibration signal by each of the mobile machines, wherein the mobile machines and the plurality of signal transmitters are all within a working area;

[0007] Obtaining sensor position coordinates of each signal transmitter within the regional coordinate system of the working area, and calculating a real-time conversion relationship between a machine coordinate system of the mobile machine and the regional coordinate system based on the plurality of sensor position coordinates and the reception time;

[0008] The mobile machines are controlled to perform coordinated operations based on the multiple real-time conversion relationships.

[0009] In some embodiments, the calculating, based on the plurality of sensor position coordinates and the reception time, respectively obtaining a real-time conversion relationship between the machine coordinate system of the mobile machine and the regional coordinate system includes:

[0010] Obtaining a coordinate difference distance based on a difference between each of the sensor position coordinates and each of the coordinate parameters of the machine coordinate system;

[0011] Obtaining a signal transmission distance based on a product of a transmission rate of the calibration signal and the receiving time;

[0012] generating a distance equivalence function between each of the mobile machines and each of the signal transmitters based on the equivalence relationship between the coordinate difference distance and the signal transmission distance;

[0013] A relationship is solved based on the distance equivalence function to obtain a real-time conversion relationship between the machine coordinate system of each mobile machine and the regional coordinate system.

[0014] In some embodiments, solving the relationship based on the distance equivalence function to obtain the real-time conversion relationship between the machine coordinate system of each mobile machine and the regional coordinate system includes:

[0015] Combining all of the distance equivalent functions to obtain an overflow distance equation system;

[0016] generating a coordinate difference function group based on all the coordinate difference distances in the overflow distance equation group;

[0017] generating a signal distance function group based on all the signal transmission distances in the overflow distance equation group;

[0018] generating a minimum distance difference function based on a difference between the signal distance function group and the coordinate difference function group;

[0019] The minimum distance difference function is solved to obtain the real-time conversion relationship.

[0020] In some embodiments, solving the minimum distance difference function to obtain the real-time conversion relationship includes:

[0021] Calculating the Jacobian matrix corresponding to the minimum distance difference function;

[0022] Obtaining a damping factor, and generating an iterative update function based on the coordinate parameters, the Jacobian matrix, and the damping factor;

[0023] The damped least square method is used to perform multiple update iterations based on an iterative update function, and the real-time conversion relationship is obtained based on the iterative update function after the update iterations.

[0024] In some embodiments, generating an iterative update function based on the coordinate parameters, the Jacobian matrix, and the damping factor includes:

[0025] Based on the signal distance function group, obtaining the iterative signal distance corresponding to the current iterative parameter, and based on the coordinate difference function group, obtaining the iterative coordinate difference corresponding to the current iterative parameter;

[0026] The iteration factor is obtained by multiplying the difference between the iteration coordinate difference and the iteration signal distance by the Jacobian matrix;

[0027] A damping matrix is obtained by multiplying the Jacobian matrix by the transpose of the Jacobian matrix and adding the product of the damping factor and the identity matrix;

[0028] Based on the product of the inverse matrix of the damping matrix and the iterative coordinate difference, and adding the coordinate parameter, the iterative update function corresponding to updating the coordinate parameter is obtained.

[0029] In some embodiments, the sensor position coordinates include a first sensor coordinate parameter, a second sensor coordinate parameter, and a third sensor coordinate parameter, the coordinate parameters include the first coordinate parameter, the second coordinate parameter, and the third coordinate parameter, and obtaining the coordinate difference distance based on the difference between each of the sensor position coordinates and each of the coordinate parameters of the machine coordinate system includes:

[0030] Based on the difference between the first sensor coordinate parameter and the first coordinate parameter, square processing is performed to obtain a first coordinate difference;

[0031] Based on the difference between the second sensor coordinate parameter and the second coordinate parameter, square processing is performed to obtain a second coordinate difference;

[0032] Based on the difference between the third sensor coordinate parameter and the third coordinate parameter, a square process is performed to obtain a third coordinate difference;

[0033] The first coordinate difference, the second coordinate difference, and the third coordinate difference are accumulated to obtain the coordinate difference distance.

[0034] In some embodiments, the mobile machine includes a first machine and a second machine, and controlling the mobile machines to perform coordinated operations based on the multiple real-time conversion relationships includes:

[0035] performing a first coordinate transformation based on the control position area coordinates corresponding to the collaborative operation in the regional coordinate system and a first transformation relationship to obtain first machine control coordinates of the collaborative control operation in a first machine coordinate system, where the first machine coordinate system is the machine coordinate system of the first machine, and the first transformation relationship is the real-time transformation relationship between the first machine coordinate system and the regional coordinate system;

[0036] performing a second coordinate transformation based on the control position area coordinates corresponding to the collaborative operation in the regional coordinate system and a second transformation relationship to obtain second machine control coordinates of the collaborative control operation in a second machine coordinate system, where the second machine coordinate system is the machine coordinate system of the second machine, and the second transformation relationship is the real-time transformation relationship between the second machine coordinate system and the regional coordinate system;

[0037] sending the first machine control coordinates to the first machine, so that the first machine performs the collaborative operation according to the first machine control coordinates;

[0038] The second machine control coordinates are sent to the second machine, so that the second machine performs the coordinated operation according to the second machine control coordinates.

[0039] In some embodiments, when the first machine and the second machine perform the coordinated operation, the method further includes:

[0040] When the first machine moves in position or rotation, the first conversion relationship is updated to obtain a first updated conversion relationship;

[0041] and / or, when the second machine moves in position or rotation, the second conversion relationship is updated to obtain a second updated conversion relationship;

[0042] Based on the first update conversion relationship and / or the second update conversion relationship, the first machine and the second machine are controlled to perform the coordinated operation.

[0043] To achieve the above-mentioned objectives, a second aspect of an embodiment of the present application provides a multi-mobile machine cooperative control device, the device comprising:

[0044] a response module, configured to control a plurality of signal transmitters to transmit a calibration signal to each of the mobile machines in response to a cooperative control request from at least two mobile machines, and obtain a reception time of the calibration signal by each of the mobile machines, wherein the mobile machines and the plurality of signal transmitters are all within a working area;

[0045] a conversion relationship calculation module, configured to obtain sensor position coordinates of each signal transmitter in the regional coordinate system of the working area, and calculate, based on the plurality of sensor position coordinates and the reception time, a real-time conversion relationship between the machine coordinate system of the mobile machine and the regional coordinate system;

[0046] A collaborative control module is used to control the mobile machines to perform collaborative operations based on a plurality of the real-time conversion relationships.

[0047] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, the memory stores a computer program, and the processor implements the multi-mobile machine collaborative control method as described in the first aspect when executing the computer program.

[0048] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a storage medium, which is a computer-readable storage medium. The storage medium stores a computer program, and when the computer program is executed by a processor, it implements the multi-mobile machine collaborative control method described in the first aspect above.

[0049] The embodiments of the present application propose a method, device, electronic device and storage medium for collaborative control of multiple mobile machines. The method includes: first, in response to a collaborative control request from at least two mobile machines, controlling multiple signal transmitters to send a calibration signal to each mobile machine, and obtaining the reception time of the calibration signal for each mobile machine, wherein the mobile machines and the multiple signal transmitters are all in a working area; then, obtaining the sensor position coordinates of each signal transmitter in the regional coordinate system of the working area, and calculating the real-time conversion relationship between the machine coordinate system and the regional coordinate system of the mobile machine based on the multiple sensor position coordinates and the reception time; finally, controlling the mobile machines to perform collaborative operations based on the multiple real-time conversion relationships. The embodiment of the present application deploys multiple signal transmitters at fixed positions in the work area, and uses the time difference of each mobile machine receiving the signal to accurately calculate the real-time conversion relationship of the machine coordinate system of each mobile machine relative to the unified regional coordinate system in the work area, thereby converting the processing position coordinates corresponding to the collaborative operation in the regional coordinate system to the machine coordinate system of each mobile machine for collaborative control. Compared with the traditional collaborative method based on fixed workstations, this method can adapt to the dynamic changes in the position of mobile machine equipment. It does not need to pre-set fixed relative position relationships and complex signal logic conversions, and can achieve precise collaborative operation of multiple mobile machines in a dynamic environment, significantly improving the reliability and efficiency of collaborative production work, and thus better adapting to flexible and intelligent production needs.

[0050] Other features and advantages of the present application will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present application. The purposes and other advantages of the present application can be achieved and obtained through the structures particularly pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 This is a structural diagram of a mobile machine collaboration system provided in one embodiment of the present application.

[0052] Figure 2 This is a flowchart of a multi-mobile machine collaborative control method provided by another embodiment of the present application.

[0053] Figure 3 yes Figure 2 Flowchart of step 202 in FIG.

[0054] Figure 4 yes Figure 3 Flowchart of step 301 in FIG.

[0055] Figure 5 yes Figure 3 Flowchart of step 304 in FIG.

[0056] Figure 6 yes Figure 5 Flowchart of step 505 in FIG.

[0057] Figure 7 yes Figure 6 Flowchart of step 602 in FIG.

[0058] Figure 8 This is a schematic diagram of a real-time conversion relationship between a machine coordinate system and a regional coordinate system provided in another embodiment of the present application.

[0059] Figure 9 yes Figure 2 Flowchart of step 203 in FIG.

[0060] Figure 10 This is a flowchart of updating the conversion relationship provided by another embodiment of the present application.

[0061] Figure 11 It is a structural diagram of a multi-mobile machine cooperative control device provided in one embodiment of the present application.

[0062] Figure 12 This is a schematic diagram of the hardware structure of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0064] It should be noted that although the functional modules are divided in the device schematic and the logical order is shown in the flowchart, in some cases, the steps shown or described can be performed in a different order than the module division in the device or the order in the flowchart.

[0065] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0066] With the increasing demand for efficiency in industrial production and the development of advanced industrial technologies, a large number of automated, semi-autonomous, or autonomous machines, as well as intelligent machines, have emerged on production sites. These include mobile machine collaboration systems, advanced CNC manufacturing machines, fixed-arm robots, automated guided vehicles (AGVs), and mobile robots. Furthermore, with the advancement of artificial intelligence, the participation of humanoid robots in production activities is a growing trend. With so many automated devices operating simultaneously in the same space, ensuring close and efficient coordination between them is crucial for improving production efficiency and product quality.

[0067] Traditional industrial production systems typically employ a fixed-station processing model, where the relative positions of various machines are simple and fixed. Collaborative control of multiple machines requires only simple positional logic conversion of control signals based on predetermined relative positions to achieve collaborative tasks. However, with the addition of various automated equipment, particularly those with autonomous mobility and collaborative capabilities, the relative positions of various devices change dynamically, as does the process logic between them. This has made collaborative production complex and difficult, making it difficult to control multiple mobile machines in real time to coordinate their work on the same workstation when collaborating on production. This results in low reliability in collaborative work.

[0068] In order to improve the reliability of collaborative control of multiple mobile machine equipment, the embodiment of the present application deploys multiple signal transmitters at fixed positions in the working area, and uses the time difference of each mobile machine receiving the signal to accurately calculate the real-time conversion relationship of the machine coordinate system of each mobile machine relative to the unified regional coordinate system in the working area, thereby converting the processing position coordinates corresponding to the collaborative operation in the regional coordinate system to the machine coordinate system of each mobile machine for collaborative control. Compared with the traditional collaborative method based on fixed workstations, this method can adapt to the dynamic changes in the position of mobile machine equipment. It does not need to pre-set fixed relative position relationships and complex signal logic conversions, and can achieve precise collaborative operation of multiple mobile machines in a dynamic environment, significantly improving the reliability and efficiency of collaborative production work, and thus better adapting to flexible and intelligent production needs.

[0069] In order to better illustrate the multi-mobile machine cooperative control method provided by the embodiment of the present application, this embodiment first describes a mobile machine cooperative system using the mover control method. Figure 1 As shown in FIG, it is a structural diagram of a mobile machine cooperative system provided by an embodiment of the present application. Figure 1 As shown in the figure, a multi-mobile machine cooperative working architecture based on a magnetic drive conveyor line is shown. In the working area of the mobile machine cooperative system, a magnetic drive conveyor system is set up. The magnetic drive conveyor system includes a magnetic drive conveyor line and multiple mobile machines (such as Figure 1 The system includes computer numerical control (CNC), mobile robotic arms, automated guided vehicles (AGVs), and humanoid robots (such as those shown in the figure). Multiple movers carrying workpieces operate on a magnetically driven conveyor line, and these mobile machines perform coordinated operations based on the processing requirements of the workpieces carried by the movers. In addition, multiple fixed signal transmitters are installed in the work area to send calibration signals to these mobile machines, facilitating the calculation of the conversion relationship between the regional coordinate system of the work area and the machine coordinate system of each mobile machine.

[0070] It is understandable that the multi-mobile machine collaborative control method provided in this application is not only applicable to magnetic drive conveying systems, but can also be applied to other collaborative work scenarios with multiple mobile machines, such as other production line scenarios, logistics warehousing scenarios, regional operation scenarios, etc. Among them, the regional coordinate system of the working area is usually fixed, while the machine coordinate system usually changes according to the movement of the mobile machine (including translation, vertical movement, and rotation). Therefore, the real-time conversion relationship between the regional coordinate system and the machine coordinate system of each mobile machine is also changing.

[0071] It is understandable that the magnetic drive conveying system, as a machine device for transferring workpiece positions, has been introduced into the production system. Its significance lies not only in achieving higher efficiency, precision, and flexibility than traditional conveying systems, but also in that it makes the conveying system a machine system rather than a mechanical system. Like robots, CNC machines and other equipment, it has higher-level control, execution, processing, analysis, and collaboration capabilities, and can perform complex process production tasks. Since the conveying system has a wide coverage in the production space, a large amount of processing and production is basically arranged and carried out around the conveying system. Therefore, the collaboration between the magnetic drive conveying system and the machine-side equipment is extremely important for leveraging the advantages of the magnetic drive conveying system and improving production efficiency.

[0072] Industrial production is usually located in a certain environmental space, and the basis for the collaboration of various machines and equipment is the determination of relative position relationships. The present application discloses a method and system for collaboration between magnetic drive equipment and machine-side equipment based on a shared coordinate system. By establishing an environmental Cartesian coordinate system as the basic coordinate system and an equipment Cartesian coordinate system as the machine coordinate system, each machine and equipment executes a production process based on its own machine coordinate system. In order to collaborate between machines and equipment, the machine coordinate system is calculated and mapped to the environmental coordinate system, and the position of each device in the environmental coordinate system is shared among the devices. When performing collaboration between machines and equipment, one solution uses a high-level main controller to control the participating mechanical equipment to perform defined actions in the environmental coordinate system to complete the collaboration task. Another solution uses real-time sharing of position coordinate systems and process information between participating machines and equipment to control, execute and adjust process actions to complete the collaboration task.

[0073] Machine collaboration in dynamic production environments. In traditional production environments, most machines are fixed and static, with the concept of static workstations. Machines work on stationary objects in fixed locations, without requiring particularly complex collaboration. To further improve production efficiency, production environments are becoming increasingly dynamic. This is driven by the addition of dynamic machines, such as AGVs, mobile articulated-arm robots, humanoid robots, overhead crane systems, and magnetic conveyor systems. These machines or their bodies can be flexibly moved within the production environment (e.g., AGVs and mobile robots), or they can carry objects while moving (e.g., magnetic conveyor systems). Furthermore, dynamic processes are emerging, and dynamic processing is a trend to improve efficiency, enabling the entire production process to continue uninterrupted. For example, while a magnetic conveyor system is transporting objects, AGVs can dynamically load and unload materials, mobile robots can dynamically perform welding and other processing operations, and inspection machines can dynamically perform inspections. The basic requirement of dynamic production is that all machines need to fully collaborate with each other, otherwise production cannot be carried out. This is impossible in the traditional production environment characterized by static processing.

[0074] Machine-to-machine collaboration in dynamic production environments is applicable to all production environments with mobile features, including but not limited to AGV carts, mobile articulated arm robots, other types of mobile robots, and magnetic drive conveying systems. Problems to be solved include collaboration and obstacle avoidance.

[0075] Collaboration between machines in a dynamic production environment is very difficult. One of the main reasons is that the environment is dynamically changing. The positions of machines such as AGV mobile carts, mobile robots, and humanoid robots in the production environment are constantly changing and unpredictable. The positions of objects are also dynamically changing, such as objects in transit in a magnetic drive conveying system. In order to achieve high-precision collaboration between machines, it is necessary to solve the problem of unpredictable positions of machines or objects in the environment, that is, to complete the positioning of the machine in the production environment. After positioning is completed, a conversion relationship between the machine and the environmental coordinate system is established, and the environmental coordinate system is used as a unified basic shared coordinate system. Further planning and control of collaborative tasks, as well as other high-level collaborative control tasks, can be carried out in the shared coordinate system based on this.

[0076] Positioning the machine in the environment and determining the conversion relationship between the machine coordinate system and the environment coordinate system are the core and basic issues of collaboration between machines in a dynamic production environment. Since each machine equipment is independently controlled and its position changes dynamically, positioning itself is difficult. The core of the present application is to provide a solution and method to solve the positioning problem for the collaboration problem in a dynamic production environment, and to provide a system solution on this basis, in which positioning the machine is the basis and key. The core idea of positioning machines in a dynamic production environment adopts an idea similar to that of the global positioning system (GPS, Beidou), establishes a spatial model through the principle of multi-point positioning, solves the conversion information such as the position of the machine in the environment coordinate system, determines the position of the machine coordinate system in the environment coordinate system, that is, the real-time conversion relationship), and uses the environment coordinate system as a shared unified coordinate system. Through conventional coordinate system transformation, collaborative task planning and control can be carried out.

[0077] The multi-mobile machine cooperative control method provided in the embodiment of the present application can be applied to Figure 1 The control processor shown in the figure can also be applied to servers and smart terminals connected to the mobile machine cooperative system, and can also be applied to mobile machines equipped with computing modules, etc. Based on the above mobile machine cooperative system, the multi-mobile machine cooperative control method in the embodiment of the present application will be described in detail below. Figure 2 , which is an optional flow chart of the multi-mobile machine cooperative control method provided in an embodiment of the present application, Figure 2 The method may include but is not limited to steps 201 to 203. It is also understood that this embodiment is for Figure 2The order of step 201 to step 203 is not specifically limited, and the order of steps can be adjusted or some steps can be reduced or added according to actual needs.

[0078] Step 201: In response to cooperative control requests from at least two mobile machines, control multiple signal transmitters to send calibration signals to each mobile machine, and obtain the reception time of the calibration signal received by each mobile machine.

[0079] Step 201 is described in detail below.

[0080] In some embodiments, based on Figure 1 In the mobile robot collaborative work system shown in FIG, when processing a workpiece carried by a certain mover requires dispatching at least two mobile machines to perform a precise material docking collaborative operation, the control processor receives this collaborative control request and sends a signal to multiple signal transmitters (such as Figure 1 The four signal transmitters (P1, P2, P3, and P4) shown in the figure send calibration signal instructions, requiring them to broadcast the calibration signal to the entire work area at a certain sending time (e.g., the sending time of the calibration signal sent by the P1 signal transmitter is T1, the sending time of the calibration signal sent by the P2 signal transmitter is T2, and so on). At the same time, the multiple mobile machines participating in this collaborative task receive the calibration signal and record the receiving time T of the calibration signal. This receiving time is then sent to the control processor, which then calculates the receiving time based on the difference between the receiving time and the sending time (e.g., the receiving time of the calibration signal received by the P1 signal transmitter is T-T1, the receiving time of the calibration signal received by the P2 signal transmitter is T-T2, and so on). The core of this process is to trigger the fixed transmitters to broadcast through unified instructions and accurately obtain the individual receiving time of these same batch of signals for each participating mobile machine, laying the foundation for the subsequent calculation of their precise position in the unified coordinate system.

[0081] Step 202: Obtain the sensor position coordinates of each signal transmitter in the regional coordinate system of the working area, and calculate the real-time conversion relationship between the machine coordinate system of the mobile machine and the regional coordinate system based on multiple sensor position coordinates and reception time.

[0082] Step 202 is described in detail below.

[0083] Furthermore, a fixed Cartesian coordinate system (i.e., regional coordinate system) WOS_XYZ is pre-established within the work area. This serves as the base reference coordinate system for the work area. Within this regional coordinate system, the sensor position coordinates of each signal transmitter are P1 (X1, Y1, Z1), P2 (X2, Y2, Z2), P3 (X3, Y3, Z3), and P4 (X4, Y4, Z4). Furthermore, assume that the machine position parameters of a mobile machine within the regional coordinate system are P(X, Y, Z), and that the machine coordinate system of the mobile machine is MOS_XYZ.

[0084] Next, based on the relationship between the machine position parameters and the sensor position coordinates of multiple signal transmitters, as well as the reception time, the real-time conversion relationship between the machine coordinate system and the regional coordinate system of each mobile machine can be obtained, as described below.

[0085] Reference Figure 3 , based on the position coordinates and receiving time of multiple sensors, respectively calculate the real-time conversion relationship between the machine coordinate system of the mobile machine and the regional coordinate system, including the following steps 301 to 304.

[0086] Step 301: Obtain a coordinate difference distance based on the difference between each sensor position coordinate and the coordinate parameters of each machine coordinate system.

[0087] Step 301 is described in detail below.

[0088] In some embodiments, the coordinate difference distance is first obtained based on the difference between each sensor position coordinate (such as P1 (X1, Y1, Z1), P2 (X2, Y2, Z2), P3 (X3, Y3, Z3), P4 (X4, Y4, Z4)) and the coordinate parameters of each machine coordinate system (such as P (X, Y, Z)), so as to facilitate the subsequent use of the coordinate difference distance to construct a set of equations corresponding to the real-time conversion relationship, as described below.

[0089] Reference Figure 4 , based on the difference between each sensor position coordinate and the coordinate parameters of each machine coordinate system, obtaining the coordinate difference distance includes the following steps 401 to 404.

[0090] Step 401: Based on the difference between the first sensor coordinate parameter and the first coordinate parameter, square processing is performed to obtain the first coordinate difference.

[0091] Step 402: Based on the difference between the second sensor coordinate parameter and the second coordinate parameter, square the difference to obtain a second coordinate difference.

[0092] Step 403: Based on the difference between the third sensor coordinate parameter and the third coordinate parameter, square processing is performed to obtain the third coordinate difference.

[0093] Step 404: Accumulate the first coordinate difference, the second coordinate difference, and the third coordinate difference to obtain a coordinate difference distance.

[0094] Steps 401 to 404 are described in detail below.

[0095] In some embodiments, each sensor position coordinate (such as P1 (X1, Y1, Z1)) typically includes a first sensor coordinate parameter (such as X1), a second sensor coordinate parameter (such as Y1), and a third sensor coordinate parameter (such as Z1), and the coordinate parameters include a first coordinate parameter (such as X), a second coordinate parameter (such as Y), and a third coordinate parameter (such as Z).

[0096] Based on this, the difference between the first sensor coordinate parameter and the first coordinate parameter is squared to obtain the first coordinate difference (X1-X) 2 , based on the difference between the second sensor coordinate parameter and the second coordinate parameter, square it and get the second coordinate difference (Y1-Y) 2 , based on the difference between the third sensor coordinate parameter and the third coordinate parameter, square it and get the third coordinate difference (Z1-Z) 2 ; Then accumulate the first coordinate difference, the second coordinate difference and the third coordinate difference to get the coordinate difference distance (X1-X) 2 +(Y1-Y) 2 +(Z1-Z) 2 .

[0097] In addition, the coordinate difference distance between the mobile machine and each signal generator is obtained in the above manner, that is, Figure 1 In the case of four signal generators shown in , there are multiple coordinate difference distances as shown in the following formula (1).

[0098]

[0099] Step 302: Obtain the signal transmission distance based on the product of the transmission rate of the calibration signal and the receiving time.

[0100] Step 303: Based on the equivalent relationship between the coordinate difference distance and the signal transmission distance, generate a distance equivalent function between each mobile machine and each signal transmitter.

[0101] Step 304: Solve the relationship based on the distance equivalence function to obtain the real-time conversion relationship between the machine coordinate system of each mobile machine and the regional coordinate system.

[0102] Steps 302 to 304 are described in detail below.

[0103] Next, based on the product of the transmission rate V of the calibration signal in the working area and the corresponding reception time of the mobile machine and each signal transmitter (such as (T-T1), etc.), the corresponding signal transmission distance between the mobile machine and each signal generator is obtained as shown in the following formula (2).

[0104]

[0105] Then, based on the equivalent relationship between the coordinate difference distance and the signal transmission distance, a distance equivalent function between each mobile machine and each signal transmitter is generated as shown in the following formula (3).

[0106] {(X1-X) 2 +(Y1-Y) 2 +(Z1-Z) 2 =(V(T-T1)) 2 (3)

[0107] Then, by solving the joint relationship of the distance equivalence function (3) between multiple signal transmitters and the mobile machine, the position (X, Y, Z) of the machine coordinate system of the mobile machine in the regional coordinate system can be obtained, thereby determining the real-time conversion relationship T between the machine coordinate system and the regional coordinate system.

[0108] It is understandable that positioning the machine and determining the position of the machine coordinate system in the environment are the basis and key to multi-device collaborative control. In particular, dynamic positioning of machines with a large range of motion is very difficult, such as the positioning of movers, mobile robots, and AGV carts on magnetic drive conveyor lines. The positioning transformation relationship T contains the necessary information to describe the precise position of the machine in the environmental coordinate system, typically the position of the machine in the environmental coordinate system (X / Y / Z), and may also include other information such as rotation information (RPY angle). The present invention focuses on processing position (X / Y / Z) information and has not yet processed rotation information.

[0109] It can be understood that the real-time transformation relationship T describes the transformation relationship between the machine coordinate system and the regional coordinate system, and contains the necessary information for transformation between the machine coordinate system and the environment coordinate system. The real-time transformation relationship T is a data structure concept. Simply put, when only considering the position, the real-time transformation relationship T contains the position (X, Y, Z) of the mobile machine in the regional coordinate system, which can be represented in the form of a vector or matrix to participate in the coordinate system transformation; when the rotation angle also needs to be considered, the real-time transformation relationship T also needs to include the rotation angle information.

[0110] How to solve the distance equivalence function will be further described below.

[0111] Reference Figure 5 , solving the relationship based on the distance equivalence function to obtain the real-time conversion relationship between the machine coordinate system of each mobile machine and the regional coordinate system, including the following steps 501 to 505.

[0112] Step 501: Combine all distance equivalent functions to obtain overflow distance equations.

[0113] Step 502: Generate a coordinate difference function group based on all coordinate difference distances in the overflow distance equation group.

[0114] Step 503: Generate a signal distance function group based on all signal transmission distances in the overflow distance equation group.

[0115] Step 504: Generate a minimum distance difference function based on the difference between the signal distance function group and the coordinate difference function group.

[0116] Step 505: Solve the minimum distance difference function to obtain a real-time conversion relationship.

[0117] Steps 501 to 505 are described in detail below.

[0118] In some embodiments, all distance equivalence functions (3) are first combined to obtain the overflow distance equation group as shown in the following formula (4).

[0119]

[0120] In this scheme, the damped least squares Levenberg-Marquardt algorithm is used for solution. Based on this algorithm, first, based on all coordinate difference distances in the overflow distance equation group, a coordinate difference function group is generated as shown in the following formula (5), and based on all signal transmission distances in the overflow distance equation group, a signal distance function group is generated as shown in the following formula (6).

[0121]

[0122]

[0123] Afterwards, based on the idea of least squares, the minimum distance difference function is generated based on the difference between the signal distance function group (6) and the coordinate difference function group (5) as shown in the following formula (7).

[0124] min‖SF(X,Y,Z)‖ 2 =minG(X,Y,Z) (7)

[0125] Finally, the minimum distance difference function (7) is solved to obtain the real-time conversion relationship T. The following will further describe how to solve the minimum distance difference function (7).

[0126] Reference Figure 6 , solving the relationship based on the distance equivalence function to obtain the real-time conversion relationship between the machine coordinate system of each mobile machine and the regional coordinate system, including the following steps 601 to 603.

[0127] Step 601: Calculate the Jacobian matrix corresponding to the minimum distance difference function.

[0128] Step 602: Obtain a damping factor, and generate an iterative update function based on the coordinate parameters, the Jacobian matrix, and the damping factor.

[0129] Steps 601 to 602 are described in detail below.

[0130] In some embodiments, after obtaining the minimum distance difference function (7), the Jacobian matrix corresponding to the minimum distance difference function (7) is first calculated as shown in the following formula (8).

[0131]

[0132] Next, an iterative update function is generated based on the predetermined damping factor u, coordinate parameters (X, Y, Z), and Jacobian matrix (8), so as to solve the overflow distance equation group (4) using the iterative update function. The following first further describes how to generate the iterative update function.

[0133] Reference Figure 7 , generating an iterative update function based on the coordinate parameters, the Jacobian matrix and the damping factor, including the following steps 701 to 704.

[0134] Step 701: Based on the signal distance function group, obtain the iterative signal distance corresponding to the current iterative parameter, and based on the coordinate difference function group, obtain the iterative coordinate difference corresponding to the current iterative parameter.

[0135] Step 702: The iteration factor is obtained by multiplying the difference between the iteration coordinate difference and the iteration signal distance by the Jacobian matrix.

[0136] Step 703: The damping matrix is obtained by multiplying the Jacobian matrix by the transpose of the Jacobian matrix and adding the product of the damping factor and the identity matrix.

[0137] Step 704: Based on the product of the inverse matrix of the damping matrix and the iterative coordinate difference, plus the coordinate parameter, an iterative update function corresponding to the updated coordinate parameter is obtained.

[0138] Steps 701 to 704 are described in detail below.

[0139] In some embodiments, first, in each iteration (such as the current iteration parameter i), based on the signal distance function group (6), the iterative signal distance S corresponding to the current iteration parameter i is obtained. i , and based on the coordinate difference function group (5), the iterative coordinate difference F corresponding to the current iteration parameter i is obtained i Then, based on the difference between the iterative coordinate difference and the iterative signal distance, multiply it by the transposed matrix of the Jacobian matrix to obtain the iteration factor w i =J T (S i -F i ).

[0140] After that, the transpose of the Jacobian matrix is multiplied by the Jacobian matrix, and the product of the damping factor and the identity matrix is added to obtain the damping matrix (J T J+uI) -1 ,Further, based on the product of the inverse matrix of the damping matrix and the iterative coordinate difference, and adding the coordinate parameters, the iterative update function corresponding to the updated coordinate parameters is obtained as shown in the following formula (9).

[0141] P i+1 =P i +(J T J+uI) -1 w i (9)

[0142] Step 603: Using the damped least squares method, perform multiple update iterations based on the iterative update function, and obtain a real-time conversion relationship based on the iterative update function after the update iterations.

[0143] Step 603 is described in detail below.

[0144] Finally, the iterative solution idea of the damped least squares method is used to perform multiple rapid update iterations based on the iterative update function (9), thereby obtaining the real-time analytical values of the coordinate parameters (X, Y, Z) based on the iterative update function after the update iteration, and obtaining the real-time conversion relationship T based on the analytical values of the coordinate parameters.

[0145] Through the above steps 401 to 404, 501 to 505, 601 to 603, and 701 to 704, by establishing the distance equivalence relationship between the physical position of the sensor and the signal propagation time, a physical and geometric foundation is laid for the pose solution, and the overflow equation group formed by multiple sensors is used, and the strategy of minimizing the difference between the measured distance and the geometric distance (especially using the accumulation of squared distance differences) is adopted. This makes the solution result able to effectively resist the measurement noise or error of a single sensor, significantly improves the robustness of positioning, and further clarifies the use of the damped least squares method (i.e., L Evenberg-Marquardt algorithm), through iterative optimization and Jacobian matrix calculation, can not only efficiently and stably solve complex nonlinear pose equations, but also ensure high precision and fast convergence of the solution, meeting the real-time requirements of industrial sites. These steps together ensure that the subsequent collaborative control can provide continuous, reliable and highly accurate real-time conversion relationship between the machine coordinate system of each mobile machine and the unified regional coordinate system. This is the key technical guarantee for realizing precise collaborative operation of multiple mobile machines in complex and dynamic environments, greatly improving the success rate, efficiency and operation accuracy of collaborative tasks.

[0146] Reference Figure 8 , is a schematic diagram of a real-time conversion relationship between a machine coordinate system and a regional coordinate system provided by an embodiment of the present application. Figure 8 As shown in , based on multiple signal transmitters (transmitter 1 to transmitter 4) with known positions (for example, coordinates Xn, Yn, Zn, and transmission time Tn), they constitute a reference base station under the regional coordinate system WOS. The mobile machine that needs to be positioned has its own machine coordinate system (MOS), and its position and posture in the regional coordinate system WOS (represented by the coordinates X, Y, Z and the real-time conversion relationship, simplified as X, Y, Z, T in the figure) are the goals to be solved, that is, it is necessary to calculate the real-time conversion relationship T of the machine coordinate system MOS relative to the regional coordinate system WOS. The mobile machine receives the calibration signal from each transmitter and records the reception time. Based on the signal transmission rate, transmission time (Tn) and reception time (T), the signal transmission distance from the mobile machine to each transmitter can be calculated (corresponding to Figure 8 At the same time, based on the desired coordinates (X, Y, Z) of the mobile machine and the known coordinates (Xn, Yn, Zn) of each signal transmitter, an expression for the coordinate difference distance can be established. By leveraging the equivalence between signal transmission distance and coordinate difference distance, and using data from multiple transmitters (four in the figure, forming an over-constrained set of equations), an optimization algorithm can be used to accurately determine the real-time transformation relationship T of the mobile machine.

[0147] After obtaining the real-time transformation relationship T between the machine coordinate system (MOS) of each mobile machine and the world coordinate system (WOS), the task trajectory F in the machine coordinate system is transformed into the environment coordinate system as shown in the following formula (10).

[0148] F wos =TF mos (10)

[0149] Similarly, the task logic in the regional coordinate system is converted to the machine coordinate system as shown in the following formula (11).

[0150] F mos =T -1 F wos (11)

[0151] Step 203: Control the mobile machines to perform coordinated operations based on the multiple real-time conversion relationships.

[0152] Step 203 is described in detail below.

[0153] In some embodiments, after obtaining the real-time conversion relationship between the robot coordinate system and the regional coordinate system of the mobile machines that need to work collaboratively, further transit processing will be performed based on these real-time conversion relationships, and the process paths corresponding to the collaborative operations will be converted into their respective machine coordinate systems, and controlled in their respective machine coordinate systems to improve the reliability of collaborative work of multiple mobile machines, as described below.

[0154] Reference Figure 9 , controlling mobile machines to perform collaborative operations based on multiple real-time conversion relationships, including the following steps 901 to 904.

[0155] Step 901: Perform a first coordinate transformation based on the control position area coordinates corresponding to the collaborative operation in the regional coordinate system and a first transformation relationship to obtain first machine control coordinates of the collaborative control operation in the first machine coordinate system.

[0156] Step 902: Perform a second coordinate transformation based on the control position area coordinates corresponding to the collaborative operation in the regional coordinate system and the second transformation relationship to obtain second machine control coordinates of the collaborative control operation in the second machine coordinate system.

[0157] Step 903: Send the first machine control coordinates to the first machine, so that the first machine performs the collaborative operation according to the first machine control coordinates.

[0158] Step 904: Send the second machine control coordinates to the second machine, so that the second machine performs the collaborative operation according to the second machine control coordinates.

[0159] Steps 901 to 904 are described in detail below.

[0160] In some implementations, within a certain work area, assume that the first machine (AGV1) and the second machine (AGV2) need to collaboratively lift a heavy steel plate and place it on a designated work surface. The host computer sends a collaborative control request, specifying the coordinates of the target placement position of the steel plate in the regional coordinate system as (X_target, Y_target, Z_target). The control processor first uses the real-time conversion relationship calculated previously to convert the target position coordinates (X_target, Y_target, Z_target) into the machine coordinate system of AGV1 and the machine coordinate system of AGV2 respectively. After the conversion, the control coordinates of AGV1 (X1_control, Y1_control, Z1_control) and the control coordinates of AGV2 (X2_control, Y2_control, Z2_control) are obtained. The control processor sends (X1_control, Y1_control, Z1_control) to AGV1, and AGV1 adjusts its own position and posture according to the coordinates so that it can accurately grasp one end of the steel plate. At the same time, the control processor sends (X2_control, Y2_control, Z2_control) to AGV2, which adjusts its position and posture based on these coordinates, allowing it to accurately grasp the other end of the steel plate. Finally, AGV1 and AGV2 work together to lift the steel plate and place it steadily on the target work surface.

[0161] In addition, suppose there are a first machine 1 and a second device 2 in the working area, and the real-time transformation relationship of the relative regional coordinate system is T1 and T2 respectively. The two machines collaborate to complete the production task, and the task definition in the first machine 1 is When collaborating, the two machines share device information and real-time conversion relationships through the control processor or directly with each other. Based on the regional coordinate system, the tasks in the machine coordinate system of the first machine 1 will be described first. Convert to regional coordinate system Next, we describe the task F in the regional coordinate system. wos Convert to the coordinate system of the second machine 2 Once the description of the collaborative task in the coordinate system of the second machine 2 is obtained, the second machine 2 can plan and execute it in its own machine coordinate system, and dynamically adjust and respond according to the status of the first machine 1 to achieve the purpose of collaboration with the first machine 1.

[0162] Through the above steps 901 to 904, the complex control position area coordinates that need to be precisely defined in each coordinate system are effectively decomposed and converted into machine control coordinates based on its own coordinate system that can be directly understood and executed by each participating mobile machine. This coordinate conversion based on real-time and precise conversion relationship ensures that although each machine performs actions independently according to its own coordinate system, their final behaviors can be accurately synchronized and aligned in the global regional coordinate system, thereby greatly improving the accuracy, reliability and success rate of collaborative operations of multiple mobile machines, so that complex collaborative tasks that were originally difficult to achieve and required high-precision coordination (such as precision assembly, synchronous handling of large objects, etc.) can be completed efficiently and stably in a dynamic, unstructured environment, overcoming the problem of collaborative failure caused by the accumulation of positioning errors of each machine or the inconsistency of reference systems.

[0163] Reference Figure 10 When the first machine and the second machine perform a coordinated operation, the multi-mobile machine coordinated control method further includes the following steps 1001 to 1003.

[0164] Step 1001: When the first machine moves in position or rotation, the first conversion relationship is updated to obtain a first updated conversion relationship.

[0165] Step 1002: and / or, when the second machine moves in position or rotation, the second conversion relationship is updated to obtain a second updated conversion relationship.

[0166] Step 1003: Based on the first update conversion relationship and / or the second update conversion relationship, control the first machine and the second machine to perform a coordinated operation.

[0167] Steps 1001 to 1003 are described in detail below.

[0168] In some embodiments, such as in a certain area, when the first machine AGV1 and the second machine AGV2 are performing collaborative operations of collaborative transportation, if AGV1 has a slight position offset (for example, moved 5 cm to the left) due to uneven ground or other reasons, the control processor will immediately detect the position change of AGV1. At this time, the control processor will recalculate the real-time conversion relationship between the machine coordinate system and the regional coordinate system of AGV1 to obtain the updated first conversion relationship (denoted as T1'). Then, based on T1', the control processor will recalculate the control instructions that need to be executed by AGV1 in the new position in order to continue to collaboratively lift the steel plate and place it at the target position. This new control instruction will be sent to AGV1, and AGV1 will adjust its own position and posture according to the new instruction to ensure that the collaborative operation with AGV2 can proceed smoothly. Similarly, if AGV2 also changes in position or posture, the control processor will also make corresponding updates and adjustments.

[0169] Through steps 1001 to 1003, the real-time monitoring and updating of the mobile machine's position and posture changes allows for dynamic adjustment of the control strategy to overcome coordination deviations caused by external interference, the device's own motion errors, and other factors, ensuring the accuracy and stability of collaborative operations. Even in the event of unexpected movement or rotation of the mobile machine, the system can quickly respond, recalculating coordinate transformations and adjusting control instructions to ensure the smooth completion of collaborative tasks, avoiding safety issues such as collisions and falls caused by positional deviations, and improving the reliability and safety of the entire system.

[0170] It should be noted that the above-mentioned multi-mobile machine collaborative control method is applicable to both mobile and stationary machines. For machines and equipment that are fixed relative to the production environment, a pre-calibrated method is preferred to improve accuracy and stability. During installation, a laser rangefinder or other means is used to accurately measure and calibrate the position in the environmental coordinate system, and then set the position in the machine and equipment. During production, the machine coordinate system is not positioned, only shared.

[0171] This solution describes the machine coordinate system under a unified regional environmental coordinate system. It can be used for positioning throughout the entire production environment, planning and controlling global collaborative tasks, including mutually visible and invisible machines. In terms of system composition, only the "machine coordinate system" needs to be installed on the mobile machine equipment. It can be understood that this "machine coordinate system" is used to provide the machine coordinate system for the mobile machine and to convert between the machine coordinate system and the regional coordinate system. Furthermore, there is no need to set up other equipment to achieve positioning accuracy and stability. Relying solely on the machine coordinate system is independent of the machines and equipment in the work area, and does not occupy the computing resources and complexity of the mobile machine itself. The overall system is simple and easy to expand. When a new mobile machine is added to the work area, only the "machine coordinate system" needs to be installed for rapid positioning. Due to the simplicity of the system, costs are lower.

[0172] The embodiments of the present application propose a method, device, electronic device and storage medium for collaborative control of multiple mobile machines. The method includes: first, in response to a collaborative control request of at least two mobile machines, controlling multiple signal transmitters to send a calibration signal to each mobile machine, and obtaining the reception time of the calibration signal for each mobile machine, wherein the mobile machines and the multiple signal transmitters are all in the working area; then, obtaining the sensor position coordinates of each signal transmitter in the regional coordinate system of the working area, obtaining the coordinate difference distance based on the difference between each sensor position coordinate and the coordinate parameters of each machine coordinate system, obtaining the signal transmission distance based on the product of the transmission rate of the calibration signal and the reception time, and obtaining the signal transmission distance based on the coordinate difference distance and the signal transmission distance. The equivalent relationship of the transmission distance is used to generate the distance equivalent function between each mobile machine and each signal transmitter, and all the distance equivalent functions are combined to obtain the overflow distance equation group. Based on all the coordinate difference distances in the overflow distance equation group, a coordinate difference function group is generated. Based on all the signal transmission distances in the overflow distance equation group, a signal distance function group is generated. Based on the difference between the signal distance function group and the coordinate difference function group, a minimum distance difference function is generated. The Jacobian matrix corresponding to the minimum distance difference function is calculated to obtain the damping factor. Based on the signal distance function group, the iterative signal distance corresponding to the current iteration parameter is obtained, and based on the coordinate difference function group, the iterative coordinate difference corresponding to the current iteration parameter is obtained. Based on the iterative coordinate difference and the iterative signal The difference in the distance between the two numbers is multiplied by the Jacobian matrix to obtain the iteration factor. The transpose of the Jacobian matrix is multiplied by the Jacobian matrix, and the product of the damping factor and the unit matrix is added to obtain the damping matrix. The product of the inverse matrix of the damping matrix and the iterative coordinate difference is added to the coordinate parameters to obtain the iterative update function corresponding to the updated coordinate parameters. The damped least squares method is used to perform multiple update iterations based on the iterative update function, and the real-time conversion relationship is obtained based on the iterative update function after the update iteration. Finally, the first coordinate transformation is performed based on the control position area coordinates corresponding to the collaborative operation in the regional coordinate system and the first conversion relationship to obtain the first machine control coordinates of the collaborative control operation in the first machine coordinate system. The first machine coordinate system is a machine coordinate system of the first machine, the first conversion relationship is a real-time conversion relationship between the first machine coordinate system and the regional coordinate system, a second coordinate conversion is performed based on the control position regional coordinates corresponding to the collaborative operation in the regional coordinate system and the second conversion relationship to obtain second machine control coordinates of the collaborative control operation in the second machine coordinate system, the second machine coordinate system is the machine coordinate system of the second machine, the second conversion relationship is a real-time conversion relationship between the second machine coordinate system and the regional coordinate system, the first machine control coordinates are sent to the first machine so that the first machine performs the collaborative operation according to the first machine control coordinates, and the second machine control coordinates are sent to the second machine so that the second machine performs the collaborative operation according to the second machine control coordinates;When the first machine and the second machine perform a coordinated operation, when the first machine moves positionally or rotationally, the first conversion relationship is updated to obtain a first updated conversion relationship, and / or when the second machine moves positionally or rotationally, the second conversion relationship is updated to obtain a second updated conversion relationship. Based on the first updated conversion relationship and / or the second updated conversion relationship, the first machine and the second machine are controlled to perform the coordinated operation.

[0173] The embodiment of the present application deploys multiple signal transmitters at fixed positions in the working area, and uses the time difference of each mobile machine receiving the signal to accurately calculate the real-time conversion relationship of the machine coordinate system of each mobile machine relative to the unified regional coordinate system in the working area, thereby converting the processing position coordinates corresponding to the collaborative operation in the regional coordinate system to the machine coordinate system of each mobile machine for collaborative control. Compared with the traditional collaborative method based on fixed workstations, this method can adapt to the dynamic changes in the position of mobile machine equipment. It does not need to pre-set a fixed relative position relationship and complex signal logic conversion, and can achieve precise collaborative operation of multiple mobile machines in a dynamic environment, significantly improving the reliability and efficiency of collaborative production work, and thus better adapting to flexible and intelligent production needs; in addition, by establishing a distance equivalence relationship between the physical position of the sensor and the signal propagation time, it lays a physical and geometric foundation for posture solution, and uses the overflow equation group formed by multiple sensors, and adopts the method of minimizing the difference between the measured distance and the geometric distance (especially using square The strategy of distance difference accumulation) makes the solution effective against the measurement noise or error of a single sensor, significantly improving the robustness of positioning. The damped least squares method (i.e., Levenberg-Marquardt algorithm) is further clarified. Through iterative optimization and Jacobian matrix calculation, it can not only efficiently and stably solve the complex nonlinear pose equations, but also ensure high accuracy and fast convergence of the solution, meeting the real-time requirements of industrial sites. These steps together ensure that the subsequent collaborative control can provide a continuous, reliable and highly accurate real-time transformation relationship between the machine coordinate system of each mobile machine and the unified regional coordinate system. This is the key technical guarantee for realizing the precise collaborative operation of multiple mobile machines in complex and dynamic environments, greatly improving the success rate, efficiency and operation accuracy of collaborative tasks. In addition, by monitoring and updating the position and attitude changes of mobile machines in real time, the control strategy can be dynamically adjusted to overcome the collaborative deviation caused by external interference, the device's own motion error and other factors, and ensure the accuracy and stability of collaborative operation.Even in the event of unexpected movement or rotation of a mobile machine, the system can react quickly, recalculate the coordinate transformation relationship, and adjust the control instructions to ensure the smooth completion of the collaborative task, avoid safety issues such as collisions and falls caused by position deviations, and improve the reliability and safety of the entire system. In addition, the complex control position area coordinates that need to be precisely defined in each coordinate system are effectively decomposed and converted into machine control coordinates based on its own coordinate system that each participating mobile machine can directly understand and execute. This coordinate transformation based on real-time and precise transformation relationships ensures that although each machine performs actions independently according to its own coordinate system, their final behaviors can be accurately synchronized and aligned in the global regional coordinate system, thereby greatly improving the accuracy, reliability and success rate of collaborative operations of multiple mobile machines. It enables complex collaborative tasks that were originally difficult to achieve and require high-precision coordination (such as precision assembly, synchronous handling of large objects, etc.) to be completed efficiently and stably in dynamic, unstructured environments, overcoming the problem of collaborative failure caused by the accumulation of positioning errors of each machine or inconsistent reference systems.

[0174] The embodiment of the present application also provides a multi-mobile machine cooperative control device, which can implement the multi-mobile machine cooperative control method described above, referring to Figure 11 , the apparatus 1100 comprises:

[0175] a response module 1110 configured to, in response to a collaborative control request from at least two mobile machines, control the plurality of signal transmitters to transmit a calibration signal to each mobile machine, and obtain a reception time of the calibration signal by each mobile machine, wherein the mobile machines and the plurality of signal transmitters are all within a working area;

[0176] A conversion relationship calculation module 1120 is configured to obtain the sensor position coordinates of each signal transmitter in the regional coordinate system of the working area, and calculate the real-time conversion relationship between the machine coordinate system of the mobile machine and the regional coordinate system based on the multiple sensor position coordinates and the reception time;

[0177] The collaborative control module 1130 is used to control the mobile machines to perform collaborative operations based on multiple real-time conversion relationships.

[0178] In some embodiments, the conversion relationship calculation module 1120 is further configured to:

[0179] Based on the difference between each sensor position coordinate and the coordinate parameters of each machine coordinate system, a coordinate difference distance is obtained;

[0180] The signal transmission distance is obtained based on the product of the transmission rate and the receiving time of the calibration signal;

[0181] Based on the equivalent relationship between the coordinate difference distance and the signal transmission distance, a distance equivalent function between each mobile machine and each signal transmitter is generated;

[0182] The relationship is solved based on the distance equivalence function to obtain the real-time conversion relationship between the machine coordinate system and the regional coordinate system of each mobile machine.

[0183] In some embodiments, the conversion relationship calculation module 1120 is further configured to:

[0184] Combining all distance equivalent functions yields the overflow distance equation system;

[0185] Generate a coordinate difference function group based on all coordinate difference distances in the overflow distance equation group;

[0186] Generate a signal distance function group based on all signal transmission distances in the overflow distance equation group;

[0187] Generate a minimum distance difference function based on the difference between the signal distance function group and the coordinate difference function group;

[0188] The minimum distance difference function is solved to obtain the real-time conversion relationship.

[0189] In some embodiments, the conversion relationship calculation module 1120 is further configured to:

[0190] Calculate the Jacobian matrix corresponding to the minimum distance difference function;

[0191] Get the damping factor and generate an iterative update function based on the coordinate parameters, Jacobian matrix and damping factor;

[0192] The damped least square method is used to perform multiple update iterations based on the iterative update function, and the real-time conversion relationship is obtained based on the iterative update function after the update iterations.

[0193] In some embodiments, the conversion relationship calculation module 1120 is further configured to:

[0194] Based on the signal distance function group, the iterative signal distance corresponding to the current iterative parameter is obtained, and based on the coordinate difference function group, the iterative coordinate difference corresponding to the current iterative parameter is obtained;

[0195] The iteration factor is obtained by multiplying the difference between the iterative coordinate difference and the iterative signal distance by the Jacobian matrix;

[0196] The damping matrix is obtained by multiplying the Jacobian matrix by the transpose of the Jacobian matrix and adding the product of the damping factor and the identity matrix;

[0197] Based on the product of the inverse matrix of the damping matrix and the iterative coordinate difference, and then adding the coordinate parameters, the iterative update function corresponding to the updated coordinate parameters is obtained.

[0198] In some embodiments, the conversion relationship calculation module 1120 is further configured to:

[0199] Based on the difference between the first sensor coordinate parameter and the first coordinate parameter, square processing is performed to obtain the first coordinate difference;

[0200] Based on the difference between the second sensor coordinate parameter and the second coordinate parameter, a square process is performed to obtain a second coordinate difference;

[0201] Based on the difference between the third sensor coordinate parameter and the third coordinate parameter, a square process is performed to obtain a third coordinate difference;

[0202] The first coordinate difference, the second coordinate difference, and the third coordinate difference are accumulated to obtain the coordinate difference distance.

[0203] In some embodiments, the collaborative control module 1130 is further configured to:

[0204] performing a first coordinate transformation based on the control position regional coordinates corresponding to the collaborative operation in the regional coordinate system and a first transformation relationship to obtain first machine control coordinates of the collaborative control operation in a first machine coordinate system, where the first machine coordinate system is a machine coordinate system of the first machine, and the first transformation relationship is a real-time transformation relationship between the first machine coordinate system and the regional coordinate system;

[0205] performing a second coordinate transformation based on the control position regional coordinates corresponding to the collaborative operation in the regional coordinate system and a second transformation relationship to obtain second machine control coordinates of the collaborative control operation in a second machine coordinate system, where the second machine coordinate system is a machine coordinate system of the second machine, and the second transformation relationship is a real-time transformation relationship between the second machine coordinate system and the regional coordinate system;

[0206] sending the first machine control coordinates to the first machine so that the first machine performs a coordinated operation according to the first machine control coordinates;

[0207] The second machine control coordinates are sent to the second machine, so that the second machine performs the coordinated operation according to the second machine control coordinates.

[0208] In some embodiments, the collaborative control module 1130 is further configured to:

[0209] When the first machine moves in position or rotation, the first conversion relationship is updated to obtain a first updated conversion relationship;

[0210] and / or, when the second machine moves in position or rotation, the second conversion relationship is updated to obtain a second updated conversion relationship;

[0211] Based on the first updated conversion relationship and / or the second updated conversion relationship, the first machine and the second machine are controlled to perform a coordinated operation.

[0212] In the above embodiments, the description of each embodiment has its own focus. For the parts that are not described in detail in a certain embodiment, the specific implementation of the multi-mobile machine cooperative control device is basically the same as the specific implementation of the above-mentioned multi-mobile machine cooperative control method, and will not be repeated here.

[0213] In the embodiment of the present application, the multi-mobile machine collaborative control device deploys multiple signal transmitters at fixed positions in the working area, and uses the time difference of each mobile machine receiving the signal to accurately calculate the real-time conversion relationship of the machine coordinate system of each mobile machine relative to the unified regional coordinate system in the working area, thereby converting the processing position coordinates corresponding to the collaborative operation in the regional coordinate system to the machine coordinate system of each mobile machine for collaborative control. Compared with the traditional collaborative method based on fixed workstations, this method can adapt to the dynamic changes in the position of mobile machine equipment. It does not need to pre-set a fixed relative position relationship and complex signal logic conversion, and can achieve precise collaborative operation of multiple mobile machines in a dynamic environment, significantly improving the reliability and efficiency of collaborative production work, and thus better adapting to flexible and intelligent production needs; in addition, by establishing a distance equivalence relationship between the physical position of the sensor and the signal propagation time, a physical and geometric foundation is laid for posture solution, and the overflow equation group formed by multiple sensors is used, and the difference between the measured distance and the geometric distance is minimized (especially In particular, the strategy of using squared distance difference accumulation is used, which makes the solution effectively resistant to the measurement noise or error of a single sensor, significantly improving the robustness of positioning. The damped least squares method (i.e., Levenberg-Marquardt algorithm) is further clarified. Through iterative optimization and Jacobian matrix calculation, it can not only efficiently and stably solve the complex nonlinear pose equations, but also ensure high accuracy and fast convergence of the solution, meeting the real-time requirements of industrial sites. These steps together ensure that the subsequent collaborative control can provide a continuous, reliable and highly accurate real-time transformation relationship between the machine coordinate system of each mobile machine and the unified regional coordinate system. This is the key technical guarantee for realizing the precise collaborative operation of multiple mobile machines in complex and dynamic environments, greatly improving the success rate, efficiency and operation accuracy of collaborative tasks. In addition, by monitoring and updating the position and posture changes of mobile machines in real time, the control strategy can be dynamically adjusted to overcome the collaborative deviation caused by external interference, the device's own motion error and other factors, and ensure the accuracy and stability of collaborative operations.Even in the event of unexpected movement or rotation of a mobile machine, the system can react quickly, recalculate the coordinate transformation relationship, and adjust the control instructions to ensure the smooth completion of the collaborative task, avoid safety issues such as collisions and falls caused by position deviations, and improve the reliability and safety of the entire system. In addition, the complex control position area coordinates that need to be precisely defined in each coordinate system are effectively decomposed and converted into machine control coordinates based on its own coordinate system that each participating mobile machine can directly understand and execute. This coordinate transformation based on real-time and precise transformation relationships ensures that although each machine performs actions independently according to its own coordinate system, their final behaviors can be accurately synchronized and aligned in the global regional coordinate system, thereby greatly improving the accuracy, reliability and success rate of collaborative operations of multiple mobile machines. It enables complex collaborative tasks that were originally difficult to achieve and require high-precision coordination (such as precision assembly, synchronous handling of large objects, etc.) to be completed efficiently and stably in dynamic, unstructured environments, overcoming the problem of collaborative failure caused by the accumulation of positioning errors of each machine or inconsistent reference systems.

[0214] An embodiment of the present application further provides an electronic device, including:

[0215] at least one memory;

[0216] at least one processor;

[0217] at least one program;

[0218] The program is stored in the memory, and the processor executes the at least one program to implement the multi-mobile machine cooperative control method implemented in this application. The electronic device can be any smart terminal including a mobile phone, a tablet computer, a personal digital assistant (PDA), an in-vehicle computer, etc.

[0219] See also Figure 12 , Figure 12 The hardware structure of an electronic device according to another embodiment is shown. The electronic device includes:

[0220] The processor 1201 can be implemented as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0221] The memory 1202 can be implemented in the form of ROM (Read Only Memory), static storage device, dynamic storage device, or RAM (Random Access Memory). The memory 1202 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1202 and is called by the processor 1201 to execute the multi-mobile machine cooperative control method of the embodiments of this application.

[0222] Input / output interface 1203, used to implement information input and output;

[0223] Communication interface 1204, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.);

[0224] Bus 1205 , which transmits information between various components of the device (e.g., processor 1201 , memory 1202 , input / output interface 1203 , and communication interface 1204 );

[0225] The processor 1201 , the memory 1202 , the input / output interface 1203 and the communication interface 1204 are connected to each other in communication within the device via the bus 1205 .

[0226] An embodiment of the present application further provides a storage medium, which is a computer-readable storage medium and stores a computer program. When the computer program is executed by a processor, the above-mentioned multi-mobile machine collaborative control method is implemented.

[0227] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0228] The embodiments described in the embodiments of this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0229] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0230] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0231] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0232] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0233] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0234] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the above-mentioned units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. The mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0235] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0236] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0237] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes multiple instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: various media that can store programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0238] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the present invention should be within the scope of the present invention.

Claims

1. A method for cooperative control of multiple mobile machines, characterized in that: The method comprises: In response to a cooperative control request from at least two mobile machines, controlling a plurality of signal transmitters to transmit a calibration signal to each of the mobile machines, and obtaining a reception time of the calibration signal by each of the mobile machines, wherein the mobile machines and the plurality of signal transmitters are all within a working area; Obtaining sensor position coordinates of each signal transmitter within the regional coordinate system of the working area, and calculating a real-time conversion relationship between a machine coordinate system of the mobile machine and the regional coordinate system based on the plurality of sensor position coordinates and the reception time; The mobile machines are controlled to perform coordinated operations based on the multiple real-time conversion relationships.

2. The method for cooperative control of multiple mobile devices according to claim 1, wherein: The calculating, based on the plurality of sensor position coordinates and the reception time, respectively obtaining a real-time conversion relationship between the machine coordinate system of the mobile machine and the regional coordinate system includes: Obtaining a coordinate difference distance based on a difference between each of the sensor position coordinates and each of the coordinate parameters of the machine coordinate system; Obtaining a signal transmission distance based on a product of a transmission rate of the calibration signal and the receiving time; generating a distance equivalence function between each of the mobile machines and each of the signal transmitters based on the equivalence relationship between the coordinate difference distance and the signal transmission distance; A relationship is solved based on the distance equivalence function to obtain a real-time conversion relationship between the machine coordinate system of each mobile machine and the regional coordinate system.

3. The method for cooperative control of multiple mobile devices according to claim 2, wherein: Solving the relationship based on the distance equivalence function to obtain a real-time conversion relationship between the machine coordinate system of each mobile machine and the regional coordinate system includes: Combining all of the distance equivalent functions to obtain an overflow distance equation system; generating a coordinate difference function group based on all the coordinate difference distances in the overflow distance equation group; generating a signal distance function group based on all the signal transmission distances in the overflow distance equation group; generating a minimum distance difference function based on a difference between the signal distance function group and the coordinate difference function group; The minimum distance difference function is solved to obtain the real-time conversion relationship.

4. The method for cooperative control of multiple mobile devices according to claim 3, wherein: Solving the minimum distance difference function to obtain the real-time conversion relationship includes: Calculating the Jacobian matrix corresponding to the minimum distance difference function; Obtaining a damping factor, and generating an iterative update function based on the coordinate parameters, the Jacobian matrix, and the damping factor; The damped least square method is used to perform multiple update iterations based on an iterative update function, and the real-time conversion relationship is obtained based on the iterative update function after the update iterations.

5. The method for cooperative control of multiple mobile devices according to claim 4, characterized in that: The generating an iterative update function based on the coordinate parameters, the Jacobian matrix and the damping factor includes: Based on the signal distance function group, obtaining the iterative signal distance corresponding to the current iterative parameter, and based on the coordinate difference function group, obtaining the iterative coordinate difference corresponding to the current iterative parameter; The iteration factor is obtained by multiplying the difference between the iteration coordinate difference and the iteration signal distance by the Jacobian matrix; A damping matrix is obtained by multiplying the Jacobian matrix by the transpose of the Jacobian matrix and adding the product of the damping factor and the identity matrix; Based on the product of the inverse matrix of the damping matrix and the iterative coordinate difference, and adding the coordinate parameter, the iterative update function corresponding to updating the coordinate parameter is obtained.

6. The method for cooperative control of multiple mobile devices according to claim 2, wherein: The sensor position coordinates include a first sensor coordinate parameter, a second sensor coordinate parameter, and a third sensor coordinate parameter, and the coordinate parameters include the first coordinate parameter, the second coordinate parameter, and the third coordinate parameter. The coordinate difference distance is obtained based on the difference between each of the sensor position coordinates and each of the coordinate parameters of the machine coordinate system, including: Based on the difference between the first sensor coordinate parameter and the first coordinate parameter, square processing is performed to obtain a first coordinate difference; Based on the difference between the second sensor coordinate parameter and the second coordinate parameter, square processing is performed to obtain a second coordinate difference; Based on the difference between the third sensor coordinate parameter and the third coordinate parameter, a square process is performed to obtain a third coordinate difference; The first coordinate difference, the second coordinate difference, and the third coordinate difference are accumulated to obtain the coordinate difference distance.

7. The method for cooperative control of multiple mobile devices according to claim 1, wherein: The mobile machine includes a first machine and a second machine, and controlling the mobile machines to perform coordinated operations based on the multiple real-time conversion relationships includes: performing a first coordinate transformation based on the control position area coordinates corresponding to the collaborative operation in the regional coordinate system and a first transformation relationship to obtain first machine control coordinates of the collaborative control operation in a first machine coordinate system, where the first machine coordinate system is the machine coordinate system of the first machine, and the first transformation relationship is the real-time transformation relationship between the first machine coordinate system and the regional coordinate system; performing a second coordinate transformation based on the control position area coordinates corresponding to the collaborative operation in the regional coordinate system and a second transformation relationship to obtain second machine control coordinates of the collaborative control operation in a second machine coordinate system, where the second machine coordinate system is the machine coordinate system of the second machine, and the second transformation relationship is the real-time transformation relationship between the second machine coordinate system and the regional coordinate system; sending the first machine control coordinates to the first machine, so that the first machine performs the collaborative operation according to the first machine control coordinates; The second machine control coordinates are sent to the second machine, so that the second machine performs the coordinated operation according to the second machine control coordinates.

8. The method for cooperative control of multiple mobile devices according to claim 7, wherein: When the first machine and the second machine perform the coordinated operation, the method further includes: When the first machine moves in position or rotation, the first conversion relationship is updated to obtain a first updated conversion relationship; and / or, when the second machine moves in position or rotation, the second conversion relationship is updated to obtain a second updated conversion relationship; Based on the first update conversion relationship and / or the second update conversion relationship, the first machine and the second machine are controlled to perform the coordinated operation.

9. A multi-mobile machine cooperative control device, characterized in that: The device comprises: a response module, configured to control a plurality of signal transmitters to transmit a calibration signal to each of the mobile machines in response to a cooperative control request from at least two mobile machines, and obtain a reception time of the calibration signal by each of the mobile machines, wherein the mobile machines and the plurality of signal transmitters are all within a working area; a conversion relationship calculation module, configured to obtain sensor position coordinates of each signal transmitter in the regional coordinate system of the working area, and calculate, based on the plurality of sensor position coordinates and the reception time, a real-time conversion relationship between the machine coordinate system of the mobile machine and the regional coordinate system; A collaborative control module is used to control the mobile machines to perform collaborative operations based on a plurality of the real-time conversion relationships.

10. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and is characterized in that when the processor executes the computer program, the method for cooperative control of multiple mobile machines as described in any one of claims 1 to 8 is implemented.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for cooperatively controlling multiple mobile machines according to any one of claims 1 to 8 is implemented.

Images