Outdoor multi-execution-mechanism cooperative control method and system

Abstract

The invention relates to a cooperative control method and system for multiple outdoor execution mechanisms, belongs to the technical field of cooperative control, and solves the problem of poor synchronism among the outdoor execution mechanisms in the prior art. The method comprises the following steps: taking a local or remote terminal with a main control authority as a main control terminal, selecting an execution mechanism as an execution node, and obtaining a to-be-executed instruction and a priority thereof according to a target position and data collected by a sensor of the execution node; creating tasks for the execution nodes according to the instructions to be executed and the priorities thereof, and adding the tasks into the task queue according to the priorities from high to low; and the main control terminal starts the task queue and sequentially issues the tasks in the task queue, and the execution nodes analyze to-be-executed instructions of the tasks by the corresponding execution controllers according to the task types, control the equipment body to execute corresponding actions and feed back execution results until the tasks in the task queue are completed. And the synchronism and the control precision of a plurality of execution mechanisms are improved.

Description

Technical Field

[0001] The present invention relates to the field of collaborative control technology, and in particular to an outdoor multi-actuator collaborative control method and system. Background Art

[0002] In some large-scale outdoor engineering scenarios, such as projects in the construction, petrochemical and other industries, multiple large mechanical mechanisms need to work together to complete multi-point synchronous lifting, hoisting, pushing and other operations of a single large object. In order to avoid tilting, rollover and other losses to personnel and property, it is necessary to ensure that large mechanical equipment moves synchronously during the entire operation process.

[0003] Collaborative control refers to a control method in which the various actuators in the system do not have a master-slave relationship, but coordinate to complete specific tasks through communication between the actuators. Synchronous action is the most common purpose of collaborative control, and it is difficult to achieve.

[0004] Existing common large-scale equipment, such as lifting equipment such as cranes and tower cranes, all achieve synchronization of actuators through manual operation. However, in work scenarios that require precision, manual operation has the disadvantages of large errors and poor synchronization of multi-point actions. Summary of the invention

[0005] In view of the above analysis, an embodiment of the present invention aims to provide a method and system for coordinated control of outdoor multiple actuators, so as to solve the problem of poor synchronization between existing outdoor actuators.

[0006] On the one hand, an embodiment of the present invention provides a collaborative control method for outdoor multiple actuators, each actuator includes a local terminal, a device body, a video device, multiple sensors and multiple actuator controllers, and the local terminals of the multiple actuators are connected to multiple remote terminals through a networking bus; the collaborative control method includes the following steps:

[0007] A local or remote terminal with master control authority acts as a master control terminal, selects an actuator as an execution node, and obtains instructions to be executed and their priorities based on the target location and data collected by sensors of the execution node;

[0008] Create tasks for execution nodes according to the instructions to be executed and their priorities, and add them to the task queue in descending order of priority;

[0009] The main control terminal starts the task queue and sends each task in the task queue in turn. The execution node parses the task's instructions to be executed according to the task type through the corresponding execution controller, controls the device body to perform the corresponding action and feedback the execution result until all tasks in the task queue are completed.

[0010] Based on the further improvement of the above method, when the instructions to be executed by multiple actuators are the same, a task with multiple execution nodes is created, and the device bodies of the multiple execution nodes execute the same action at the same time; when the instructions to be executed by multiple actuators are different, tasks with their own execution nodes are created respectively, added to the task queue corresponding to each execution node, and multiple task queues are started at the same time, and the device bodies of the multiple execution nodes execute different actions at the same time.

[0011] Based on the further improvement of the above method, if the target position is the final destination position, the instructions to be executed are obtained according to the data collected by the sensors of the target position and the execution node, including:

[0012] Get the final destination position as the target position;

[0013] According to the data collected by the sensor of the execution node, the position of the action execution component in the execution node is obtained and converted into the earth coordinate system to obtain the actual position;

[0014] According to the difference between the actual position and the target position, the distance difference in each direction in the geodetic coordinate system is obtained;

[0015] Create instructions to be executed in each direction, use the distance difference in the corresponding direction as the distance to be moved in the instructions to be executed, and obtain each instruction to be executed.

[0016] Based on the further improvement of the above method, if the target position is the position of the operation object obtained by identifying the video stream output by the video device of the execution node, the instruction to be executed is obtained according to the target position and the data collected by the sensor of the execution node, including:

[0017] S11, obtaining a target position according to the video stream identification output by the video device of the execution node;

[0018] S12, according to the data collected by the sensor of the execution node, obtain the position of the action execution component in the execution node, and convert it to the earth coordinate system to obtain the actual position;

[0019] S13, obtaining the distance difference in each direction in the geodetic coordinate system according to the difference between the actual position and the target position;

[0020] S14. Identify whether the distance difference in each direction is greater than a threshold value. If the distance difference is greater than the threshold value, create a to-be-executed instruction in the corresponding direction, take half of the distance difference in the corresponding direction as the distance to be moved in the to-be-executed instruction in that direction, and obtain each to-be-executed instruction; when the execution node completes each to-be-executed instruction, return to step S11; if the distance difference in each direction is less than the threshold value, end the acquisition operation of the to-be-executed instruction at the target position.

[0021] Based on the further improvement of the above method, the target position is obtained according to the video stream output by the video device of the execution node, including:

[0022] Extract each frame of image from the video stream, and use the trained neural network model to obtain the image with the target as the image to be identified;

[0023] After converting the image frame to be identified into a grayscale image, the edge of the target is determined using an edge detection algorithm, and then binarized to obtain an edge binary image;

[0024] A binocular ranging algorithm is used to obtain the distance between each pixel in the edge binary image and the video device of the execution node, and the minimum distance is taken as the vertical distance from the target to the video device of the execution node; the horizontal offset distance of the target is obtained according to the vertical distance and the parameters of the video device; the vertical distance and the horizontal offset distance are converted to the geodetic coordinate system to obtain the target position.

[0025] Based on the further improvement of the above method, the vertical distance and the parameters of the video device, the horizontal offset distance of the target is obtained by the following formula:

[0026] y=(z(W-2X h )-Bf) / 2f

[0027] Where y represents the horizontal offset distance, z represents the vertical distance, W represents the imaging length of the video device, B represents the center distance between the two infrared cameras in the video device, f represents the focal length of the video device, and X represents the vertical distance. h Indicates the smaller horizontal coordinate of the imaging point of the target on the two infrared cameras in the video device.

[0028] Based on the further improvement of the above method, when the main control terminal executes the corresponding action on the device body, it dynamically creates a new task and adds it to the started task queue according to the data collected by the sensor of the corresponding execution node. If the priority of the new task is lower than the priority of the currently executed task, the new task is added to the corresponding position after sorting from high to low priority. Otherwise, the currently executed task is suspended, and the new task is inserted into the first position of the task queue and issued. When the new task is completed, the suspended task will continue to be executed.

[0029] Based on the further improvement of the above method, the task also includes a timeout period and a timeout timer. The timeout timer is started each time a task is issued. If no execution result is received within the timeout period, the task is changed to a task timeout, an exception is thrown, and the execution node stops executing all actions.

[0030] Based on the further improvement of the above method, when the main control terminal receives an emergency stop command, it automatically creates a stop task with the highest priority, adds it to the first position of all task queues and sends it down, and the task being executed is automatically suspended. When the main control terminal receives a cancel emergency stop command, it destroys the stop tasks in all task queues, continues the subsequent tasks in the task queue, or ends all tasks and destroys the task queue.

[0031] On the other hand, an embodiment of the present invention provides an outdoor multi-actuator collaborative control system, including: multiple sets of actuators and multiple remote terminals, each set of actuators includes a local terminal, a device body, a video device, multiple sensors and multiple execution controllers, the local terminals of the multiple sets of actuators are connected to the multiple remote terminals through a networking bus; a local or remote terminal with master control authority serves as a master control terminal, and uses any of the above-mentioned collaborative control methods to collaboratively control the multiple sets of actuators.

[0032] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:

[0033] 1. Acquire the pending instructions of collaborative control through real-time data collection, use tasks to represent each pending action, add them to the task queue according to priority to manage a series of tasks, flexibly control the actions of multiple sets of actuators, ensure high synchronization of multiple sets of actuators, and greatly improve control accuracy.

[0034] 2. The control message frame is used to complete the issuance of control instructions and the execution status check, and the online mounting and removal of the local terminal or remote terminal of the actuator are supported; on-site video monitoring is achieved through the transmission of video streams, and intelligent control of each actuator is realized. It supports remote operation of on-site personnel, improves the safety of the operation process, and is more conducive to the overall control of the site for large-scale outdoor operations, reduces operating costs, and improves the efficiency and safety of large-scale outdoor operations.

[0035] In the present invention, the above-mentioned technical solutions can also be combined with each other to achieve more preferred combination solutions. Other features and advantages of the present invention will be described in the subsequent description, and some advantages can become obvious from the description, or can be understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained through the contents particularly pointed out in the description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The drawings are only for the purpose of illustrating particular embodiments and are not to be considered limiting of the present invention. Like reference symbols denote like components throughout the drawings.

[0037] Figure 1 This is a flow chart of an outdoor multi-actuator collaborative control method in Example 1 of the present invention;

[0038] Figure 2 This is a structural block diagram of an outdoor multi-actuator collaborative control system in Example 2 of the present invention. DETAILED DESCRIPTION

[0039] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not used to limit the scope of the present invention.

[0040] Example 1

[0041] A specific embodiment of the present invention discloses a method for coordinated control of outdoor multiple actuators, wherein each actuator comprises a local terminal, a device body, a video device, multiple sensors and multiple actuator controllers, and the local terminals of the multiple actuators are connected to multiple remote terminals via a networking bus; Figure 1 As shown, the collaborative control method includes the following steps:

[0042] S1. A local or remote terminal with master control authority is used as a master control terminal, and at least one execution mechanism is used as an execution node. The instructions to be executed and their priorities are obtained according to the target location and the data collected by the sensors of the execution node;

[0043] S2. Create tasks for execution nodes according to the instructions to be executed and their priorities, and add them to the task queue in descending order of priority;

[0044] S3. The main control terminal starts the task queue and sends each task in the task queue in sequence. The execution node parses the instructions to be executed of the task according to the task type through the corresponding execution controller, controls the device body to perform the corresponding action and feedback the execution result until all tasks in the task queue are completed.

[0045] It should be noted that the local terminal of each actuator, that is, the local host computer terminal, provides a human-computer interaction interface, supports functions such as control parameter input, one-key equipment start-up, and manual emergency intervention; it supports the same bus communication protocol as the execution controller, analyzes and displays the control status in real time, and provides sound and light alarms when the warning conditions are met.

[0046] The execution controller uses ADC to complete sensor data acquisition and packages and uploads it to the local terminal and the main control terminal in accordance with the protocol; the software communication module of the execution controller parses the instructions of the local terminal and the main control terminal, and completes the action execution through the switch output; and provides a fault detection function to report hardware system faults to the local terminal and the main control terminal in real time for alarm.

[0047] The video device is one or more groups of infrared cameras, each group of infrared cameras includes at least 2 infrared cameras, providing infrared night vision function, thereby supporting the actuator to work at night; the video viewing angle can be switched to ensure multi-angle safety monitoring of the work site; the resolution can be up to 1280*800, the frame rate can be up to 90FPS, and the image data is sufficient; it supports multiple image formats such as YUYV, UYVU, RGB8, Y8, Y16, etc., which is convenient for image analysis.

[0048] The multiple sensors include but are not limited to: TOF laser sensor, displacement sensor, inclination sensor and hydraulic system pressure sensor. The TOF laser sensor is used to measure the distance; the displacement sensor and inclination sensor are used to obtain the position data of the actuator; the pressure sensor is used to collect the actuator data as a mark of the hydraulic action being executed.

[0049] The local terminals of multiple actuators are connected to multiple remote terminals through the networking bus, and communicate in the message format of the message frame to achieve bus data sharing and mutual video stream push. Each actuator is connected to the network through a self-organizing network, that is, the local terminal obtains its own network address and identity by reading the local configuration file. One of them is the administrator node, which completes the node address record, online status monitoring, node identity statistics and sends it down.

[0050] Specifically, each set of actuators is implanted with a local configuration file, which includes its own CAN bus ID and identity. The CAN bus ID includes the priority of sending messages, command number and its own network address, and the identity is an administrator or ordinary member. Each time the system is powered on, the configuration file is obtained and the CAN bus initialization is completed. At this time, its own network address has been determined, and then the heartbeat message frame containing its own network address and identity is started. In each network, a unique node with the identity of administrator is set to monitor all heartbeat message frames on the network, parse and store each node address and corresponding identity, and then package and send it to the remaining nodes. If a new heartbeat address appears, this process is repeated until the statistics are complete; ordinary member nodes receive the data frame after the statistics are completed, parse and store it. After completing the above process, all node addresses and identities will be saved locally, and new nodes can be added online at any time; similarly, if a node exits, the heartbeat disappears, and after the timeout, it is removed from the local recorded network and notified to other members to complete the online removal of the node. This method improves the scalability of the network and avoids the problem of repeated program writing.

[0051] In step S1 of the collaborative control method, according to the outdoor operation task, the operator applies for master control authority from any one of the local terminals and remote terminals of multiple actuators. The terminal broadcasts the master control authority request to the bus and waits for the other terminals to respond. If no other terminal responds or the other terminals agree to the request, the terminal obtains the master control authority, and the other terminals release the control authority, only completing the data reporting function, and no longer sending control messages. Preferably, one of the remote terminals is selected as the master control terminal to improve the safety of the operation.

[0052] It should be noted that in this embodiment, multiple sets of actuators are laid on a linear guide rail, and the main control terminal selects multiple actuators as execution nodes according to the outdoor operation task, such as selecting an actuator that is closer to the target, or selecting a suitable actuator according to the operation task.

[0053] Next, according to the type of the target location, the instructions to be executed in the corresponding situation are obtained.

[0054] If the target is the final destination, the X, Y, and Z axis coordinate values ​​of the target position are determined, and then the instructions to be executed are obtained according to the data collected by the sensors of the target position and the execution node, including:

[0055] ① Obtain the final destination position as the target position;

[0056] ②According to the data collected by the sensor of the execution node, the position of the action execution component in the execution node is obtained and converted to the geodetic coordinate system to obtain the actual position;

[0057] It should be noted that the geodetic coordinate system in this embodiment takes the linear guide as the X-axis, the position of the first set of actuators on the linear guide as the origin, the direction perpendicular to the linear guide on the ground as the Y-axis, and the direction perpendicular to the ground as the Z-axis. The actuator moves horizontally along the X-axis, vertically in the Y-axis direction, and telescopically in the Z-axis direction.

[0058] For example, the device body in the actuator is a hydraulic lifting device. If it is a horizontal movement on the guide rail, the action execution component is the device body. If it is a grabbing movement in the air, the action execution component is the hook on the device body. The distance to the action execution component is obtained according to the TOF laser sensor in the actuator, and converted to the geodetic coordinate system to obtain the actual position.

[0059] ③According to the difference between the actual position and the target position, the distance difference in each direction in the geodetic coordinate system is obtained, that is, the coordinate difference between the actual position and the target position on the X, Y and Z axes;

[0060] ④ Create pending instructions in each direction, use the distance difference in the corresponding direction as the distance to be moved in the pending instructions, and obtain each pending instruction.

[0061] It should be noted that, in this case, the job task is completed after the execution node completes the pending instructions.

[0062] If the target is an operation object, the position of the operation object is unknown, and the target position is the position of the operation object obtained by identifying the video stream output by the video device of the execution node, then the instruction to be executed is obtained according to the target position and the data collected by the sensor of the execution node, including:

[0063] S11, obtaining a target position according to the video stream identification output by the video device of the execution node;

[0064] S12, according to the data collected by the sensor of the execution node, obtain the position of the action execution component in the execution node, and convert it to the earth coordinate system to obtain the actual position;

[0065] S13, obtaining the distance difference in each direction in the geodetic coordinate system according to the difference between the actual position and the target position;

[0066] S14. Identify whether the distance difference in each direction is greater than a threshold value. If the distance difference is greater than the threshold value, create a to-be-executed instruction in the corresponding direction, take half of the distance difference in the corresponding direction as the distance to be moved in the to-be-executed instruction in that direction, and obtain each to-be-executed instruction; when the execution node completes each to-be-executed instruction, return to step S11; if the distance difference in each direction is less than the threshold value, end the acquisition operation of the to-be-executed instruction at the target position.

[0067] It should be noted that in this case, the position of the operating object is dynamically identified as the target position. In order to achieve precise control, this embodiment uses a dichotomy method for the distance difference between the actual position and the target position, and the distance to be moved in the generated instruction to be executed is half of the current distance difference. After the execution mechanism completes the instruction to be executed, it adopts a step-by-step approach, returns to step S11 to identify the position of the operating object according to the video stream, updates the target position, returns to step S12 to update the actual position according to the data collected by the sensor, calculates a new distance difference in step S13, and again uses half of the distance difference as the distance to be moved in step S14 to generate a new instruction to be executed, and so on, until the distance difference is less than the threshold, and no more instructions to be executed need to be generated, and the execution node completes the task.

[0068] Specifically, the main control terminal pulls the video stream of the actuator and obtains the target position according to the video stream output by the video device of the execution node, including the following steps:

[0069] Extract each frame of image from the video stream, and use the trained neural network model to obtain the image with the target as the image to be identified;

[0070] After converting the image frame to be identified into a grayscale image, the edge of the target is determined using an edge detection algorithm, and then binarized to obtain an edge binary image;

[0071] Preferably, the averaging method is used to convert the grayscale of the image to be identified to obtain a grayscale image, and then a Gaussian filter is used to perform noise reduction and convolution operations on the grayscale image to weaken the interference information of individual pixels and improve the accuracy of edge detection; the Candy operator is used to calculate the gradient and direction of each pixel and perform non-maximum suppression. After threshold screening, an edge binary image is obtained according to the high and low thresholds, and image correction is performed according to the camera intrinsic parameters, distortion parameters and correction parameters.

[0072] The binocular ranging algorithm is used to obtain the distance between each pixel in the edge binary image and the video device of the execution node through formula (1), and the minimum distance is taken as the vertical distance from the target to the video device of the execution node; the horizontal offset distance of the target is obtained according to the vertical distance and the parameters of the video device through formula (2); the vertical distance and the horizontal offset distance are converted to the geodetic coordinate system to obtain the target position.

[0073] z=Bf / |X R -X T | Formula (1)

[0074] y=(z(W-2X h )-Df) / 2f formula (2)

[0075] Where z represents the vertical distance, y represents the horizontal offset distance, W represents the imaging length of the video device, B represents the center distance between the two infrared cameras in the video device, f represents the focal length of the video device, and X ... horizontal offset distance. R , X T They represent the horizontal coordinates of the imaging points of the target on the two infrared cameras in the video device, X h =min{X R ,X T}, take X R and X T The smaller horizontal coordinate in R -X T When representing parallax, take X T For X h ; When X T -X R When representing parallax, take X R For X h .

[0076] Preferably, when the distance difference is less than the distance threshold, the execution node acts according to the instruction to be executed, and at the same time fuses the sensor data to generate a new instruction to be executed, and according to the actual situation, the new instruction to be executed is executed first, or inserted into the original instruction to be executed.

[0077] It should be noted that this embodiment uses the distance difference in the X, Y and Z axis directions as an example. In practice, if the device body of the actuator supports rotation, the rotation angle and rotation direction can be calculated based on the target position and the actual position as the instructions to be executed.

[0078] The priority of the instructions to be executed is preset according to the direction of action.

[0079] In step S2, each action to be executed is represented by a task, and is added to a task queue according to priority to manage a series of tasks, flexibly control the actions of multiple actuators, and improve synchronization.

[0080] Specifically, each task includes: instructions to be executed, priority, list of execution nodes, task status, timeout, timeout timer, bus data processing interface, task status change signal, task start function and task suspension function. Among them, the instructions to be executed include the direction of action and the distance to be moved, which are issued in the form of bus control messages; the priority is used to mark the importance level of this task; the execution node is one or more execution mechanisms selected by the master terminal to execute the task; the task status is used to record the current task execution status, including instructions sent, instructions received, executing, execution completed, action suspended, instructions refused to execute, unknown instructions and action timeout; the timeout time and timeout timer are used to determine whether the task has timed out, that is, whether the execution result is received within the specified timeout after the instruction is sent; the bus data interface receives the bus message, parses the task execution result of the current execution controller and assigns the current task status; the task status change signal is used to trigger when the task status changes, for the task queue management task; the task start function is used to start the task; the task suspension function is used to record the completion of the current task and set the task status to action suspension.

[0081] This embodiment ensures that tasks are strictly executed in the specified priority order through a task queue. The task queue includes: a task list, a task adding interface, a task queue starting interface, a current task state change callback function, and a task queue status signal. The task list is used to store one or more task classes; the task adding interface is used to add tasks to the task list; the task queue starting interface is used to start the entire task queue; the current task state change callback function is bound to the task state change signal in the current task, and the task queue is processed in real time in the callback function when the current task state changes; the task queue status signal is used to trigger when all tasks in the queue are completed or when a task has an exception.

[0082] When the master terminal controls multiple sets of actuators to act synchronously, when the instructions to be executed by the multiple actuators are the same, a task with multiple execution nodes is created, and the device bodies of the multiple execution nodes execute the same action at the same time; when the instructions to be executed by the multiple actuators are different, tasks with their own execution nodes are created respectively, added to the task queue corresponding to each execution node, and multiple task queues are started at the same time, and the device bodies of the multiple execution nodes execute different actions at the same time.

[0083] Preferably, when the main control terminal only needs to control a set of actuators to perform a task, it can also only create and start a task without using a task queue, thus saving resource space.

[0084] After the task is created and instantiated, the task is added to the task list through the task queue's task adding interface. During the adding process, the task queue retrieves the task priority in the current queue and the priority of the task to be added, and arranges them in order from high to low.

[0085] In step S3, the main control terminal calls the task queue start interface to start the task queue. The task queue automatically calls the task start function of the first task. The first task starts to execute and the timeout timer starts working. The task queue modifies the status of this task to the instruction has been sent.

[0086] The execution node parses the instructions to be executed of the task according to the corresponding execution controller based on the task type, controls the device body to execute the corresponding action and feedback the execution result.

[0087] If a feedback message corresponding to the execution result of the actuator is received at the bus processing interface within the timeout period, the task queue will modify the status of this task to the action status in the feedback message, such as the instruction is received, the action is being executed, the action is completed, the instruction is rejected or the instruction is unknown. If no feedback is received within the timeout period, the task status is set to task timeout.

[0088] If the task status of the currently executing task changes to completed, this task is destroyed from the task list of the task queue, the next task becomes the first task, and the task start function is automatically called to start the new first task, and so on until the tasks in the entire task queue are completed. At this time, the task queue triggers the task queue status signal to inform the main control terminal that all tasks are completed, and the task queue is destroyed by itself to avoid memory leaks.

[0089] If the status of the currently executing task is changed to rejected execution, unknown instruction or action timeout, this task queue immediately triggers the task queue status signal to throw an exception and stop executing the task. The execution node stops executing all actions and leaves it to the operator to decide whether to continue.

[0090] Furthermore, when the main control terminal performs the corresponding action on the device body, it dynamically creates a new task and adds it to the started task queue based on the data collected by the sensor of the corresponding execution node. If the priority of the new task is lower than the priority of the currently executed task, the new task is added to the corresponding position after sorting from high to low priority. Otherwise, the currently executing task is suspended through the task suspension function, and the new task is inserted into the first position of the task queue and issued. When the new task is executed, the suspended task continues to be executed.

[0091] Specifically, the currently executing task is suspended through the task suspension function, including:

[0092] According to the action status of the execution node, the completion status of the currently executed task is recorded, including the completed and remaining tasks to be completed, and the task status is set to action pending.

[0093] Furthermore, when the main control terminal receives an emergency stop command, it automatically creates a stop task with the highest priority, adds it to the first position of all task queues and sends it down, and the task being executed is automatically suspended. When the main control terminal receives a cancel emergency stop command, it destroys the stop tasks in all task queues and continues with subsequent tasks in the task queue, or ends all tasks and destroys the task queue.

[0094] Compared with the prior art, the outdoor multi-actuator collaborative control method provided in this embodiment acquires the to-be-executed instructions of collaborative control by real-time data collection, represents each to-be-executed action with a task, and manages a series of tasks by adding them to the task queue according to priority, flexibly controls the actions of multiple sets of actuators, ensures the high synchronization of multiple sets of actuators, and greatly improves the control accuracy. The control instructions are issued and the execution status is checked through the control message frame, and the online mounting and removal of the actuator or remote console are supported; the on-site video monitoring is realized by transmitting the video stream, and the intelligent control of each actuator is realized, and the remote operation of the on-site personnel is supported, which improves the safety of the operation process, and is more conducive to the overall control of the site for large-scale outdoor operations, reduces the operation cost, and improves the operation efficiency and safety of large-scale outdoor operations.

[0095] Example 2

[0096] Another embodiment of the present invention discloses an outdoor multi-actuator coordinated control system, such as Figure 2 As shown, the system includes: multiple sets of actuators and multiple remote terminals, each set of actuators includes a local terminal, a device body, a video device, multiple sensors and multiple execution controllers, and the local terminals of the multiple sets of actuators are connected to the multiple remote terminals through a networking bus; the local or remote terminal with master control authority serves as the master control terminal, thereby realizing an outdoor multi-actuator collaborative control method in Example 1 to collaboratively control multiple sets of actuators.

[0097] It should be noted that Figure 2 Among them, N1, N2, M1 and M2 represent quantities. There is no corresponding relationship between these quantities. Each execution controller is connected to one or more sensors through a sampling line.

[0098] It should be noted that the entire system is powered by three-phase four-wire 380V / 50Hz AC power externally, and the internal actuator is equipped with an AC / DC power supply that outputs 24V DC power for the control system; the actuator communicates internally, between actuators, and with the remote terminal in the message format of a message frame. Buses such as CAN-bus, RS232, and Profibus can be used, or wireless communication solutions such as Wifi6 and 5G with good real-time performance, strong stability, and sufficient bandwidth can be used; the video stream push of the video device can be pushed to the local terminal using the Ethernet internal networking after software processing, or it can be directly transmitted using standard transmission interface lines such as VGA, DVI, and HDMI, or it can be completed together relying on the above-mentioned wireless communication methods.

[0099] Since the outdoor multi-actuator coordinated control system of this embodiment and the aforementioned outdoor multi-actuator coordinated control method can be mutually referenced, it is a repeated description here, so it will not be repeated here. Since the principles of this system embodiment and the aforementioned method embodiment are the same, this system embodiment also has the corresponding technical effects of the aforementioned method embodiment.

[0100] Example 3

[0101] In another embodiment of the present invention, two hydraulic lifting devices are used as actuators to form a collaborative control system of Example 2 with a remote terminal. The remote terminal serves as a master control terminal and adopts the collaborative control method of Example 1 to control the two hydraulic lifting devices to synchronously grab the cable and move to the final destination.

[0102] Specifically, the local terminal is equipped with an i.MX6Q processor, runs the Linux operating system, and is equipped with an electrical control box operation panel and remote control function for the actuator; the actuator controller runs the FreeRTOS operating system, and the three actuator controllers control horizontal movement, vertical movement, and hook extension and retraction respectively; each actuator controller is connected to two sensors through a sampling line, including a TOF laser sensor and a hydraulic pressure sensor; the video device uses Intel RealSense D400, which provides USB infrared video output. The power supply method in the networking bus uses an industrial general 24V DC power supply; the communication bus uses the CAN bus, and the communication protocol definition follows the message frame of the SAE-J1939 protocol to realize the sending and receiving of data messages; a 100M Ethernet network is used for video stream push; the emergency stop solution is realized by connecting multiple sets of actuators in series with emergency stop switches.

[0103] The steps of the main control terminal controlling two hydraulic lifting devices to synchronously grab the cables and move them to the final destination include:

[0104] ① It is known that the cable that needs to be hoisted synchronously is 25m long. The positions of the two hydraulic lifting devices are obtained according to the TOF laser sensor. An x-axis translation task is created and started. The two hydraulic lifting devices are controlled to be 24m apart on the x-axis first and stop after reaching the corresponding position.

[0105] ② The video device located at the hook starts to search for the target downward along the Z-axis direction, uses the V4L2 video framework interface provided by Linux to read the video output of the USB video device, and pushes the acquired video stream to the local display device and the Ethernet designated port for the main control terminal to pull.

[0106] ③ The main control terminal performs memory mapping in the memory and creates processing spaces for the images sent back by the two actuators. According to the pulled video stream, the vertical distance and horizontal offset distance of the two actuators from the cable are obtained using formulas (1) and (2), and the geodetic coordinate system is converted to obtain the target position.

[0107] ④According to the TOF laser sensors in the two sets of actuators, the hook distances of the two sets of actuators are obtained respectively, and converted into the geodetic coordinate system to obtain the actual position of the hook.

[0108] ⑤According to the actual position and target position of the hook, the distance difference required to move in the Y-axis direction and the Z-axis direction is obtained. The two sets of actuators are ΔY1 and ΔZ1, ΔY2 and ΔZ2 respectively. Then the binary approach cable is adopted to instantiate two task queues to control the two sets of actuators respectively. The execution nodes of the tasks in task queue 1 are all actuator 1, and the execution nodes of the tasks in task queue 2 are all execution units 2; the Y-axis direction hydraulic cylinder movement task and the Z-axis direction hydraulic cylinder movement task with distances of ΔY1 / 2 and ΔZ1 / 2 are instantiated respectively, and inserted into task queue 1; the Y-axis direction hydraulic cylinder movement task and the Z-axis direction hydraulic cylinder movement task with distances of ΔY2 / 2 and ΔZ2 / 2 are instantiated respectively, and inserted into task queue 2; the priorities of the above Y-axis direction movement tasks are higher than those of the Z-axis direction, and the two task queues are started simultaneously after completion;

[0109] ⑥After the above task queue is successfully completed, re-execute steps ③-⑤, gradually approach the cable until ΔY1 and ΔZ1, ΔY2 and ΔZ2 are less than 20cm. In the process of executing the task queue, the hydraulic pressure sensor data is integrated at the same time to judge the hydraulic pressure with a period of 200ms. If the cylinder pressure is detected to increase, it is considered that the hook and the cable have completed contact; instantiate the hook grabbing task with a higher priority than the moving task (that is, the hook controls the reversing action of the hydraulic valve), and call the task queue adding task function to add them to task queue 1 and task queue 2 respectively. At this time, other tasks in the queue are suspended, and the actuator prioritizes the grabbing task and then continues to complete the Y-axis and Z-axis moving tasks in the task queue as posture fine-tuning;

[0110] ⑦ The operator inputs the final destination position (X-axis, Y-axis and Z-axis coordinates) on the main control terminal, and calculates the distances to be moved in the X-axis direction, Y-axis direction and Z-axis direction (ΔX', ΔY' and ΔZ' respectively) in combination with the hook position obtained by the current TOF laser sensor. The X-axis hydraulic cylinder movement task, Y-axis hydraulic cylinder movement task and Z-axis hydraulic cylinder movement task with distances of ΔX', ΔY' and ΔZ' are instantiated respectively. The execution nodes are two sets of actuators, and then a task queue is instantiated to synchronously control the two sets of actuators. The tasks are added to the task queue according to priority and then start execution until the task queue is destroyed when the task is completed.

[0111] Those skilled in the art will appreciate that all or part of the processes of the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, wherein the computer-readable storage medium is a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0112] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. A method for coordinated control of outdoor multiple actuators, characterized in that: Each set of actuators includes a local terminal, a device body, a video device, multiple sensors and multiple execution controllers. The local terminals of multiple sets of actuators are connected to multiple remote terminals through a networking bus. The collaborative control method includes the following steps: A local or remote terminal with master control authority acts as a master control terminal, selects an actuator as an execution node, and obtains instructions to be executed and their priorities based on the target location and data collected by sensors of the execution node; Create tasks for execution nodes according to the instructions to be executed and their priorities, and add them to the task queue in descending order of priority; The main control terminal starts the task queue and sends each task in the task queue in turn. The execution node parses the task's instructions to be executed according to the task type through the corresponding execution controller, controls the device body to perform the corresponding action and feedback the execution result until all tasks in the task queue are completed.

2. The outdoor multi-actuator coordinated control method according to claim 1, characterized in that: When the instructions to be executed by multiple actuators are the same, a task with multiple execution nodes is created, and the device bodies of the multiple execution nodes execute the same action at the same time; when the instructions to be executed by multiple actuators are different, tasks with their own execution nodes are created respectively, added to the task queue corresponding to each execution node, and multiple task queues are started at the same time, and the device bodies of the multiple execution nodes execute different actions at the same time.

3. The outdoor multi-actuator coordinated control method according to claim 1, characterized in that: If the target location is the final destination location, obtaining the instruction to be executed according to the target location and the data collected by the sensor of the execution node includes: Get the final destination position as the target position; According to the data collected by the sensor of the execution node, the position of the action execution component in the execution node is obtained and converted into the earth coordinate system to obtain the actual position; According to the difference between the actual position and the target position, the distance difference in each direction in the geodetic coordinate system is obtained; Create instructions to be executed in each direction, use the distance difference in the corresponding direction as the distance to be moved in the instructions to be executed, and obtain each instruction to be executed.

4. The outdoor multi-actuator coordinated control method according to claim 1, characterized in that: If the target position is the position of the operation object obtained by identifying the video stream output by the video device of the execution node, obtaining the instruction to be executed according to the target position and the data collected by the sensor of the execution node includes: S11, obtaining a target position according to the video stream identification output by the video device of the execution node; S12, according to the data collected by the sensor of the execution node, obtain the position of the action execution component in the execution node, and convert it to the earth coordinate system to obtain the actual position; S13, obtaining the distance difference in each direction in the geodetic coordinate system according to the difference between the actual position and the target position; S14. Identify whether the distance difference in each direction is greater than a threshold value. If the distance difference is greater than the threshold value, create a to-be-executed instruction in the corresponding direction, take half of the distance difference in the corresponding direction as the distance to be moved in the to-be-executed instruction in that direction, and obtain each to-be-executed instruction; when the execution node completes each to-be-executed instruction, return to step S11; if the distance difference in each direction is less than the threshold value, end the acquisition operation of the to-be-executed instruction at the target position.

5. The outdoor multi-actuator coordinated control method according to claim 4, characterized in that: The step of obtaining the target position according to the video stream output by the video device of the execution node includes: Extract each frame of image from the video stream, and use the trained neural network model to obtain the image with the target as the image to be identified; After converting the image frame to be identified into a grayscale image, the edge of the target is determined using an edge detection algorithm, and then binarized to obtain an edge binary image; A binocular ranging algorithm is used to obtain the distance between each pixel in the edge binary image and the video device of the execution node, and the minimum distance is taken as the vertical distance from the target to the video device of the execution node; the horizontal offset distance of the target is obtained according to the vertical distance and the parameters of the video device; the vertical distance and the horizontal offset distance are converted to the geodetic coordinate system to obtain the target position.

6. The outdoor multi-actuator coordinated control method according to claim 5, characterized in that: According to the vertical distance and the parameters of the video device, the horizontal offset distance of the target is obtained by the following formula: <h2 style=";text-align:left;direction:ltr">y = (z(W-2X<h2 style=";text-align:left;direction:ltr"> h <h2 style=";text-align:left;direction:ltr"> )-Bf) / 2f Where y represents the horizontal offset distance, z represents the vertical distance, W represents the imaging length of the video device, B represents the center distance between the two infrared cameras in the video device, f represents the focal length of the video device, and X represents the vertical distance. h Indicates the smaller horizontal coordinate of the imaging point of the target on the two infrared cameras in the video device.

7. The outdoor multi-actuator coordinated control method according to claim 2, characterized in that: When the main control terminal performs the corresponding action on the device body, it dynamically creates a new task and adds it to the started task queue based on the data collected by the sensor of the corresponding execution node. If the priority of the new task is lower than the priority of the currently executed task, the new task is added to the corresponding position after sorting from high to low priority. Otherwise, the currently executed task is suspended, and the new task is inserted into the first position of the task queue and issued. When the new task is executed, the suspended task continues to be executed.

8. The outdoor multi-actuator coordinated control method according to claim 1, characterized in that: The task also includes a timeout period and a timeout timer. The timeout timer is started each time a task is issued. If no execution result is received within the timeout period, the task is changed to a task timeout, an exception is thrown, and the execution node stops executing all actions.

9. The outdoor multi-actuator coordinated control method according to claim 1, characterized in that: When the master terminal receives an emergency stop command, it automatically creates a stop task with the highest priority, adds it to the first position of all task queues and sends it down. The task being executed is automatically suspended. When the master terminal receives a cancel emergency stop command, it destroys all stop tasks in the task queues and continues with subsequent tasks in the task queues, or ends all tasks and destroys the task queues.

10. An outdoor multi-actuator coordinated control system, characterized in that: include: Multiple sets of actuators and multiple remote terminals, each set of actuators includes a local terminal, a device body, a video device, multiple sensors and multiple execution controllers, and the local terminals of the multiple sets of actuators are connected to the multiple remote terminals through a networking bus; a local or remote terminal with master control authority serves as a master control terminal, and uses the collaborative control method described in any one of claims 1 to 9 to collaboratively control the multiple sets of actuators.

Images