Rail vehicle multi-arm inspection control system and method

Abstract

The application discloses a rail vehicle multi-arm type maintenance control system and method, and the system comprises a plurality of mechanical arms, a motion execution mechanism for moving the mechanical arms, an image acquisition mechanism mounted on the mechanical arms and used for acquiring maintenance images, a walking mechanism for moving the rail vehicle multi-arm type maintenance control system, and a controller; wherein a single-process multi-thread EtherCAT master station is arranged in the controller, and the controller is used for respectively performing data interaction with each of the mechanical arms, the walking mechanism, the motion execution mechanism and the image acquisition mechanism in different threads of the EtherCAT network with different cycle periods. The system performance can be maximally exerted while meeting the communication cycle requirements of different functions, and intelligent maintenance is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robot control, and in particular to a rail vehicle multi-arm type maintenance control system and method. BACKGROUND

[0002] In the field of rail transit, the bogie of a vehicle is a key link for safe operation of a train and is also a focus of vehicle maintenance. At present, bogie maintenance has gradually transitioned from manual to robot automatic maintenance. Image acquisition is automatically performed by a robot to achieve intelligent and rapid maintenance and rapid positioning of defect locations. Automatic detection by a robot instead of manual detection is a future trend.

[0003] However, for a bogie system that is complex in structure and narrow in space, one robot arm cannot quickly complete a maintenance task. When multiple robot arms are used to work together, real-time control, camera data transmission and communication have always been a problem. How to achieve precise linkage control of multiple robot arms while ensuring the performance of the robot arms is a difficult problem to be solved. In addition, the bogie system is complex in structure, and how to use vision for high-precision maintenance is also a difficult problem.

[0004] At present, most robot arm communications are one-to-one mode, that is, one controller controls one robot arm. In the case of multiple robot arm linkage, multiple controllers need to be provided. The walking mechanism, action execution mechanism and image acquisition system of the multi-arm type maintenance system all need to be equipped with different controllers, resulting in a relatively cumbersome system.

[0005] At the same time, the existing one-to-many communication mode of EtherCAT cannot meet the needs of different devices with different communication cycles and cannot maximize the performance of the system. For example, the communication cycle of a camera is 8ms, and the communication cycle of a robot arm is 1ms. According to the previous one-to-many system, if the real-time performance of the system is 1ms, it meets the communication cycle of the robot arm, but it is redundant for the 8ms camera. If the real-time performance of the system is 8ms, it meets the communication cycle of the camera, but it cannot meet the real-time performance of 1ms of the robot arm. SUMMARY

[0006] The present application proposes a rail vehicle multi-arm type maintenance control system and method to solve the technical problems that the existing technology cannot meet the needs of different devices with different communication cycles and cannot maximize the performance of the system.

[0007] In a first aspect, the embodiments of the present application provide a rail vehicle multi-arm maintenance control system, comprising: a plurality of mechanical arms, an action execution mechanism for moving the mechanical arms, an image acquisition mechanism mounted on the mechanical arms for acquiring maintenance images, a walking mechanism for moving the rail vehicle multi-arm maintenance control system, and a controller; wherein a single-process multi-thread EtherCAT master station is run in the controller, and the controller is configured to interact with each of the mechanical arms, the walking mechanism, the action execution mechanism, and the image acquisition mechanism in different threads of the EtherCAT network with different cycle periods.

[0008] The rail vehicle multi-arm maintenance control system described above, wherein the mechanical arms, the walking mechanism, the action execution mechanism, and the image acquisition mechanism run EtherCAT slave stations, the EtherCAT master station and the EtherCAT slave stations constitute different EtherCAT networks, and different EtherCAT networks use different DC synchronous clocks.

[0009] The rail vehicle multi-arm maintenance control system described above, wherein the DC synchronous clock selects the clock of the first slave station in the EtherCAT network as a reference clock for synchronous operation, obtains the transmission delay and time offset of the remaining slave stations and the reference clock through the timestamp information in the EtherCAT information frame, obtains a preliminary synchronization time, and realizes the synchronization of the clocks of the slave stations and the reference clock based on the preliminary synchronization time through periodic time drift compensation, thereby realizing the synchronous operation of the slave stations.

[0010] The rail vehicle multi-arm maintenance control system described above, wherein the EtherCAT master station is created by a multi-instance version of an API function in a process through multi-threading, different master stations are bound to different network card devices in the controller, and the clock synchronization between the master stations is realized through signal synchronization between threads.

[0011] The rail vehicle multi-arm maintenance control system described above, wherein the robot control code for controlling the mechanical arms is embedded in the master station in the form of a dynamic link library and is loaded in a segmented manner.

[0012] The rail vehicle multi-arm maintenance control system described above, wherein when the master station runs to a PREOP state, a myECatSetup() function is called for initialization operation; after the master station is initialized, a myECatLoop() function is called to periodically run a get_args() of the dynamic link library to periodically interact with the slave stations of the mechanical arms.

[0013] The rail vehicle multi-arm maintenance control system, wherein a defect detection unit is further included, the maintenance image collected by the image collection mechanism is input into a model combining CNN and Transfromer for 2D coarse positioning to obtain two-dimensional position information, 3D point cloud matching algorithm is used for accurate positioning based on the two-dimensional position information to obtain six-dimensional position information, and the six-dimensional position information is returned to the controller.

[0014] In a second aspect, the embodiments of the present application provide a rail vehicle multi-arm maintenance control method, applied to the rail vehicle multi-arm maintenance control system in the first aspect, and characterized in that the method comprises the following steps.

[0015] Position moving step: the controller sends a control instruction to the walking mechanism, controls the walking mechanism to move to a specified position, and receives feedback information returned by the walking mechanism;

[0016] Image collection step: the controller sends a control instruction to the action execution mechanism, the mechanical arm and the image collection mechanism, controls the action execution mechanism to move the mechanical arm to a maintenance position, and controls the image collection mechanism to collect a maintenance image;

[0017] Defect detection step: the maintenance image is input into a defect detection unit for defect detection.

[0018] The rail vehicle multi-arm maintenance control method, wherein the defect detection step comprises the following steps: the maintenance image is input into a model combining CNN and Transfromer for 2D coarse positioning to obtain two-dimensional position information, 3D point cloud matching algorithm is used for accurate positioning based on the two-dimensional position information to obtain six-dimensional position information, and the six-dimensional position information is returned to the controller.

[0019] The rail vehicle multi-arm maintenance control method, wherein the method further comprises the following steps.

[0020] Master station creating step: a plurality of EtherCAT master stations are created in a process by a multi-threaded way through a multi-instance version of an API function, and the EtherCAT master stations are bound to different network card devices in the controller.

[0021] Compared with the prior art, the control system has the following advantages and positive effects.

[0022] 1. The control system runs an EtherCAT master station in a single-process multi-thread mode in a controller. Each thread can access the memory of the entire process, realize collaborative control of multiple devices, and interact with information; while meeting the communication cycle requirements of different functions, the system performance can be maximized;

[0023] 2、The intelligent visual defect detection and positioning algorithm developed by the application adopts a combined mode of 2D deep learning and 3D point cloud matching algorithm, the 2D algorithm is used for rough positioning of defects, and the 3D algorithm is used for accurate positioning, so that the defect positioning accuracy can be greatly improved, and intelligent maintenance can be realized. BRIEF DESCRIPTION OF DRAWINGS

[0024] Figure 1 A principle diagram of a multi-arm type maintenance control system of a rail vehicle is provided for the application;

[0025] Figure 2 A structural diagram of a multi-arm type maintenance control system of a rail vehicle is provided for the application;

[0026] Figure 3 An example diagram in which the controller controls different mechanisms in different control periods is provided for the application;

[0027] Figure 4 A time difference between a reference clock and a slave clock is provided for the application;

[0028] Figure 5 Another time difference between a reference clock and a slave clock is provided for the application;

[0029] Figure 6 A multi-instance API function is provided for the application, and an example diagram in which the multi-instance API function controls different mechanisms is provided for the application;

[0030] Figure 7 A 3D point cloud matching algorithm flowchart is provided for the application;

[0031] Figure 8 A 3D point cloud template is provided for the application;

[0032] Figure 9 A posture schematic diagram of a hydrogenation port obtained through 3D point cloud matching is provided for the application;

[0033] Figure 10 A step schematic diagram of a multi-arm type maintenance control method of a rail vehicle is provided for the application. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical scheme and advantages of the application clearer, the application is described and explained below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the application, and are not used to limit the application. Based on the examples provided in the application, all other examples obtained by those of ordinary skill in the art without creative labor fall within the scope of the application.

[0035] It is apparent that the drawings in the following description merely show some examples or embodiments of the present application, and the present application can be applied to other similar situations without creative labor by those skilled in the art based on these drawings. In addition, it can be understood that, although the efforts made in the development process can be complex and lengthy, some modifications, such as design, manufacture or production, etc. based on the technical content disclosed in the present application, are only routine technical means for those skilled in the art related to the content disclosed in the present application, and should not be understood as insufficient disclosure of the content disclosed in the present application.

[0036] Reference to "an embodiment" in this application means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily mutually exclusive of one another. It is explicitly contemplated that embodiments described herein can be combined with other embodiments in a non- conflicting manner.

[0037] Unless otherwise defined, technical terms or scientific terms used in the present application shall have the ordinary meaning understood by one of ordinary skill in the art to which the present application pertains. The terms "a", "an", "one", "this", and similar terms as used in the present application are not limited to the singular form but include plural forms unless otherwise defined. The terms "include", "comprise", "have", and any variations thereof as used in the present application are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or device including a list of steps or modules (units) is not limited to the listed steps or units, but can further include other steps or units not listed or can further include other steps or units inherent to such a process, method, product, or device. The terms "connect", "connected", "coupled", and similar terms as used in the present application are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The term "multiple" as used in the present application means two or more. The term "and / or" describes the association relationship of the associated objects, which means that there can be three relationships, for example, "A and / or B" can mean that A exists alone, A and B exist together, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects. The terms "first", "second", "third", and the like as used in the present application are merely to distinguish similar objects, and do not represent a specific order for the objects.

[0038] The present application will be described in detail below with reference to the embodiments shown in the drawings, but it should be noted that these embodiments are not limiting to the present application, and equivalent transformations or substitutions of function, method or structure made by those skilled in the art according to these embodiments are within the protection scope of the present application.

[0039] Before the various embodiments of the present application are described in detail, the core inventive idea of the present application is outlined, and is described in detail through the following several embodiments.

[0040] The intelligent visual defect detection and positioning algorithm is developed, the 2D deep learning + 3D point cloud matching algorithm is adopted, the self-developed seven-axis collaborative mechanical arm is matched, and the intelligent maintenance of the bogie of the railway vehicle is completed. The control system innovatively uses an EtherCAT master station in one process, which is divided into different threads, and controls the mechanical arm, camera communication and the like. Different threads can correspond to different cycle periods, that is, a single-process multi-thread bus mode is used, which meets the communication cycle requirements of different functions, maximizes the performance of the system, matches the intelligent visual algorithm, and realizes intelligent maintenance.

[0041] EtherCAT is a representative of real-time bus, and EtherCAT technology is more mature, stable, has more alternative solutions and strong technology extension than other industrial field real-time buses. In order to solve the problems of low efficiency and poor real-time communication of single-arm space robots in maintenance tasks, the present application provides a railway vehicle multi-arm maintenance control system, that is, a single-process multi-thread EtherCAT master station is run in one controller, and the EtherCAT network of each thread can control the mechanical arm, walking mechanism, action execution mechanism and image acquisition mechanism respectively, to realize cooperative control of multiple seven-axis mechanical arms, walking mechanisms, action execution mechanisms and image acquisition mechanisms.

[0042] Figure 1 A schematic diagram of the railway vehicle multi-arm maintenance control system provided by the present application is shown in Figure 1 The present application provides an air backup brake system (hereinafter referred to as "system").

[0043] The above-mentioned railway vehicle multi-arm maintenance control system comprises a plurality of mechanical arms, an action execution mechanism for moving the mechanical arms, an image acquisition mechanism mounted on the mechanical arms for acquiring maintenance images, a walking mechanism for moving the railway vehicle multi-arm maintenance control system, and a controller; wherein a single-process multi-thread EtherCAT master station is run in the controller, and the controller is used for data interaction with each of the mechanical arms, the walking mechanism, the action execution mechanism and the image acquisition mechanism in the EtherCAT network of different threads with different cycle periods.

[0044] Specifically, such as Figure 2 The multi-arm maintenance control system for rail vehicles shown mainly includes a controller, a seven-axis robotic arm, a traveling mechanism, a motion execution mechanism, and an image acquisition mechanism. The controller controls the robotic arm, the traveling mechanism, the motion execution mechanism, and the image acquisition mechanism. The traveling mechanism includes a drive wheel set, a driven wheel set, a vibration-damping wheel set 1, and a vibration-damping wheel set 2. The motion execution mechanism includes a horizontal movement module, a forward / backward movement module, and a lifting module that control the movement of the robotic arm. When performing a maintenance task, the controller controls the drive wheel set to move to the designated maintenance position. The lifting module moves, controlling the robotic arm to lift. At this time, the 3D camera in the image acquisition mechanism takes its first picture. Based on the camera's feedback, the controller controls the horizontal and forward / backward movement modules to move the seven-axis maintenance robotic arm to the maintenance position. Simultaneously, the controller controls the 3D camera in the image acquisition mechanism to take another picture. The results are then transmitted to the vision processing system for fault diagnosis.

[0045] This control system can achieve coordinated control of the above modules under the same controller, and different threads can exchange data, such as... Figure 1 As shown, the controller, as the most important unit of this system, can run a single-process, multi-threaded EtherCAT master station within a single controller, controlling multiple EtherCAT buses. Depending on the cycle requirements of different mechanisms, each thread's EtherCAT network can run different cycle cycles, ensuring that each thread is unaffected and can exchange data.

[0046] In the above embodiments, the robotic arm, the walking mechanism, the motion execution mechanism, and the image acquisition mechanism operate EtherCAT slave stations. The EtherCAT master station and the EtherCAT slave station constitute different EtherCAT networks, and the different EtherCAT networks use different DC synchronization clocks.

[0047] The DC synchronization clock selects the clock of the first slave station in the EtherCAT network as the reference clock for synchronous operation. After obtaining the transmission delay and time offset of the other slave stations and the reference clock through the timestamp information in the EtherCAT information frame, the initial synchronization time is obtained. Based on the initial synchronization time, after periodic time drift compensation, the clocks of each slave station and the reference clock are synchronized, thereby realizing the synchronous operation between each slave station.

[0048] The application innovatively uses DC synchronous clock, each EtherCAT network uses different DC synchronous clock, so that the real-time of each network can be guaranteed, and the characteristics of multi-network are used to realize complex EtherCAT network equipment control, such as Figure 3 As shown in the figure, the controller controls different mechanisms with different control periods.

[0049] Specifically, the DC distributed clock is a time synchronization mechanism used to synchronize the operation of each node in the EtherCAT communication protocol, which is used to eliminate the clock asynchronization caused by propagation delay, time offset and time drift. Propagation delay refers to the transmission delay of messages between slaves, time offset refers to the asynchronization of slave clocks caused by different start times of slaves, and time drift refers to the timer difference caused by the different vibration frequencies of the crystal oscillator of different slaves, which further causes cumulative clock difference. The DC synchronous clock selects the clock of the first slave in the EtherCAT network as the reference clock for synchronous operation, calculates the transmission delay and time offset of the remaining slaves and the reference clock through the timestamp information in the EtherCAT information frame, and obtains a basically accurate time. At this time, the synchronization error is basically maintained within 100ns. Then, through periodic time drift compensation, the clocks of each slave and the reference clock can be accurately synchronized. In this way, the slaves can realize synchronous operation.

[0050] Through Figure 4 、 Figure 5 It can be seen that the difference between the reference clock and the clock of different slaves includes transmission delay, time offset and time drift. After filling the three parts, the reference clock and the slave clock can be synchronized. Among them, the transmission delay and the time offset can be calculated once, while the time drift needs to be calibrated periodically. Time drift compensation requires the slave acting as the reference clock to broadcast its own time every period. The slave compares the write time and the current system time, and takes the comparison error as the input of the time control register to increase or slow down the local clock in turn, so as to eliminate the drift error.

[0051] In some embodiments, the EtherCAT master station is created by a multi-instance version of API function in a process through multi-threading, and different master stations are bound to different network card devices in the controller, and the clock synchronization between the master stations is realized through signal synchronization between threads.

[0052] Specifically, as Figure 6As shown, the master core is implemented by using multi-instance API functions in the application, each API function has a single-instance version (prefixed es, such as esStopMaster) and a multi-instance version (prefixed em, such as emStopMaster), the first parameter of all multi-instance functions emXXX is an instance identifier (EtherCAT master handle), the master multi-threading function developed by the system can realize the cooperative control of different mechanisms under different cycles. Through the multi-instance version of the API function, the application can create multiple EtherCAT masters in a process through multi-threading, and bind the master to different network card devices. Therefore, the controller of the embodiment of the application needs to include at least 6 Ethernet cards to bind and deploy EtherCAT masters. After the application creates 6 EtherCAT masters in 6 threads through multi-instance functions, the synchronous running between multiple threads can be realized through the synchronization mechanism between threads. Since each master is deployed in the same process of the same host, there is no transmission delay, time offset and time drift between each master, and the clock synchronization between each master can be realized through simple signal synchronization between threads. Since each master has the ability to run the DC synchronous clock independently, through thread synchronization and DC clock synchronization, all devices in the entire network are in a synchronous running state, greatly improving the controllability and stability of the system.

[0053] The communication real-time requirement of the mechanical arm is high, usually 1ms, the real-time requirements of other walking mechanisms, action execution mechanisms and image acquisition systems are not high, 2-8ms can be achieved, the single-process multi-thread EtherCAT master is adopted, different threads can be allocated different communication periods, the cooperative control of different communication periods of each thread of the EtherCAT network can be realized, multiple EtherCAT controllers are not needed to realize multi-network control, and the precision control of high-performance EtherCAT servo is not needed to sacrifice for the low-performance EtherCAT servo, the scheme cost is saved, the overall performance is improved, and the mechanical arm end position control precision is improved. At the same time, the system can adapt to mechanical arms of different manufacturers and different configurations: six-axis, seven-axis, DELTA, SCARA, etc., by modifying the configuration parameters, cycle period, DH parameter, joint limit, encoder zero position and other information in the json file, the rapid control of the mechanical arm can be realized.

[0054] In some embodiments, when the maintenance system is running, the controller sends control instructions to the walking mechanism through the EtherCAT bus, and when the entire system moves to the specified position on the track, the completion signal is fed back through the bus, and the next control action is performed to lift or translate the mechanical arm body, so that the three mechanical arms run to the appropriate position. Then the controller drives the image acquisition mechanism to collect high-definition images of the area to be inspected at multiple angles. The entire operation process is completed by one controller, saving space and resources. The mechanical arm control is the main link of system control, and its main functions are as follows:

[0055] The controller sends control instructions to the slave station through the EtherCAT bus and receives feedback information from the slave station. The seven-axis mechanical arm has a built-in FPGA drive board as an EtherCAT slave station, which receives master station instructions and feeds back the status of the servo to the master station.

[0056] The robot control code for controlling the mechanical arm is embedded in the master station in the form of a dynamic link library and loaded in a segmented manner. When the master station runs to the PREOP state, the myECatSetup() function is called for initialization operation. After the master station is initialized, the myECatLoop() function is called to periodically run the get_args() of the dynamic link library and periodically interact with the slave station of the mechanical arm.

[0057] Specifically, the robot control code is embedded in the master station in the form of a dynamic link library. Since the robot control code contains time-consuming operations such as reading configuration file parameters and initializing the mechanical arm, it will affect the real-time performance of the entire system of the master station. Therefore, this embodiment adopts a segmented loading control code method to achieve a real-time requirement of 1ms.

[0058] The reading robot json file and initialization part of the control code are placed in the myECatSetup() function. When the master station runs to the PREOP state, the myECatSetup() function is called to complete the time-consuming initialization operation.

[0059] At the same time, the PDO data interaction part with higher real-time requirements is placed in the myECatLoop() function, which periodically performs data interaction with a specified cycle period (e.g., 1ms). After the master station is started, the function periodically runs the get_args() and set_args() functions of the dynamic link library to perform periodic data interaction. The main interaction data is shown in the following table.

[0060]

[0061] The data received by the slave station includes but is not limited to: the operation mode of the motor, the control keyword of the motor, the target position of the motor, the target speed of the motor, the target torque of the motor;

[0062] The data transmitted by the slave station includes but is not limited to: the state of the motor, the actual position of the motor, the actual speed of the motor, the actual torque of the motor.

[0063] Thus, under the premise of ensuring real-time, through the multi-thread communication function of the master station, precise cooperative control of the multi-arm seven-axis maintenance robot can be realized.

[0064] In some embodiments, the system further comprises a defect detection unit for inputting the maintenance image collected by the image acquisition mechanism into a model combining CNN and Transfromer for 2D coarse positioning to obtain two-dimensional position information, adopting a 3D point cloud matching algorithm based on the two-dimensional position information for precise positioning to obtain six-dimensional position information, and returning the six-dimensional position information to the controller.

[0065] Specifically, the defect detection unit based on deep learning adopted in this embodiment is a defect detection unit based on CNN and Transfromer. Transfromer is widely used in the field of natural language. The present application combines the Transformer model with the CNN architecture, splits the bogie image photographed by the image acquisition mechanism into small blocks, and provides linear embedding sequences of the small blocks as inputs of the model combining transformer and CNN.

[0066] During the training process, a cross-entropy loss function is used to supervise the training of the two branches of CNN and Transformer to obtain features with both CNN style and Transformer style. The model sends global features from the Transformer branch into the CNN to enhance the global perception ability of the CNN branch, and sends local features from the CNN branch into the Transformer to enhance the local perception ability of the Transformer branch, while making up for the defects of the two branches and improving the accuracy and speed of detection and positioning.

[0067] During the test, the image photographed by the vision is split into small blocks 224x224x3, which are sent into the Stem module. The Stem module is a 7x7 convolution with a step of 2 and a 3x3 max pooling with a step of 2, which is used to extract initial local features, and then sent into two branches respectively. The CNN branch and the Transformer branch are respectively composed of N repeated convolution and Transformer blocks. After down-sampling, multi-stage operation, etc., a 7x7 feature map is finally output, followed by avgpool and positioning to achieve classification and positioning.

[0068] Compared with the traditional convolutional neural network, the network is matched with the convolutional neural network and the Transform self-attention mechanism. In the main module of the convolutional neural network, the feature map calculated by the algorithm is first flattened, for example: if the feature map has a shape (h x w x), the flattened result will have a shape (h w x). Moreover, a learnable position encoding is added in each dimension, and the encoder takes the result sequence as input. The encoder uses multiple self-attention blocks to combine the features between different Embeddings, and the processed Embedding is transmitted to a decoder module. The decoder module processes all visual features to perform defect detection and 2D coarse positioning of the defect.

[0069] The 3D camera in the image acquisition mechanism can shoot 2D and 3D images, and the 2D and 3D position information is one-to-one corresponding. For the above-mentioned 2D coarse positioning algorithm, the defect position in XY two dimensions can be obtained, but the depth Z direction and the attitude information cannot be accurately obtained. Therefore, the following embodiment can realize the precise positioning of the six-dimensional attitude according to the two-dimensional position information and in combination with the 3D point cloud algorithm.

[0070] After coarse positioning, precise positioning is performed by using 3D point cloud information. The surface-based 3D matching algorithm is developed in the embodiment. The point cloud information is obtained by using the 3D camera in the image acquisition mechanism. First, the point cloud is denoised, and then the 3D precise positioning of the defect position is performed by using the 3D point cloud matching algorithm, so that the six-dimensional position information can be obtained. Finally, the six-dimensional information is transmitted to the controller, the mechanical arm is controlled to move to the position, and the information is recorded, so that the staff can quickly locate the defect position.

[0071] In a specific implementation, the implementation process of the 3D point cloud matching algorithm is as shown in Figure 7-9 A plurality of 3D point cloud data containing point cloud images are shot, and a template point cloud containing only a target is obtained through filtering, cropping and triangulation processing. At the same time, a template is made by using the create_surface_model algorithm, and a 3D point cloud template graph is obtained as shown in Figure 8 Through the 3D point cloud matching algorithm, the pose of the hydrogen inlet can be calculated. As shown in Figure 9 The left side of the figure shows the pose data of the hydrogen inlet, and the right side shows the attitude diagram of the matched hydrogen inlet.

[0072] The intelligent visual defect detection and positioning algorithm developed in the embodiment adopts the combination of the 2D deep learning and the 3D point cloud matching algorithm. The 2D algorithm performs coarse positioning of the defect, and the 3D algorithm performs precise positioning, which can greatly improve the defect positioning accuracy.

[0073] In practical applications, the control mode of each robot arm is different, and the control cycle of each robot arm is also different, which leads to little similarity between each robot arm slave station. In view of the actual situation, if the performance of each robot arm can be exerted, a plurality of EtherCAT networks can be used to control the slave station device in groups. A plurality of independent controllers or a plurality of independent EtherCAT master programs run in a multi-network controller. Whether it is a multi-hardware or a multi-program running mode, the influence of the cycle period limit on the whole control does not need to be worried about, and each slave station can exert its maximum benefit. However, such a scheme also has disadvantages: in the multi-hardware form, the data exchange between the hardware must also have real-time performance, which increases the workload and cost budget. In the multi-program running mode, the memory between each program cannot be called. Fortunately, compared with the multi-hardware form, the plurality of master programs can rely on shared memory for data exchange, but in this way, the read-write order of each program to the memory cannot be guaranteed, which will cause more instability.

[0074] The control system provided by the application is a single-process multi-thread EtherCAT master station running in a controller. Each thread can access the memory of the whole process, as long as the real-time performance of each master thread does not affect each other, the whole system can exert its full capacity. In a controller, the cooperative control of multiple devices is realized, and each thread can access the memory of the whole process to interact information.

[0075] The whole system of the application can be used as a general system, has strong portability, and can be adapted to robot arms of different manufacturers and different configurations; the system can be adapted to robot arms of different configurations such as six-axis, seven-axis, DELTA, SCARA, etc., and the information such as configuration parameters, cycle period, DH parameters, joint limit, encoder zero position in the json file can be modified to realize the rapid initialization of the robot arm. The key parameters of the json file are as follows:

[0076]

[0077]

[0078]

[0079]

[0080] The json file can modify the parameters in the file according to different robot arm configurations, and quickly adapt to robot arm bodies of different manufacturers, different brands and different structures.

[0081] Specifically, in the json file, Num_joints represents the number of axes of the robot arm, which can be set to 4, 6 or 7, representing a 4-axis, 6-axis or 7-axis robot arm, respectively;

[0082] Dh_table represents the dh parameter table of the robot arm. DH parameters (Denavit-Hartenberg parameters) are a mechanical arm mathematical model and coordinate system determination system that uses four parameters to express the angular relationship between two pairs of joint connecting rods. Different structures of robot arms have different dh parameters. According to the dh parameter table, the forward and inverse kinematics models of the robot arm can be established to realize point operation, trajectory planning and other operations.

[0083] Zero_point represents the zero point position of the robot arm, which records the encoder zero point of each joint motor.

[0084] If a new size of robot arm is obtained, the num_joints, dh_table and zero_point parameters in the json file can be modified to quickly control the robot arm with the controller.

[0085] The control system can be applied to a rail vehicle maintenance system. The main station reserves a control io api interface. By importing the external devices of the manufacturer io, different port digital inputs and outputs can be quickly obtained by importing the xml file of the manufacturer. Through the system, the cooperative control of multiple maintenance robots can be realized. The system can adapt to robot arms of different manufacturers and different configurations, and can add external IO control, saving the cost of the scheme, improving the overall performance, and improving the control accuracy of the end pose of the robot arm.

[0086] In combination with the rail vehicle multi-arm maintenance control system disclosed in the above embodiment, a specific implementation example of a rail vehicle multi-arm maintenance control method (hereinafter referred to as “method”) is disclosed.

[0087] Referring to Figure 10 , the method comprises:

[0088] Step S1: through the API function of the multi-instance version, multiple EtherCAT master stations are created in one process through the multi-threaded way, and the EtherCAT master stations are bound to different network card devices in the controller;

[0089] Step S2: the controller sends a control instruction to the walking mechanism, controls the walking mechanism to move to a specified position, and receives feedback information returned by the walking mechanism;

[0090] Step S3: sending a control instruction to the action execution mechanism, the mechanical arm and the image acquisition mechanism through the controller, controlling the action execution mechanism to move the mechanical arm to the maintenance position, and controlling the image acquisition mechanism to acquire a maintenance image;

[0091] Step S4: inputting the maintenance image into a defect detection unit for defect detection.

[0092] In the step S3, the maintenance image is input into a model combining CNN and Transfromer for 2D coarse positioning to obtain two-dimensional position information, a 3D point cloud matching algorithm is used for accurate positioning based on the two-dimensional position information to obtain six-dimensional position information, and the six-dimensional position information is returned to the controller.

[0093] The technical scheme of the rest of the same part in the multi-arm maintenance control method of the rail vehicle disclosed in the embodiment and the multi-arm maintenance control system of the rail vehicle disclosed in the above embodiment is described in the above embodiment, which will not be described here.

[0094] The technical features of the above embodiments can be combined arbitrarily. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combination of the technical features does not exist contradictory, it should be considered as the scope of the present application.

[0095] The above embodiments only express several embodiments of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, some modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of the patent of the present application should be subject to the appended claims.

Claims

1. A rail vehicle multi-arm service control system, characterized by, The rail vehicle multi-arm maintenance control system comprises a plurality of mechanical arms, a motion execution mechanism for moving the mechanical arms, an image acquisition mechanism mounted on the mechanical arms for collecting maintenance images, a walking mechanism for moving the rail vehicle multi-arm maintenance control system, a controller, and a defect detection unit; wherein a single-process multi-thread EtherCAT master station is run in the controller, and the controller is used for data interaction with each of the mechanical arms, the walking mechanism, the motion execution mechanism, and the image acquisition mechanism in different threads of the EtherCAT network with different cycle periods. The mechanical arms, the walking mechanism, the motion execution mechanism, and the image acquisition mechanism run EtherCAT slave stations, the EtherCAT master station and the EtherCAT slave stations constitute different EtherCAT networks, and different EtherCAT networks use different DC synchronous clocks. The DC synchronous clock selects the clock of the first slave station in the EtherCAT network as the reference clock for synchronous operation, obtains the transmission delay and time offset of the remaining slave stations and the reference clock through the timestamp information in the EtherCAT information frame, obtains the preliminary synchronization time, and realizes the synchronization of the clocks of each slave station and the reference clock based on the preliminary synchronization time through periodic time drift compensation, thereby realizing the synchronous operation of each slave station. The EtherCAT master station is created by a multi-instance version of an API function in a process through multi-threading, different master stations are bound to different network card devices in the controller, and the clock synchronization between the master stations is realized through signal synchronization between threads. The defect detection unit is used for 2D coarse positioning of the maintenance images collected by the image acquisition mechanism in a model combining CNN and Transfromer, obtaining two-dimensional position information, performing accurate positioning based on the two-dimensional position information by using a 3D point cloud matching algorithm, obtaining six-dimensional position information, and returning the six-dimensional position information to the controller. The robot control code for controlling the mechanical arms is embedded in the master station in the form of a dynamic link library and is loaded in a segmented manner.

2. The rail vehicle multi-arm service control system of claim 1, wherein, When the master station runs to the PREOP state, the myECatSetup() function is called for initialization operation; after the master station is initialized, the myECatLoop() function is called to periodically run the get_args() of the dynamic link library, and the periodic data interaction with the slave station of the mechanical arm is realized.

3. The rail vehicle multi-arm service control system of claim 2, wherein, The rail vehicle multi-arm maintenance control system comprises a plurality of mechanical arms, a motion execution mechanism for moving the mechanical arms, an image acquisition mechanism mounted on the mechanical arms for collecting maintenance images, a walking mechanism for moving the rail vehicle multi-arm maintenance control system, a controller, and a defect detection unit; wherein a single-process multi-thread EtherCAT master station is run in the controller, and the controller is used for data interaction with each of the mechanical arms, the walking mechanism, the motion execution mechanism, and the image acquisition mechanism in different threads of the EtherCAT network with different cycle periods.

4. A rail vehicle multi-arm maintenance control method applied to the rail vehicle multi-arm maintenance control system of any one of claims 1-3, characterized in that, The mechanical arms, the walking mechanism, the motion execution mechanism, and the image acquisition mechanism run EtherCAT slave stations, the EtherCAT master station and the EtherCAT slave stations constitute different EtherCAT networks, and different EtherCAT networks use different DC synchronous clocks. The DC synchronous clock selects the clock of the first slave station in the EtherCAT network as the reference clock for synchronous operation, obtains the transmission delay and time offset of the remaining slave stations and the reference clock through the timestamp information in the EtherCAT information frame, obtains the preliminary synchronization time, and realizes the synchronization of the clocks of each slave station and the reference clock based on the preliminary synchronization time through periodic time drift compensation, thereby realizing the synchronous operation of each slave station. The EtherCAT master station is created by a multi-instance version of an API function in a process through multi-threading, different master stations are bound to different network card devices in the controller, and the clock synchronization between the master stations is realized through signal synchronization between threads. The defect detection unit is used for 2D coarse positioning of the maintenance images collected by the image acquisition mechanism in a model combining CNN and Transfromer, obtaining two-dimensional position information, performing accurate positioning based on the two-dimensional position information by using a 3D point cloud matching algorithm, obtaining six-dimensional position information, and returning the six-dimensional position information to the controller. The robot control code for controlling the mechanical arms is embedded in the master station in the form of a dynamic link library and is loaded in a segmented manner. When the master station runs to the PREOP state, the myECatSetup() function is called for initialization operation; after the master station is initialized, the myECatLoop() function is called to periodically run the get_args() of the dynamic link library, and the periodic data interaction with the slave station of the mechanical arm is realized. The rail vehicle multi-arm maintenance control system comprises a plurality of mechanical arms, a motion execution mechanism for moving the mechanical arms, an image acquisition mechanism mounted on the mechanical arms for collecting maintenance images, a walking mechanism for moving the rail vehicle multi-arm maintenance control system, a controller, and a defect detection unit; wherein a single-process multi-thread EtherCAT master station is run in the controller, and the controller is used for data interaction with each of the mechanical arms, the walking mechanism, the motion execution mechanism, and the image acquisition mechanism in different threads of the EtherCAT network with different cycle periods. The mechanical arms, the walking mechanism, the motion execution mechanism, and the image acquisition mechanism run EtherCAT slave stations, the EtherCAT master station and the EtherCAT slave stations constitute different EtherCAT networks, and different EtherCAT networks use different DC synchronous clocks. The DC synchronous clock selects the clock of the first slave station in the EtherCAT network as the reference clock for synchronous operation, obtains the transmission delay and time offset of the remaining slave stations and the reference clock through the timestamp information in the EtherCAT information frame, obtains the preliminary synchronization time, and realizes the synchronization of the clocks of each slave station and the reference clock based on the preliminary synchronization time through periodic time drift compensation, thereby realizing the synchronous operation of each slave station. The EtherCAT master station is created by a multi-instance version of an API function in a process through multi-threading, different master stations are bound to different network card devices in the controller, and the clock synchronization between the master stations is realized through signal synchronization between threads. The defect detection unit is used for 2D coarse positioning of the maintenance images collected by the image acquisition mechanism in a model combining CNN and Transfromer, obtaining two-dimensional position information, performing accurate positioning based on the two-dimensional position information by using a 3D point cloud matching algorithm, obtaining six-dimensional position information, and returning the six-dimensional position information to the controller. The robot control code for controlling the mechanical arms is embedded in the master station in the form of a dynamic link library and is loaded in a segmented manner. When the master station runs to the PREOP state, the myECatSetup() function is called for initialization operation; after the master station is initialized, the myECatLoop() function is called to periodically run the get_args() of the dynamic link library, and the periodic data interaction with the slave station of the mechanical arm is realized. Master station creation step: through the multi-instance version of the API function, multiple EtherCAT master stations are created in a process through multi-threading, and the EtherCAT master stations are bound to different network card devices in the controller; The defect detection step further comprises: inputting the inspection image into a model combining CNN and Transfromer for 2D coarse positioning to obtain two-dimensional position information, performing accurate positioning based on the two-dimensional position information by using a 3D point cloud matching algorithm to obtain six-dimensional position information, and returning the six-dimensional position information to the controller.

Images