A method for identifying the trapped state of rail vehicle bogie springs based on multi-sensor

Abstract

The present invention relates to the field of intelligent equipment for railway freight car maintenance, and specifically to a method for identifying the trapped state of a rail vehicle bogie pillow spring based on multiple sensors. A composite detection system composed of an image acquisition module, a point cloud acquisition module and a force acquisition module is used to realize three-dimensional posture detection of a trapped pillow spring and identification of a jamming situation between the trapped pillow spring and obstacles such as locating pins in a bogie side frame. By acquiring information such as the posture and trapped state of the trapped pillow spring, a disassembly and assembly robot can be controlled to complete the grasping of the trapped pillow spring, and path planning can be performed at the same time to avoid obstacles such as locating pins, so as to smoothly complete the disassembly and assembly operation of the pillow spring, thereby improving the automation and intelligence level of the maintenance equipment and reducing manual intervention during the maintenance process.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent equipment for railway freight car maintenance, and in particular to a method for identifying a trapped state of a bolster spring of a railway vehicle bogie based on multiple sensors. Background Art

[0002] At present, when disassembling and assembling the vibration damping device of the bogie, the pillow spring and the wedge are usually placed in or taken out of the frame by manual handling from the pallet. However, the weight of each pillow spring is more than 10kg, which leads to very high labor intensity and low efficiency for workers. Therefore, a set of pillow spring and wedge disassembly and assembly manipulator and bogie vibration damping device disassembly and assembly system are proposed. The six-axis robot arm is equipped with an end clamp to realize the disassembly and assembly of the wedge and pillow spring on the bogie, so as to reduce labor intensity, improve work efficiency and avoid safety accidents.

[0003] In most cases, the disassembly and assembly of the bogie vibration damping device can be completed normally by using a general detection system in conjunction with a disassembly and assembly system; however, due to the tilt and offset of the parking position of the bogie side frame, the various postures of the pillow spring, and the small and complex space of the side frame, when the manipulator clamps the pillow spring during the installation or disassembly operation, the pillow spring and obstacles such as the positioning pins in the bogie side frame may get stuck, and the pillow spring may be in a tilted state. At this time, the disassembly and assembly system cannot complete the disassembly and assembly of the bogie vibration damping device, and manual intervention is required to continue the maintenance task. Therefore, there is currently a lack of an intelligent identification method for the trapped state of the pillow spring, so that the maintenance robot can perceive the trapped state of the trapped pillow spring, which is convenient for the subsequent control of the manipulator to adjust the posture of the pillow spring and successfully complete the disassembly and installation of the pillow spring. Summary of the invention

[0004] In view of the above problems, an embodiment of the present invention provides a method for identifying a trapped state of a rail vehicle bogie bolster spring based on multiple sensors.

[0005] An embodiment of the present invention provides a method for identifying a trapped state of a rail vehicle bogie bolster spring based on multiple sensors, comprising:

[0006] The disassembly and assembly robot is moved to the calibration position, and the actual distance value corresponding to the pixel value on the pixel plane is determined by using the depth distance from the calibration position to the bogie side frame;

[0007] An image acquisition module installed on the disassembly and assembly robot is used to acquire an image containing the pillow spring, and the center of the pillow spring is determined by identifying the acquired image;

[0008] The pixel coordinates of the identified center are converted into actual coordinate information in the world coordinate system according to the actual distance value, and the disassembly and assembly robot is driven to perform visual servoing while keeping the coordinates in the depth direction unchanged according to the actual coordinate information until the center of the field of view of the image acquisition module coincides with the center of the pillow spring;

[0009] Continue to collect images including the pillow spring through the image acquisition module, and determine whether the pillow spring is in a horizontal direction according to the images obtained by continuing to collect;

[0010] When the pillow spring is in a horizontal direction, it is determined that the pillow spring is in a trapped state, otherwise it is determined that the pillow spring is in an untrapped state.

[0011] Optionally, the process of determining that the pillow spring is in an untrapped state includes:

[0012] The disassembly and assembly robot is driven to move the depth distance along the depth direction and grasp the pillow spring. During the grasping process, the force acquisition module on the disassembly and assembly robot determines whether the grasping force exceeds the threshold. If it does not exceed the threshold, it is determined that the pillow spring is in an untrapped state and the disassembly and assembly robot is driven to complete the disassembly and assembly operation. If it exceeds the threshold, it is determined that the pillow spring is in a trapped state.

[0013] Optionally, after determining that the pillow spring is in a trapped state, the method further includes:

[0014] The point cloud acquisition module on the disassembly and assembly robot scans to obtain the outer contour point cloud data of the pillow spring, and uses the outer contour point cloud data to fit the outer contour of the pillow spring and extract the center axis vector of the pillow spring and the center depth value of the pillow spring;

[0015] The position information of the pillow spring is integrated in the world coordinate system by combining the actual coordinate information of the pillow spring center, the axis vector of the pillow spring, the depth value of the pillow spring center and the spatial position information of the image acquisition module installation;

[0016] According to the actual size information of the bogie side frame, a point cloud model of the bogie side frame is imported into the point cloud space, and at the same time, according to the position information of the bolster spring, a point cloud model of the bolster spring is generated in the point cloud space;

[0017] In the point cloud space, the coordinate information of the overlapping portion between the point cloud model of the pillow spring and the point cloud model of the bogie side frame and the shape size information of the obstacle belonging to the overlapping portion are determined as the trapped state information.

[0018] Optionally, also include:

[0019] Based on the trapped state information, the path of the disassembly and assembly robot is planned after clamping the pillow spring, and the disassembly and assembly robot is driven according to the planned path to complete the disassembly and assembly operation.

[0020] Optionally, the step of determining whether the pillow spring is in a horizontal direction according to the image obtained by continuing to collect includes:

[0021] Determine whether the pillow spring in the image obtained by continuously acquiring is the end surface of the pillow spring. If it is the end surface of the pillow spring, determine that the pillow spring is in a horizontal direction.

[0022] Optionally, the step of determining whether the pillow spring is in a horizontal direction according to the image obtained by continuing to collect includes:

[0023] It is determined whether the long side of the bounding box of the pillow spring in the continuously acquired image is in the horizontal direction, and if so, it is determined that the pillow spring is in the horizontal direction.

[0024] Compared with the prior art, the beneficial effects of the present invention are: a composite detection system composed of an image acquisition module, a point cloud acquisition module and a force acquisition module is used to realize the three-dimensional posture detection of the trapped pillow spring and the identification of the jamming situation between the trapped pillow spring and obstacles such as locating pins in the bogie side frame; by obtaining information such as the posture and trapped state of the trapped pillow spring, it can be used to control the disassembly and assembly robot to complete the grasping of the trapped pillow spring, and at the same time perform path planning to avoid obstacles such as locating pins, and smoothly complete the disassembly and assembly operation of the pillow spring, thereby improving the automation and intelligence level of the maintenance equipment and reducing manual intervention during the maintenance process. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of the present application, and do not constitute a limitation of the present invention. In the drawings:

[0026] Figure 1 A schematic flow chart of a method for identifying a trapped state of a rail vehicle bogie bolster spring based on multiple sensors provided in an embodiment of the present invention;

[0027] Figure 2 A schematic diagram of a disassembly and assembly robot and installation positions of various modules provided in an embodiment of the present invention;

[0028] Figure 3 A schematic diagram of a pillow spring in an untrapped state provided by an embodiment of the present invention;

[0029] Figure 4 A schematic diagram of an inverted state of a pillow spring provided by an embodiment of the present invention;

[0030] Figure 5 A schematic diagram of another pillow spring in an inverted state provided by an embodiment of the present invention.

[0031] Figure 6 A schematic diagram of a pillow spring stuck state provided by an embodiment of the present invention DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0033] The embodiment of the present invention provides a method for identifying the trapped state of a rail vehicle bogie pillow spring based on a multi-sensor, which is applied to an identification system, wherein the identification system includes a robot module, an industrial computer, a PLC, a communication module, a visual system module, a force acquisition module, an image acquisition module, and a point cloud acquisition module; wherein the image acquisition module is composed of an industrial camera 1 and a light source, and the force acquisition module is composed of a force sensor 3 and a data acquisition card; the visual system module on the industrial computer is used to process the image captured by the industrial camera 1 and the point cloud obtained by scanning the line laser 2 of the cloud acquisition module; Figure 2 As shown, the industrial camera 1, the force sensor 3 and the line laser 2 are all arranged on the disassembly and assembly manipulator in the robot module; the specific identification process of the trapped state of the pillow spring is as follows: Figure 1 As shown, including:

[0034] Step 1: Move the disassembly and assembly robot to the calibration position, and use the depth distance from the calibration position to the bogie side frame to determine the actual distance value corresponding to the pixel value on the pixel plane.

[0035] Step 2: The image acquisition module installed on the disassembly and assembly robot acquires an image including the pillow spring, and the center of the pillow spring is determined by identifying the acquired image.

[0036] In implementation, after collecting the image, the rotation target detection algorithm based on deep learning can be used for image recognition to determine the center of the pillow spring, and the bounding box of the pillow spring can also be obtained.

[0037] Step 3: Convert the pixel coordinates of the identified center into actual coordinate information in the world coordinate system according to the actual distance value, and drive the disassembly and assembly robot to perform visual servoing while keeping the coordinates in the depth direction unchanged according to the actual coordinate information until the center of the field of view of the image acquisition module coincides with the center of the pillow spring.

[0038] Step 4: Continue to collect images containing the pillow spring through the image acquisition module, and determine whether the pillow spring is in a horizontal direction based on the images obtained by continuing to collect.

[0039] In implementation, when determining whether it is in the horizontal direction, it can be determined whether the pillow spring in the image acquired continuously is the end face of the pillow spring, or whether the long side in the boundary box of the pillow spring is in the horizontal direction. Specifically, Figure 4 As shown in FIG. 1 , the pillow spring in the acquired image is the end face of the pillow spring, or as shown in FIG. Figure 5As shown, the long side of the bounding box of the pillow spring in the acquired image is in the horizontal direction, which can determine that the pillow spring is in the horizontal state, that is, in an inverted state;

[0040] Step 5: When the pillow spring is in a horizontal direction, determine that the pillow spring is in a trapped state; otherwise, determine that the pillow spring is in an untrapped state.

[0041] In practice, the present invention determines whether the pillow spring is in a horizontal direction, that is, the first trapped state identification. If it is in a horizontal direction, it is determined that the pillow spring is in a trapped state; otherwise, it is determined that the pillow spring is in an untrapped state, for example, Figure 3 When the pillow spring is shown in an upright position, it is determined that the pillow spring is in an untrapped state.

[0042] In a specific implementation, when it is determined that the pillow spring is in an untrapped state, the disassembly and assembly robot is driven to move the depth distance in the depth direction and grab the pillow spring. During the grabbing process, the force acquisition module on the disassembly and assembly robot determines whether the grabbing force exceeds the threshold, that is, a secondary trapped state judgment is performed. If it does not exceed the threshold, it is determined that the pillow spring is in an untrapped state and the disassembly and assembly robot is driven to complete the disassembly and assembly operation. If it exceeds the threshold, it indicates that the pillow spring is stuck with obstacles such as locating pins in the bogie side frame, and therefore it can be determined that the pillow spring is in a trapped state.

[0043] After the pillow spring is determined to be in a trapped state by the first or second identification, the outer contour point cloud data of the pillow spring can be obtained by scanning through the point cloud acquisition module on the disassembly and assembly robot, and the outer contour of the pillow spring can be fitted by combining the outer contour point cloud data with the point cloud contour fitting algorithm provided by the PLC library, and the center axis vector of the pillow spring and the center depth value of the pillow spring can be extracted;

[0044] Combining the actual coordinate information of the center of the pillow spring, the axis vector of the pillow spring, the depth value of the center of the pillow spring and the spatial position information of the image acquisition module installation, the spatial position registration method is used to integrate the posture information of the pillow spring in the world coordinate system;

[0045] According to the actual size information of the bogie side frame, a point cloud model of the bogie side frame is imported into the point cloud space, and at the same time, according to the position information of the bolster spring, a point cloud model of the bolster spring is generated in the point cloud space;

[0046] In the point cloud space, the coordinate information of the overlapping portion between the point cloud model of the pillow spring and the point cloud model of the bogie side frame and the shape size information of the obstacle belonging to the overlapping portion are determined as the trapped state information.

[0047] During implementation, after obtaining the trapped state information, the path of the disassembly and assembly robot after clamping the pillow spring can be planned based on the trapped state information, and the disassembly and assembly robot can be driven according to the planned path to complete the disassembly and assembly operation.

[0048] The solution provided by the embodiment of the present invention utilizes a composite detection system composed of an image acquisition module, a point cloud acquisition module and a force acquisition module to realize three-dimensional posture detection of a trapped pillow spring and identification of a jamming condition between the trapped pillow spring and obstacles such as locating pins in the bogie side frame; by acquiring information such as the posture and trapped state of the trapped pillow spring, it can be used to control a disassembly and assembly robot to complete the grasping of the trapped pillow spring, while performing path planning to avoid obstacles such as locating pins, and smoothly complete the disassembly and assembly operation of the pillow spring, thereby improving the automation and intelligence level of maintenance equipment and reducing manual intervention during the maintenance process.

[0049] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A method for identifying a trapped state of a rail vehicle bogie bolster spring based on multiple sensors, characterized in that: include: The disassembly and assembly robot is moved to the calibration position, and the actual distance value corresponding to the pixel value on the pixel plane is determined by using the depth distance from the calibration position to the bogie side frame; An image acquisition module installed on the disassembly and assembly robot is used to acquire an image containing the pillow spring, and the center of the pillow spring is determined by identifying the acquired image; The pixel coordinates of the identified center are converted into actual coordinate information in the world coordinate system according to the actual distance value, and the disassembly and assembly robot is driven to perform visual servoing while keeping the coordinates in the depth direction unchanged according to the actual coordinate information until the center of the field of view of the image acquisition module coincides with the center of the pillow spring; Continue to collect images including the pillow spring through the image acquisition module, and determine whether the pillow spring is in a horizontal direction according to the images obtained by continuing to collect; When the pillow spring is in a horizontal direction, it is determined that the pillow spring is in a trapped state, otherwise it is determined that the pillow spring is in an untrapped state.

2. The method for identifying the trapped state of a rail vehicle bogie bolster spring based on multiple sensors according to claim 1, characterized in that: The process of determining that the bolster spring is in an untrapped state includes: The disassembly and assembly robot is driven to move the depth distance along the depth direction and grasp the pillow spring. During the grasping process, the force acquisition module on the disassembly and assembly robot determines whether the grasping force exceeds the threshold. If it does not exceed the threshold, it is determined that the pillow spring is in an untrapped state and the disassembly and assembly robot is driven to complete the disassembly and assembly operation. If it exceeds the threshold, it is determined that the pillow spring is in a trapped state.

3. The method for identifying the trapped state of a rail vehicle bogie bolster spring based on multiple sensors according to claim 1 or 2, characterized in that: After determining that the pillow spring is in a trapped state, also include: The point cloud acquisition module on the disassembly and assembly robot scans to obtain the outer contour point cloud data of the pillow spring, and uses the outer contour point cloud data to fit the outer contour of the pillow spring and extract the center axis vector of the pillow spring and the center depth value of the pillow spring; The position information of the pillow spring is integrated in the world coordinate system by combining the actual coordinate information of the pillow spring center, the axis vector of the pillow spring, the depth value of the pillow spring center and the spatial position information of the image acquisition module installation; According to the actual size information of the bogie side frame, a point cloud model of the bogie side frame is imported into the point cloud space, and at the same time, according to the position information of the bolster spring, a point cloud model of the bolster spring is generated in the point cloud space; In the point cloud space, the coordinate information of the overlapping portion between the point cloud model of the pillow spring and the point cloud model of the bogie side frame and the shape size information of the obstacle belonging to the overlapping portion are determined as the trapped state information.

4. The method for identifying the trapped state of a rail vehicle bogie bolster spring based on multiple sensors as claimed in claim 3, characterized in that: Also includes: Based on the trapped state information, the path of the disassembly and assembly robot is planned after clamping the pillow spring, and the disassembly and assembly robot is driven according to the planned path to complete the disassembly and assembly operation.

5. The method for identifying the trapped state of a rail vehicle bogie bolster spring based on multiple sensors as claimed in claim 1, characterized in that: The step of determining whether the pillow spring is in a horizontal direction according to the image obtained by continuing to collect includes: Determine whether the pillow spring in the image obtained by continuously acquiring is the end surface of the pillow spring. If it is the end surface of the pillow spring, determine that the pillow spring is in a horizontal direction.

6. The method for identifying the trapped state of a rail vehicle bogie bolster spring based on multiple sensors as claimed in claim 1, characterized in that: The step of determining whether the pillow spring is in a horizontal direction according to the image obtained by continuing to collect includes: It is determined whether the long side of the bounding box of the pillow spring in the continuously acquired image is in the horizontal direction, and if so, it is determined that the pillow spring is in the horizontal direction.

Images