A positioning and control method for a liquid loading and unloading operation mechanism of a special vehicle

Abstract

The invention discloses a positioning and control method for a liquid loading and unloading operation mechanism of a special purpose vehicle, relates to the field of positioning and control technology, and solves the problems of low efficiency, low safety factor, and low intelligence level of traditional liquid loading and unloading operations of special purpose vehicles; the positioning and control method comprises: obtaining visual information of a loading and unloading port on the top of the special purpose vehicle through a depth camera, and transmitting the information to an industrial computer in a control room; at the same time, turning on a laser radar on the top to perform point cloud mapping of the peripheral environment, the industrial computer running a deep learning module to identify the visual information, identify and locate the center position of the loading and unloading port, and obtain the coordinates under the basic coordinate system through coordinate transformation; constructing an environmental point cloud map based on rich point cloud data and realizing automatic obstacle avoidance operation; planning and optimizing a global path based on the obtained target position; and realizing that the loading and unloading operation mechanism can realize long-term automatic loading and unloading operations without human intervention.

Description

Technical Field

[0001] The invention relates to the technical field of positioning and control, and in particular to a positioning and control method for a liquid loading and unloading operation mechanism of a special vehicle. Background Art

[0002] The importance of special purpose vehicles lies in meeting the needs of specific industries for customized transportation and operations. Although the loading and unloading mechanism of special vehicles, especially the liquid loading and unloading mechanism of chemical special vehicles, has reached international standards after years of rapid development, the design and research of automated liquid loading and unloading mechanisms are relatively small. The intelligent research of its positioning and control system is of great significance to ensure personnel safety, improve the level of industrial automation and improve operation efficiency.

[0003] Manual operation is the most common and traditional way of loading and unloading materials. Before loading and unloading, the operator needs to accurately place the vertical pipe of the operating mechanism into the loading and unloading port of the special vehicle. After confirming that the vertical pipe is accurately placed in the loading and unloading port, the operator will open the pipeline valve system to start loading and unloading materials; after loading and unloading, the operator will close the pipeline valve system, stop loading and unloading materials, pull out the vertical pipe of the operating mechanism, and finally reset the operating mechanism.

[0004] Patent No. CN2022101987195 discloses a barbell recognition and tracking control method based on YOLO and improved template matching, which specifically includes the following steps: S1, collecting side views of various barbells as training data, labeling the training data, and dividing them into training sets and test sets; S2, standardizing the images in the training set, and using the YOLO algorithm to train the barbell recognition model by adjusting hyperparameters; S3, obtaining the original barbell video, detecting the first frame image of the barbell video through the barbell recognition model and locating the position of the barbell; S4, inputting the barbell position obtained from the first frame image and the original barbell video into the improved template matching algorithm, and calculating the barbell tracking video with the barbell trajectory drawn. Compared with the prior art, the above invention has the advantages of improving the accuracy of barbell recognition and being able to recognize barbells of different styles, improving frame rate and accuracy, etc.

[0005] Patent No. CN2021100189973 discloses a machine room inspection method, including the following steps: A. Input several inspection points, image shooting parameters, inspection content, inspection time, and generate a network topology map to the inspection robot; B. Perform calibration work; C. Perform inspection when the inspection time is up, otherwise, enter step I; D. Take an image of the indicator light group at the inspection point; E. Detect the status of the indicator light; F. Take an image of the device containing the air switch to be detected and locate the air switch to be detected; G. Input the image of the switch to be detected into the yolo model to obtain the position and status of each switch; H. Determine whether the state of the air switch is abnormal and whether there is any missed detection; I. Receive the alarm information of the network management system, control the inspection robot to reach the fault location, and repeat steps D and E. The above invention also provides a machine room inspection device. The present invention can generate an accurate network topology map, realize detection with strong anti-interference, low false detection rate, and high accuracy, and conduct a more comprehensive inspection of the machine room.

[0006] At present, material loading and unloading mainly relies on manpower, especially in the chemical industry and oil and gas facilities, which account for a large proportion; the operator needs to move to the loading and unloading port on the top of the special vehicle, accurately move the vertical pipe of the operating mechanism to the loading and unloading port, and finally remove the vertical pipe. This loading and unloading method consumes human resources, and the loading and unloading speed is slow and the efficiency is low. During the loading and unloading process, part of the material will be diffused into the air, the working environment is harsh, and it has potential harm to health; and manual judgment of the alignment position of the loading and unloading port and the vertical pipe of the operating mechanism is prone to certain errors, and the alignment accuracy is low, resulting in low accuracy of the entire material loading and unloading operation. Summary of the invention

[0007] The purpose of the present invention is to provide a positioning and control method for a liquid loading and unloading operation mechanism of a special purpose vehicle, which can realize the positioning and control of the liquid loading and unloading operation mechanism through a target detection model and a MoveIt function package, thereby improving the liquid loading and unloading speed and loading and unloading efficiency, and improving the alignment accuracy of the vertical pipes between the loading and unloading port and the loading and unloading operation mechanism.

[0008] The present invention utilizes the following technical solutions:

[0009] A method for positioning and controlling a liquid loading and unloading mechanism of a special vehicle comprises the following steps:

[0010] S1: Acquire image information of the loading and unloading port and working environment of the special vehicle, and transmit the acquired image information to the industrial computer of the control console;

[0011] S2: The industrial computer on the console performs deep learning training on the acquired image information to detect the loading and unloading ports of special vehicles;

[0012] S3: Calculate the target center position of the loading and unloading port according to the detected loading and unloading port to obtain the three-dimensional coordinates of the target center, and convert the three-dimensional coordinates of the target center to the basic coordinates of the loading and unloading operation mechanism to construct the target center coordinate pose;

[0013] S4: performing inverse kinematics calculation on the converted target center coordinate posture to obtain the inverse solution angles of each joint of the loading and unloading operation mechanism;

[0014] S5: During the operation of the loading and unloading operation mechanism, the working environment information is mapped in real time in three dimensions, and the movement path is planned based on the inverse angle;

[0015] S6: The underlying controller PLC manipulates the loading and unloading mechanism according to the obtained inverse angle and global motion path, moves to the target center coordinate position, and provides real-time feedback on the changes in the angles of each joint.

[0016] Preferably, in step S1, the image information of the loading and unloading port and the working environment of the special purpose vehicle is collected and processed by the information collection and processing module; the information collection and processing module includes an image collection unit, an information processing unit and a data transmission unit; the image collection unit uses a depth camera and a laser radar to obtain image information of the loading and unloading port and the working environment of the special purpose vehicle respectively; the information processing unit uses a Gaussian projection filter algorithm and a Euclidean octree algorithm to denoise the acquired image information; the data transmission unit uses a communication protocol to transmit the denoised image information to the console industrial computer.

[0017] Preferably, in step S2, the collected loading and unloading port images are annotated and trained by the deep recognition and positioning module to obtain the data weight matrix of the loading and unloading port; the deep recognition and positioning module includes a data enhancement unit, a data annotation unit and a model training unit; the data enhancement unit rotates, scales, adds noise, mirrors, flips and cuts and splices the collected loading and unloading port images using an image enhancement algorithm to expand the scale and style of the loading and unloading port images; the data annotation unit annotates the loading and unloading port images that have completed data enhancement using LabeImg software to form a training data set and a test data set; the model training unit first imports the training data set into the YOLO model for feature extraction and iterative training to obtain the data weight matrix of the loading and unloading port, and uses the test data set to continuously verify and update the acquired data weight matrix, thereby obtaining the optimal solution of the data weight matrix.

[0018] Preferably, the YOLO model first inputs the training data set into the feature extraction layer for channel expansion and deep feature extraction; the feature extraction layer includes 2 CBS blocks and 2 CBL blocks; the CBS block includes 1 3X3 convolution layer with a step size of 2, 1 batch standard layer and Silu activation function; the CBL block includes 1 3X3 convolution layer with a step size of 1, 1 batch standard layer and LeakyRelu activation function; then the extracted deep feature map is input into the backbone network layer for iterative training to obtain and verify the data weight matrix; the backbone network layer includes 3 C2F blocks, 3 CBL blocks, 3 CSP blocks and 1 SPPF block; the C2F block includes 1 CBL block, 1 segmentation layer and 3 BackBone blocks; the BackBone block includes two network branches; One network branch includes 4 CBL blocks and 2 combined convolution blocks consisting of 1X3 convolutional layers and 3X1 convolutional layers; the second network branch includes 2 CBS blocks; the two network branches are added and fused and then pass through 1 CBS block; the CSP block includes two branches, the first branch includes 1 CBS block, 4 residual blocks and 2 3X3 convolutional layers; the second branch includes 2 CBL blocks and 4 inverted residual blocks; the two branches are spliced ​​and pass through 1 batch standard layer and Relu activation function in turn; the SPPF block includes 1 CBL block, 3 residual structured maximum pooling layers and 1 CBS block; finally, the test data set is input into the network output layer to infer and verify the data weight matrix, and the recognition result is fed back; the network output layer includes ELAN-W block, MP block and upsampling block.

[0019] Preferably, in step S3, the depth camera coordinate system and the basic coordinates of the loading and unloading mechanism are converted by a coordinate conversion module to obtain the target center coordinate posture; the coordinate conversion module includes a base calibration unit and a coordinate transformation unit; the base calibration unit performs eye-out-of-hand calibration on the depth camera and the loading and unloading mechanism respectively to obtain the rotation matrix between the coordinate systems of the depth camera and the loading and unloading mechanism; the coordinate transformation unit uses the obtained rotation matrix to convert the position coordinates of the target center in the depth camera coordinate system into the target center coordinate posture in the basic coordinate system of the loading and unloading mechanism.

[0020] Preferably, in step S4, the converted target center coordinate posture is reversely analyzed in combination with the working process of the loading and unloading operation mechanism through a kinematic inverse analysis module to obtain the inverse solution angle of each joint of the loading and unloading operation mechanism; the kinematic inverse analysis module includes a mechanism modeling unit, a configuration transmission unit and a posture inverse solution unit; the mechanism modeling unit uses solidworks software to model the loading and unloading operation mechanism and convert it into urdf format; the posture inverse solution unit uses the MoveIt function package to perform a global path reverse analysis of the loading and unloading operation mechanism according to the target center coordinate posture and the loading and unloading operation mechanism model, and performs inverse solution on the motion angle of each joint; the configuration transmission unit uses the MoveIt configuration assistant to configure the self-collision matrix, motion planning group and mobile terminal of the loading and unloading operation mechanism model, and compresses it into a ROS function package for transmission to the graphical interface GUI.

[0021] Preferably, in step S5, the image information of the working environment of the loading and unloading operation mechanism is compiled into point cloud information through the path construction and planning module to construct an environmental point cloud map, and the working path of the loading and unloading operation mechanism is planned according to the environmental point cloud map; the path construction and planning module includes an environmental simulation unit and a path planning unit; the environmental simulation unit uses a point cloud mapping algorithm to compile the acquired image information of the working environment of the loading and unloading operation mechanism into point cloud information to construct an environmental point cloud map; the path planning unit uses a local roadblock planning algorithm to plan the motion path of the loading and unloading operation mechanism according to the global path environment of the inverse analysis, the motion angles of each joint that completes the inverse solution, and the point cloud map.

[0022] Preferably, the workflow of the MoveIt function package is:

[0023] A: Define the kinematic constraints, initial pose and target pose of the loading and unloading mechanism;

[0024] Kinematic constraints include joint constraints, position constraints and direction constraints; joint constraints include joint angle constraints, joint speed constraints and joint acceleration constraints; position constraints are used to limit the motion area of ​​the loading and unloading mechanism; direction constraints are used to limit the end direction of the loading and unloading mechanism; the initial posture is the current joint angle and posture of the loading and unloading mechanism, and the target posture is the target center coordinate posture;

[0025] B: Based on the initial and target positions of the loading and unloading mechanism, a global path search is performed using a path search algorithm;

[0026] C: During the path search process, the global path is restricted by combining the environmental point cloud map and kinematic constraints to ensure that the global path does not collide with obstacles;

[0027] D: After determining the final global path, the final global path is converted into a smooth motion trajectory using an interpolation algorithm;

[0028] E: Optimize and adjust the smooth motion trajectory after the conversion, and inversely solve it into a time-parameterized sequence of joint angles;

[0029] F: Convert the inverse solution of each joint angle sequence into a control signal and transmit it to the underlying controller PLC for execution.

[0030] Preferably, in step S6, the bottom communication operation module is used to control the console industrial computer and the bottom controller PLC to communicate, so as to manipulate the loading and unloading operation mechanism to move to the target center coordinate posture; the bottom communication operation module includes a signal encryption unit, a signal distribution unit and a signal feedback unit; the signal encryption unit uses a chaotic encryption algorithm to encrypt the planned motion path and the motion angles of each joint that have completed the inverse solution; the signal distribution unit uses a clustering algorithm to cluster and distribute the encrypted motion angles of each joint according to the time series and the joint type of the loading and unloading operation mechanism; the signal feedback unit uses an inertial sensor to feed back the motion process of the loading and unloading operation mechanism to the bottom controller PLC in real time according to the preset motion path and the motion angles of each joint.

[0031] A positioning and control device for a liquid loading and unloading mechanism of a special vehicle, comprising an information acquisition and processing module, a depth recognition and positioning module, a coordinate conversion module, a kinematic inverse analysis module, a path construction and planning module, and a bottom-level communication operation module; wherein,

[0032] An information collection and processing module is used to collect and process image information of the loading and unloading ports and working environment of special vehicles;

[0033] A depth recognition and positioning module is used to annotate and train the collected loading and unloading port images to obtain the data weight matrix of the loading and unloading port;

[0034] The coordinate conversion module is used to convert the depth camera coordinate system and the basic coordinates of the loading and unloading operation mechanism to obtain the target center coordinate position;

[0035] The kinematic inverse analysis module is used to perform inverse analysis on the converted target center coordinate posture in combination with the working process of the loading and unloading operation mechanism to obtain the inverse angles of each joint of the loading and unloading operation mechanism;

[0036] A path construction and planning module is used to compile the image information of the working environment of the loading and unloading operation mechanism into point cloud information to construct an environmental point cloud map, and at the same time plan the working path of the loading and unloading operation mechanism according to the environmental point cloud map;

[0037] The bottom-level communication operation module is used to control the communication between the console industrial computer and the bottom-level controller PLC, thereby manipulating the loading and unloading operation mechanism to move to the target center coordinate position.

[0038] The present invention uses the YOLO model to perform feature extraction and iterative training of the loading and unloading port to obtain a data weight matrix, and continuously verifies and updates the data weight matrix to obtain the optimal solution of the data weight matrix; through the MoveIt function package, the global path of the loading and unloading operation mechanism is reversely analyzed according to the target center coordinate posture and the loading and unloading operation mechanism model, and the motion angle of each joint is inversely solved; the liquid loading and unloading speed and loading and unloading efficiency are improved, the vertical pipe alignment accuracy of the loading and unloading port and the loading and unloading operation mechanism is improved, and the harm of the liquid loading and unloading process to the workers is reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0040] Figure 1 It is a principle block diagram of the positioning and control method of the liquid loading and unloading mechanism of special vehicles;

[0041] Figure 2 It is a flow chart of positioning and control method;

[0042] Figure 3 Schematic diagram of the initial position A and target position B of the liquid loading and unloading mechanism;

[0043] Figure 4 This is the configuration diagram of the MoveIt function package for liquid loading and unloading operations. DETAILED DESCRIPTION

[0044] The present invention is described in detail below with reference to the accompanying drawings and embodiments:

[0045] like Figures 1 to 4 As shown, the positioning and control method of a liquid loading and unloading operation mechanism of a special vehicle described in the present invention comprises the following steps:

[0046] S1: Acquire image information of the loading and unloading port and working environment of the special vehicle, and transmit the acquired image information to the industrial computer of the control console;

[0047] In the present invention, the image information of the loading and unloading port and the working environment of the special vehicle can be collected and processed through the information collection and processing module; the information collection and processing module includes an image collection unit, an information processing unit and a data transmission unit; the image collection unit uses a depth camera and a laser radar to obtain the image information of the loading and unloading port and the working environment of the special vehicle respectively; the information processing unit uses a Gaussian projection filter algorithm and a Euclidean octree algorithm to perform denoising on the acquired image information; the data transmission unit uses a communication protocol to transmit the denoised image information to the control console industrial computer;

[0048] S2: The industrial computer on the console performs deep learning training on the acquired image information to detect the loading and unloading ports of special vehicles;

[0049] In the present invention, the collected loading and unloading port images can be annotated and trained by the deep recognition and positioning module to obtain the data weight matrix of the loading and unloading port; the deep recognition and positioning module includes a data enhancement unit, a data annotation unit and a model training unit; the data enhancement unit rotates, scales, adds noise, mirrors, flips and cuts and splices the collected loading and unloading port images using an image enhancement algorithm to expand the scale and style of the loading and unloading port images; the data annotation unit annotates the loading and unloading port images that have completed data enhancement using LabeImg software to form a training data set and a test data set; the model training unit first imports the training data set into the YOLO model for feature extraction and iterative training to obtain the data weight matrix of the loading and unloading port, and uses the test data set to continuously verify and update the acquired data weight matrix, thereby obtaining the optimal solution of the data weight matrix;

[0050] S3: According to the detected loading and unloading port, the target center position of the loading and unloading port is calculated, the three-dimensional coordinates of the target center are obtained, and the three-dimensional coordinates of the target center are converted to the basic coordinates of the loading and unloading operation mechanism to construct the coordinate pose of the target center;

[0051] In the present invention, the coordinate conversion module can be used to convert the coordinate system of the depth camera and the basic coordinates of the loading and unloading operation mechanism to obtain the coordinate position and posture of the target center; the coordinate conversion module includes a base calibration unit and a coordinate conversion unit; the base calibration unit performs eye-out-of-hand calibration on the depth camera and the loading and unloading operation mechanism respectively to obtain the rotation matrix between the coordinate systems of the depth camera and the loading and unloading operation mechanism; the coordinate conversion unit uses the obtained rotation matrix to convert the position coordinates of the target center in the depth camera coordinate system into the coordinate position and posture of the target center in the basic coordinate system of the loading and unloading operation mechanism;

[0052] In this embodiment, the calibration package needs to be compiled and configured when the eye is calibrated outside the hand;

[0053] S4: performing inverse kinematics calculation on the converted target center coordinate posture to obtain the inverse solution angles of each joint of the loading and unloading operation mechanism;

[0054] In the present invention, the converted target center coordinate posture can be combined with the working process of the loading and unloading operation mechanism for reverse analysis through the kinematic inverse analysis module to obtain the inverse solution angle of each joint of the loading and unloading operation mechanism; the kinematic inverse analysis module includes a mechanism modeling unit, a configuration transmission unit and a posture inverse solution unit; the mechanism modeling unit uses solidworks software to model the loading and unloading operation mechanism and convert the URDF format; the posture inverse solution unit uses the MoveIt function package to perform a global path reverse analysis on the loading and unloading operation mechanism according to the target center coordinate posture and the loading and unloading operation mechanism model, and performs inverse solution on the motion angle of each joint; the configuration transmission unit uses the MoveIt configuration assistant to configure the self-collision matrix, motion planning group and mobile terminal of the loading and unloading operation mechanism model, and compress it into a ROS function package to transmit it to the graphical interface GUI;

[0055] In this embodiment, the GUI is an rviz plug-in interface, including Cartesian coordinate system, forward and inverse kinematics, path planning, path execution and planning scenes, and bidirectional communication with the MoveIt function package; the ROS function package also includes a ROS parameter server, and the ROS parameter server uses the URDF format to bidirectionally communicate with the MoveIt function package; the motion planning group communicates bidirectionally with the MoveIt function package through operations such as moving, picking and placing;

[0056] S5: During the operation of the loading and unloading operation mechanism, the working environment information is mapped in real time in three dimensions, and the movement path is planned based on the inverse angle;

[0057] In the present invention, the image information of the working environment of the loading and unloading operation mechanism can be compiled into point cloud information through the path construction and planning module to construct an environmental point cloud map, and the working path of the loading and unloading operation mechanism is planned according to the environmental point cloud map; the path construction and planning module includes an environmental simulation unit and a path planning unit; the environmental simulation unit uses a point cloud mapping algorithm to compile the acquired image information of the working environment of the loading and unloading operation mechanism into point cloud information to construct an environmental point cloud map; the path planning unit uses a local roadblock planning algorithm to plan the motion path of the loading and unloading operation mechanism according to the global path environment of the reverse analysis, the motion angles of each joint that completes the inverse solution, and the point cloud map;

[0058] In this embodiment, the workflow of the MoveIt function package is as follows:

[0059] A: Define the kinematic constraints, initial pose and target pose of the loading and unloading mechanism;

[0060] Kinematic constraints include joint constraints, position constraints and direction constraints; joint constraints include joint angle constraints, joint speed constraints and joint acceleration constraints; position constraints are used to limit the motion area of ​​the loading and unloading operation mechanism; direction constraints are used to limit the end direction of the loading and unloading operation mechanism; the initial posture is the current joint angle and posture of the loading and unloading operation mechanism, and the target posture is the target center coordinate posture;

[0061] B: Based on the initial and target positions of the loading and unloading mechanism, a global path search is performed using a path search algorithm;

[0062] C: During the path search process, the global path is restricted by combining the environmental point cloud map and kinematic constraints to ensure that the global path does not collide with obstacles;

[0063] D: After determining the final global path, the final global path is converted into a smooth motion trajectory using an interpolation algorithm;

[0064] E: Optimize and adjust the smooth motion trajectory after the conversion, and inversely solve it into a time-parameterized sequence of joint angles;

[0065] F: Convert the inverse solution of each joint angle sequence into a control signal and transmit it to the underlying controller PLC for execution;

[0066] S6: The bottom-level controller PLC manipulates the loading and unloading mechanism according to the obtained inverse angle and global motion path, moves to the target center coordinate position, and provides real-time feedback on the changes in the angles of each joint;

[0067] In the present invention, the communication between the control console industrial computer and the underlying controller PLC can be realized through the underlying communication operation module, so as to manipulate the loading and unloading operation mechanism to move to the target center coordinate posture; the underlying communication operation module includes a signal encryption unit, a signal distribution unit and a signal feedback unit; the signal encryption unit uses a chaotic encryption algorithm to encrypt the planned motion path and the motion angles of each joint that have completed the inverse solution; the signal distribution unit uses a clustering algorithm to cluster and distribute the encrypted motion angles of each joint according to the time series and the joint type of the loading and unloading operation mechanism; the signal feedback unit uses an inertial sensor to feed back the motion process of the loading and unloading operation mechanism to the underlying controller PLC in real time according to the preset motion path and the motion angles of each joint.

[0068] In this embodiment, the bottom controller PLC transmits the joint trajectory to the MoveIt function package in real time; the inertial sensor transmits the joint state to the MoveIt function package in real time, and at the same time, after the mechanical state is released, it is transmitted to the MoveIt function package in real time using TF coordinate transformation;

[0069] In the present invention, depth camera, laser radar, Gaussian projection filter algorithm, Euclidean octree algorithm, communication protocol, image enhancement algorithm, Labelmg software, solidworks software, MoveIt configuration assistant, point cloud mapping algorithm, self-collision matrix, motion planning group, terminal, local roadblock planning algorithm, chaos encryption algorithm, clustering algorithm and inertial sensor are all commonly used technologies in this field and will not be described in detail here.

[0070] Furthermore, the YOLO model first inputs the training data set into the feature extraction layer for channel expansion and deep feature extraction; the feature extraction layer includes 2 CBS blocks and 2 CBL blocks; the CBS block includes 1 3X3 convolution layer with a step size of 2, 1 batch standard layer and Silu activation function; the CBL block includes 1 3X3 convolution layer with a step size of 1, 1 batch standard layer and LeakyRelu activation function; then the extracted deep feature map is input into the backbone network layer for iterative training to obtain and verify the data weight matrix; the backbone network layer includes 3 C2F blocks, 3 CBL blocks, 3 CSP blocks and 1 SPPF block; the C2F block includes 1 CBL block, 1 segmentation layer and 3 BackBone blocks; the BackBone block includes two network branches; One network branch includes 4 CBL blocks and 2 combined convolution blocks consisting of 1X3 convolutional layers and 3X1 convolutional layers; the second network branch includes 2 CBS blocks; the two network branches are added and fused and then pass through 1 CBS block; the CSP block includes two branches, the first branch includes 1 CBS block, 4 residual blocks and 2 3X3 convolutional layers; the second branch includes 2 CBL blocks and 4 inverted residual blocks; the two branches are spliced ​​and then pass through 1 batch standard layer and Relu activation function in turn; the SPPF block includes 1 CBL block, 3 maximum pooling layers of residual structure and 1 CBS block; finally, the test data set is input into the network output layer to infer and verify the data weight matrix, and the recognition result is fed back; the network output layer includes ELAN-W block, MP block and upsampling block;

[0071] The working principle of the YOLO model is as follows: First, the input original image is processed using an image preprocessing algorithm, including operations such as image scaling, contrast transformation, grayscale, and normalization, to convert it into a data structure that can be processed by a neural network; the preprocessed data is input into the feature extraction layer, including two CBS (Convolution, BatchNorm, Silu) blocks and two CBL (Convolution, BatchNorm, LeakyReLU) blocks. The CBS block includes a 3x3 convolution layer with a stride of 2, a batch standard layer, and a Silu activation function to expand the depth and width of the input vector; the CBL block includes a 3x3 convolution layer with a stride of 1, a batch standard layer, and a LeakyReLU activation function for feature extraction and feature enhancement. These operations can effectively extract deep features of the image for subsequent classification, positioning, and recognition operations;

[0072] Then, the deep feature map extracted by the feature extraction layer is input into the backbone network layer for object detection. The backbone network layer includes 3 C2F (Conv2d to Flow) blocks, 3 CBL blocks, 3 CSP (Cross Stage Partial) blocks and 1 SPPF (Spatial Pyramid Pooling for Feature) block. The C2F block includes a CBL block, a segmentation layer and 3 BackBone blocks; the BackBone block includes two branches, the first branch includes 4 CBL blocks and 2 combined convolution blocks; the second branch includes 2 CBS blocks. The CSP block includes two branches for feature fusion. The SPPF block includes a CBL block, 3 residual structured maximum pooling layers and 1 CBS block to enhance the robustness of feature expression. These operations can effectively detect information such as the position, size and category of objects in the image; the detected object position, size and category information is output to the network output layer for classification and positioning operations, and the recognition results are output. The network output layer includes ELAN-W (Efficient Last-layer Anymorphic Network with Weighted loss) block, MP (MaxPooling) block and upsampling block to further enhance the robustness and robustness of feature expression;

[0073] Finally, the recognition results are fed back to the feature extraction layer and the backbone network layer. The network weights are optimized and adjusted through the error back propagation algorithm to improve the recognition performance and robustness of the model. The test data set is used to verify the model and tune the parameters to ensure the accuracy and stability of the model. The optimized model is used for object detection and target recognition in actual applications to achieve automation and real-time object detection.

[0074] The present invention also includes a positioning and control device for a liquid loading and unloading operation mechanism of a special vehicle, including an information collection and processing module, a depth recognition and positioning module, a coordinate conversion module, a kinematic inverse analysis module, a path construction and planning module, and a bottom-level communication operation module; wherein,

[0075] An information collection and processing module is used to collect and process image information of the loading and unloading ports and working environment of special vehicles;

[0076] A depth recognition and positioning module is used to annotate and train the collected loading and unloading port images to obtain the data weight matrix of the loading and unloading port;

[0077] The coordinate conversion module is used to convert the depth camera coordinate system and the basic coordinates of the loading and unloading operation mechanism to obtain the target center coordinate position;

[0078] The kinematic inverse analysis module is used to perform inverse analysis on the converted target center coordinate posture in combination with the working process of the loading and unloading operation mechanism to obtain the inverse angles of each joint of the loading and unloading operation mechanism;

[0079] A path construction and planning module is used to compile the image information of the working environment of the loading and unloading operation mechanism into point cloud information to construct an environmental point cloud map, and at the same time plan the working path of the loading and unloading operation mechanism according to the environmental point cloud map;

[0080] The bottom-level communication operation module is used to control the communication between the console industrial computer and the bottom-level controller PLC, thereby manipulating the loading and unloading operation mechanism to move to the target center coordinate position.

[0081] Example:

[0082] When the liquid loading and unloading mechanism of the special vehicle is working, the image acquisition unit of the information acquisition and processing module uses the depth camera and the laser radar to obtain the image information of the loading and unloading port and the working environment of the special vehicle; the information processing unit uses the Gaussian projection filter algorithm and the Euclidean octree algorithm to denoise the acquired image information; and the data transmission unit transmits the denoised image information to the control console industrial computer using the communication protocol;

[0083] The data enhancement unit of the deep recognition and positioning module uses the image enhancement algorithm to rotate, scale, add noise, mirror flip and cut and splice the collected loading and unloading port images to expand the scale and style of the loading and unloading port images; the data annotation unit uses the LabeImg software to annotate the loading and unloading port images that have completed data enhancement to form a training data set and a test data set; then the model training unit first imports the training data set into the YOLO model for feature extraction and iterative training to obtain the data weight matrix of the loading and unloading port, and uses the test data set to continuously verify and update the acquired data weight matrix to obtain the optimal solution of the data weight matrix; the YOLO model first inputs the training data set into the feature extraction layer for channel expansion and deep feature extraction; the feature extraction layer includes 2 CBS blocks and 2 CBL blocks; the CBS block includes 1 3X3 convolution layer with a step size of 2, 1 batch standard layer and Silu activation function; the CBL block includes 1 3X3 convolution layer with a step size of 1, 1 batch standard layer and LeakyRelu activation function; then the extracted deep feature map is input into the backbone network The layers are iteratively trained to obtain and verify the data weight matrix; the backbone network layer includes 3 C2F blocks, 3 CBL blocks, 3 CSP blocks and 1 SPPF block; the C2F block includes 1 CBL block, 1 segmentation layer and 3 BackBone blocks; the BackBone block includes two network branches; the first network branch includes 4 CBL blocks and 2 combined convolution blocks consisting of 1X3 convolutional layers and 3X1 convolutional layers; the second network branch includes 2 CBS blocks; the two network branches are added and fused through 1 CBS block ; The CSP block includes two branches. The first branch includes 1 CBS block, 4 residual blocks and 2 3X3 convolutional layers; the second branch includes 2 CBL blocks and 4 inverted residual blocks; the two branches are spliced ​​and passed through 1 batch standard layer and Relu activation function in turn; the SPPF block includes 1 CBL block, 3 residual structured maximum pooling layers and 1 CBS block; finally, the test data set is input into the network output layer to infer and verify the data weight matrix, and the recognition result is fed back; the network output layer includes ELAN-W block, MP block and upsampling block;

[0084] Then, the base calibration unit of the coordinate conversion module performs eye-out-of-hand calibration on the depth camera and the loading and unloading mechanism respectively to obtain the rotation matrix between the coordinate systems of the depth camera and the loading and unloading mechanism; the coordinate conversion unit uses the obtained rotation matrix to convert the position coordinates of the target center in the depth camera coordinate system into the target center coordinate pose of the basic coordinate system of the loading and unloading mechanism; the mechanism modeling unit of the kinematic inverse analysis module uses solidworks software to model the loading and unloading mechanism and convert it into urdf format; the pose inverse solution unit uses the MoveIt function package to perform global path reverse analysis of the loading and unloading mechanism according to the target center coordinate pose and the loading and unloading mechanism model, and inversely solves the motion angle of each joint; the configuration transmission unit uses the MoveIt configuration assistant to configure the self-collision matrix, motion planning group and mobile terminal of the loading and unloading mechanism model, and compresses them into a ROS function package to transmit them to the graphical interface GUI;

[0085] At the same time, the environmental simulation unit of the path construction and planning module uses the point cloud mapping algorithm to compile the image information of the working environment of the loading and unloading operation mechanism into point cloud information to build an environmental point cloud map; the path planning unit uses the local roadblock planning algorithm to plan the motion path of the loading and unloading operation mechanism based on the global path environment resolved by reverse analysis, the motion angles of each joint that have completed the inverse solution, and the point cloud map; the MoveIt function package defines the kinematic constraints, initial posture, and target posture of the loading and unloading operation mechanism;

[0086] Kinematic constraints include joint constraints, position constraints and direction constraints; joint constraints include joint angle constraints, joint speed constraints and joint acceleration constraints; position constraints are used to limit the motion area of ​​the loading and unloading mechanism; direction constraints are used to limit the terminal direction of the loading and unloading mechanism; the initial posture is the current joint angle and posture of the loading and unloading mechanism, and the target posture is the target center coordinate posture; according to the initial posture and target posture of the loading and unloading mechanism, a path search algorithm is used to perform a global path search; during the path search process, the global path is restricted in combination with the environmental point cloud map and kinematic constraints to ensure that the global path does not collide with obstacles; after determining the final global path, the interpolation algorithm is used to convert the final global path into a smooth motion trajectory; the smooth motion trajectory that has completed the conversion is optimized and adjusted, and at the same time, the inverse solution is converted into a time-parameterized sequence of joint angles; the inverse solution of each joint angle sequence is converted into a control signal and transmitted to the underlying controller PLC for execution;

[0087] Finally, the signal encryption unit of the underlying communication operation module uses a chaotic encryption algorithm to encrypt the planned motion path and the motion angles of each joint that have completed the inverse solution; the signal distribution unit uses a clustering algorithm to cluster and distribute the encrypted motion angles of each joint according to the time series and the joint type of the loading and unloading operation mechanism; the signal feedback unit uses an inertial sensor to feed back the motion process of the loading and unloading operation mechanism to the underlying controller PLC in real time based on the preset motion path and the motion angles of each joint.

Claims

1. A method for positioning and controlling a liquid loading and unloading mechanism of a special vehicle, characterized in that: The steps include: S1: Acquire image information of the loading and unloading port and working environment of the special vehicle, and transmit the acquired image information to the industrial computer of the control console; S2: The industrial computer on the console performs deep learning training on the acquired image information to detect the loading and unloading ports of special vehicles; S3: Calculate the target center position of the loading and unloading port according to the detected loading and unloading port to obtain the three-dimensional coordinates of the target center, and convert the three-dimensional coordinates of the target center to the basic coordinates of the loading and unloading operation mechanism to construct the target center coordinate pose; S4: performing inverse kinematics calculation on the converted target center coordinate posture to obtain the inverse solution angles of each joint of the loading and unloading operation mechanism; S5: During the operation of the loading and unloading operation mechanism, the working environment information is mapped in real time in three dimensions, and the movement path is planned based on the inverse angle; S6: The bottom-level controller PLC manipulates the loading and unloading mechanism according to the obtained inverse angle and global motion path, moves to the target center coordinate position, and provides real-time feedback on the changes in the angles of each joint; In step S2, the collected loading and unloading port images are annotated and trained by the deep recognition and positioning module to obtain the data weight matrix of the loading and unloading port; the deep recognition and positioning module includes a data enhancement unit, a data annotation unit and a model training unit; the data enhancement unit rotates, scales, adds noise, mirrors, flips and cuts and splices the collected loading and unloading port images using an image enhancement algorithm to expand the scale and style of the loading and unloading port images; The data annotation unit annotates the loading and unloading port images that have completed data enhancement using LabeImg software to form a training data set and a test data set; The model training unit first imports the training data set into the YOLO model for feature extraction and iterative training to obtain the data weight matrix of the loading and unloading port. At the same time, the obtained data weight matrix is ​​continuously verified and updated using the test data set to obtain the optimal solution of the data weight matrix.

2. The positioning and control method of the liquid loading and unloading mechanism of a special vehicle according to claim 1 is characterized in that: In the step S1, the image information of the loading and unloading port and the working environment of the special vehicle is collected and processed by the information collection and processing module; the information collection and processing module includes an image collection unit, an information processing unit and a data transmission unit; the image collection unit uses a depth camera and a laser radar to obtain image information of the loading and unloading port and the working environment of the special vehicle respectively; the information processing unit uses a Gaussian projection filter algorithm and a Euclidean octree algorithm to denoise the acquired image information; the data transmission unit uses a communication protocol to transmit the denoised image information to the console industrial computer.

3. The positioning and control method of the liquid loading and unloading mechanism of a special vehicle according to claim 1, characterized in that: The YOLO model first inputs the training data set into the feature extraction layer for channel expansion and deep feature extraction; the feature extraction layer includes 2 CBS blocks and 2 CBL blocks; the CBS block includes 1 3X3 convolution layer with a step size of 2, 1 batch standard layer and Silu activation function; the CBL block includes 1 3X3 convolution layer with a step size of 1, 1 batch standard layer and LeakyRelu activation function; then the extracted deep feature map is input into the backbone network layer for iterative training to obtain and verify the data weight matrix; the backbone network layer includes 3 C2F blocks, 3 CBL blocks, 3 CSP blocks and 1 SPPF block; the C2F block includes 1 CBL block, 1 segmentation layer and 3 BackBone blocks; the BackBone block includes two network branches; the first The first network branch includes 4 CBL blocks and 2 combined convolution blocks consisting of 1X3 convolutional layers and 3X1 convolutional layers; the second network branch includes 2 CBS blocks; the two network branches are added and fused and then pass through 1 CBS block; the CSP block includes two branches, the first branch includes 1 CBS block, 4 residual blocks and 2 3X3 convolutional layers; the second branch includes 2 CBL blocks and 4 inverted residual blocks; the two branches are spliced ​​and then pass through 1 batch standard layer and Relu activation function in turn; the SPPF block includes 1 CBL block, 3 maximum pooling layers of residual structure and 1 CBS block; finally, the test data set is input into the network output layer to infer and verify the data weight matrix, and the recognition result is fed back; the network output layer includes ELAN-W block, MP block and upsampling block.

4. The positioning and control method of the liquid loading and unloading mechanism of a special vehicle according to claim 1, characterized in that: In the step S3, the coordinate conversion module is used to convert the coordinate system of the depth camera and the basic coordinates of the loading and unloading operation mechanism to obtain the target center coordinate pose; the coordinate conversion module includes a base calibration unit and a coordinate transformation unit; the base calibration unit performs eye-out-of-hand calibration on the depth camera and the loading and unloading operation mechanism respectively to obtain the rotation matrix between the coordinate systems of the depth camera and the loading and unloading operation mechanism; The coordinate transformation unit uses the obtained rotation matrix to transform the position coordinates of the target center in the depth camera coordinate system into the target center coordinate posture in the basic coordinate system of the loading and unloading operation mechanism.

5. The positioning and control method of the liquid loading and unloading mechanism of a special vehicle according to claim 1, characterized in that: In the step S4, the converted target center coordinate posture is reversely analyzed in combination with the working process of the loading and unloading operation mechanism through the kinematic inverse analysis module to obtain the inverse solution angle of each joint of the loading and unloading operation mechanism; the kinematic inverse analysis module includes a mechanism modeling unit, a configuration transmission unit and a posture inverse solution unit; the mechanism modeling unit uses solidworks software to model the loading and unloading operation mechanism and converts it into urdf format; the posture inverse solution unit uses the MoveIt function package to perform a global path reverse analysis on the loading and unloading operation mechanism according to the target center coordinate posture and the loading and unloading operation mechanism model, and performs inverse solution on the motion angle of each joint; the configuration transmission unit uses the MoveIt configuration assistant to configure the self-collision matrix, motion planning group and mobile terminal of the loading and unloading operation mechanism model, and compresses them into a ROS function package to transmit them to the graphical interface GUI.

6. The positioning and control method of the liquid loading and unloading mechanism of a special vehicle according to claim 1, characterized in that: In step S5, the image information of the working environment of the loading and unloading operation mechanism is compiled into point cloud information through the path construction and planning module to construct an environmental point cloud map, and the working path of the loading and unloading operation mechanism is planned according to the environmental point cloud map; the path construction and planning module includes an environmental simulation unit and a path planning unit; the environmental simulation unit uses a point cloud mapping algorithm to compile the acquired image information of the working environment of the loading and unloading operation mechanism into point cloud information to construct an environmental point cloud map; the path planning unit uses a local roadblock planning algorithm to plan the motion path of the loading and unloading operation mechanism according to the global path environment of the reverse analysis, the motion angles of each joint that completes the inverse solution, and the point cloud map.

7. The method for positioning and controlling the liquid loading and unloading mechanism of a special vehicle according to claim 5, characterized in that: The workflow of the MoveIt function package is as follows: A: Define the kinematic constraints, initial pose and target pose of the loading and unloading mechanism; Kinematic constraints include joint constraints, position constraints and direction constraints; joint constraints include joint angle constraints, joint speed constraints and joint acceleration constraints; position constraints are used to limit the motion area of ​​the loading and unloading mechanism; direction constraints are used to limit the end direction of the loading and unloading mechanism; the initial posture is the current joint angle and posture of the loading and unloading mechanism, and the target posture is the target center coordinate posture; B: Based on the initial and target positions of the loading and unloading mechanism, a global path search is performed using a path search algorithm; C: During the path search process, the global path is restricted by combining the environmental point cloud map and kinematic constraints to ensure that the global path does not collide with obstacles; D: After determining the final global path, the final global path is converted into a smooth motion trajectory using an interpolation algorithm; E: Optimize and adjust the smooth motion trajectory after the conversion, and inversely solve it into a time-parameterized sequence of joint angles; F: Convert the inverse solution of each joint angle sequence into a control signal and transmit it to the underlying controller PLC for execution.

8. The method for positioning and controlling the liquid loading and unloading mechanism of a special vehicle according to claim 1, characterized in that: In the step S6, the bottom communication operation module is used to control the console industrial computer and the bottom controller PLC to communicate, so as to manipulate the loading and unloading operation mechanism to move to the target center coordinate posture; the bottom communication operation module includes a signal encryption unit, a signal distribution unit and a signal feedback unit; the signal encryption unit uses a chaotic encryption algorithm to encrypt the planned motion path and the motion angles of each joint that have completed the inverse solution; the signal distribution unit uses a clustering algorithm to cluster and distribute the encrypted motion angles of each joint according to the time series and the joint type of the loading and unloading operation mechanism; the signal feedback unit uses an inertial sensor to feed back the motion process of the loading and unloading operation mechanism to the bottom controller PLC in real time according to the preset motion path and the motion angles of each joint.

9. The positioning and control device for a liquid loading and unloading mechanism according to any one of claims 1 to 8, characterized in that: It includes information collection and processing module, depth recognition and positioning module, coordinate conversion module, kinematic inverse analysis module, path construction and planning module, and underlying communication operation module; among them, An information collection and processing module is used to collect and process image information of the loading and unloading ports and working environment of special vehicles; A depth recognition and positioning module is used to annotate and train the collected loading and unloading port images to obtain the data weight matrix of the loading and unloading port; The coordinate conversion module is used to convert the depth camera coordinate system and the basic coordinates of the loading and unloading operation mechanism to obtain the target center coordinate position; The kinematic inverse analysis module is used to perform inverse analysis on the converted target center coordinate posture in combination with the working process of the loading and unloading operation mechanism to obtain the inverse angles of each joint of the loading and unloading operation mechanism; A path construction and planning module is used to compile the image information of the working environment of the loading and unloading operation mechanism into point cloud information to construct an environmental point cloud map, and at the same time plan the working path of the loading and unloading operation mechanism according to the environmental point cloud map; The bottom-level communication operation module is used to control the communication between the console industrial computer and the bottom-level controller PLC, thereby manipulating the loading and unloading operation mechanism to move to the target center coordinate position.

Images