A method for planning the cleaning path of a TBM cleaning robot, including a cleaning system and computer device, and considering personnel early warning in the cleaning area.

By acquiring information on slag accumulation and personnel status in TBM tunnels, and combining this with trajectory prediction and path planning, the problem of insufficient path flexibility of slag removal robots has been solved, improving the efficiency of slag removal and tunnel construction, and ensuring personnel safety.

CN120538508BActive Publication Date: 2026-11-17CHINA RAILWAY HI TECH IND CORP LTD +1
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202510404439.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2025-04-01
Publication Date
2026-11-17
Estimated Expiration
2045-04-01

AI Technical Summary

Technical Problem

Existing TBM tunnel cleaning methods have low cleaning efficiency, resulting in low TBM tunnel construction efficiency. Furthermore, the cleaning robots lack flexibility in their cleaning paths and cannot effectively avoid human workers.

Method used

By acquiring information on the distribution of slag and the status of personnel in the slag removal area, a movement trajectory prediction model is used to predict the movement trajectory of personnel. Combined with a path planning algorithm, the slag removal robot's slag removal path is planned to avoid dynamic and static obstacles, thus achieving flexible slag removal operations.

Benefits of technology

It improved the efficiency of slag removal operations, ensured personnel safety, enhanced the efficiency of TBM tunnel construction, and avoided work stoppages caused by personnel avoiding obstacles.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120538508B_ABST
    Figure CN120538508B_ABST
Patent Text Reader

Abstract

This invention belongs to the field of tunnel construction technology, specifically relating to a method and system for planning the path of a TBM (Tunnel Boring Machine) cleaning robot, considering personnel early warning in the cleaning area, and a computer device. The method includes: S1, acquiring information on the distribution of accumulated debris and the status of personnel in the cleaning operation area; personnel include moving personnel and personnel stationary; S2, inputting the status information of moving personnel into a pre-established movement trajectory prediction model to obtain the movement trajectory of the moving personnel within a set time period; S3, using moving personnel following the movement trajectory as dynamic obstacles and stationary personnel as static obstacles, employing a path planning algorithm based on the accumulated debris distribution information to obtain the cleaning operation path of the cleaning robot. This invention solves the technical problem of low cleaning efficiency in existing TBM tunnel cleaning methods, which leads to low efficiency in TBM tunnel construction.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of tunnel construction technology, specifically relating to a method for planning the path of a TBM cleaning robot that takes into account personnel early warning in the cleaning area, as well as a cleaning system and computer device. Background Technology

[0002] During TBM excavation, rock bursts and roof collapses are common occurrences, leading to severe debris accumulation at the tunnel bottom. If debris and rocks are not cleared promptly, delays in arch assembly, invert block installation, and track laying will hinder construction progress and affect support quality. Debris removal is primarily done manually, resulting in low efficiency, a poor working environment, and high risks. Currently, extensive research has been conducted both domestically and internationally on automatic debris removal devices for rapid and effective debris removal. However, these studies focus on the device design and specific automatic debris removal processes, lacking case studies on the control and planning of debris removal actions in environments where automatic debris removal robots and workers coexist. In actual TBM construction, due to TBM construction procedures such as arch erection and cutterhead replacement, and the narrow, unstructured space at the tunnel bottom, it is inevitable that workers will pass through or linger in the debris removal area.

[0003] A Chinese invention patent application with publication number CN116901064A and publication date of October 20, 2023, discloses a tunnel bottom debris cleaning system suitable for TBM construction. This system includes a debris cleaning machine controller, which selects different debris cleaning paths based on the operating speed of the debris cleaning robot and controls the robot's debris cleaning operation. It also includes a vision system for acquiring images of the debris cleaning area and detecting the presence of people. When people are detected in the debris cleaning area, the controller prevents the robot from starting and / or stops the robot in operation via an emergency stop system. This technical solution has the following problems: First, the debris cleaning path of the robot is preset in the controller, limiting its operation to a fixed path and lacking flexibility. Second, the handling of personnel in the debris cleaning area is limited to stopping debris cleaning only when someone is present. While this improves safety, it significantly impacts the efficiency of debris cleaning, thus affecting the efficiency of TBM tunnel construction. Summary of the Invention

[0004] The purpose of this invention is to provide a TBM cleaning robot path planning method, cleaning system, and computer device that takes into account personnel early warning in the cleaning area, so as to solve the technical problem that the existing TBM tunnel cleaning methods have low cleaning efficiency, which leads to low efficiency in TBM tunnel construction.

[0005] To solve the above-mentioned technical problems, the present invention provides a method for planning the cleaning path of a TBM cleaning robot that considers personnel early warning in the cleaning area. The method includes:

[0006] S1. Obtain information on the distribution of accumulated slag and the status of personnel in the slag removal operation area; the personnel include mobile personnel and personnel moving in place.

[0007] S2. Input the status information of the mobile person into the pre-established movement trajectory prediction model to obtain the movement trajectory of the mobile person within a set time period.

[0008] S3. Using the moving personnel moving according to the moving trajectory as dynamic obstacles and the stationary personnel as static obstacles, a path planning algorithm is used to plan the path of the slag cleaning robot according to the slag distribution information, so that the slag cleaning robot can avoid the dynamic obstacles and static obstacles when cleaning slag according to the slag cleaning path.

[0009] The beneficial effects of the above technical solution are as follows: The technical solution of the TBM slag removal robot path planning method considering personnel early warning in the slag removal area according to the present invention belongs to an improved invention. When planning the slag removal path, the present invention not only considers the current distribution of personnel in the slag removal operation area, but also comprehensively considers the personnel movement trajectory over a future period. This allows the slag removal robot to flexibly avoid the current location and upcoming location of personnel during slag removal operations, without needing to stop slag removal operations when personnel are present in the slag removal operation area, greatly improving the efficiency of slag removal operations, and consequently improving the efficiency of TBM construction. Furthermore, since the slag removal avoids areas where personnel may work, there is no need for personnel to avoid the area, ensuring the efficiency of personnel operations. The present invention solves the technical problem of low slag removal efficiency in existing TBM tunnel slag removal methods, which leads to low efficiency in TBM tunnel construction.

[0010] Furthermore, the personnel status information includes a safety attribute space: the safety attribute space refers to a space constructed with the personnel's location as the origin to ensure that the slag removal robot will not collide with the personnel;

[0011] The scope of the dynamic obstacle in S3 includes the safety attribute space of moving personnel, and the scope of the static obstacle includes the safety attribute space of stationary personnel.

[0012] Furthermore, the position is (x i ,y i The safety attribute space f of personnel i) i (x, y) is determined by a two-dimensional Gaussian function:

[0013]

[0014] Among them, A p The amplitude is predetermined; and Let θ be the standard deviation of the Gaussian function. i Let i be the orientation angle of person i.

[0015] Furthermore, the safety attribute space of the person moving at time p is:

[0016]

[0017] in, θ = arctan[(yy i ) / (xx i )]; A p The amplitude is predetermined; (x) i ,y i ) represents the spatial location of the moving person i; Let σ be the direction of the velocity of person i moving at time p; F σ B σ L and σ R Two-dimensional Gaussian functions along direction, opposite direction, with The left side in the vertical direction and with The standard deviation on the right side in the vertical direction.

[0018] Furthermore, the σ F σ B σ L and σ R Determined according to the following formula:

[0019]

[0020] Where σ is the initial safe space size of the mobile personnel i; r is the mechanism range of the slag removal robot; c is the current size of the TBM tunnel space; v is the moving speed of the mobile personnel i; f F f B f L and f R All are proportionality coefficients.

[0021] Furthermore, the movement trajectory prediction model is a generative adversarial network based on LSTM.

[0022] Furthermore, S1 also includes acquiring obstacle information in the slag removal operation area; correspondingly, in S3, the obstacles are used as static obstacles for path planning.

[0023] Furthermore, the process of obtaining personnel status information in S1 includes: inputting the image information of the slag removal operation area into the pre-trained personnel detection model to obtain the personnel location information and safety helmet location information in the slag removal operation area;

[0024] If no safety helmet is detected in the area where personnel were detected, a safety alarm will be triggered and the cleaning operation will be stopped until a safety helmet is detected in the area or no personnel are found in the area.

[0025] Furthermore, the process of obtaining personnel status information in S1 also includes: when a safety helmet is detected in the area where a person is detected, if the distance the person moves within a unit of time is less than a preset distance threshold, then the person is considered to be a person moving in place; otherwise, the person is considered to be a person moving.

[0026] Furthermore, the path planning algorithm is either the A* algorithm or the RRT algorithm.

[0027] This invention also provides a technical solution for a TBM slag removal system: a TBM slag removal system, including a sensing module and a data processing module; the sensing module is used to acquire slag distribution information and personnel status information in the slag removal operation area; the data processing module is used to execute a computer program to implement the steps of the slag removal path planning method for the TBM slag removal robot that considers personnel early warning in the slag removal area as described above.

[0028] The present invention also provides a technical solution for a computer device: a computer device including a processor, the processor being used to execute a computer program to implement the steps of the TBM slag cleaning robot slag cleaning path planning method as described above, which takes into account personnel early warning in the slag cleaning area. Attached Figure Description

[0029] Figure 1 This is a schematic diagram of the hardware layout of the slag removal system in an embodiment of the slag removal path planning method for a TBM slag removal robot that considers personnel early warning in the slag removal area according to the present invention.

[0030] Figure 2 This is a flowchart illustrating an embodiment of the TBM slag cleaning robot path planning method considering personnel early warning in the slag cleaning area according to the present invention.

[0031] Figure 3 This is a personnel observation coordinate system for an embodiment of the TBM slag cleaning robot path planning method considering personnel early warning in the slag cleaning area according to the present invention;

[0032] Figure 4 This is a safety attribute space diagram of an embodiment of the TBM slag cleaning robot path planning method for considering personnel early warning in the slag cleaning area according to the present invention.

[0033] Figure 5 This invention provides a personnel trajectory prediction model for a TBM cleaning robot path planning method that considers personnel early warning in the cleaning area, as an embodiment of the present invention.

[0034] Figure 6 This is a flowchart of the A* algorithm-based method for a TBM cleaning robot with early warning of personnel in the cleaning area, as described in this invention.

[0035] Figure 7 This invention relates to a method for planning the path of a TBM cleaning robot that considers personnel early warning in the cleaning area, specifically a group detection module in the personnel trajectory prediction model.

[0036] Among them, 1. TBM main beam; 2. Slag removal robot; 3. Slag accumulation; 4. Vision sensor. Detailed Implementation

[0037] This invention, when planning the slag removal path, considers not only the current distribution of personnel within the slag removal area but also their future movement trajectories. This allows the slag removal robot to flexibly avoid the current and upcoming locations of personnel, eliminating the need to stop slag removal when personnel are present in the area, significantly improving slag removal efficiency and consequently, TBM construction efficiency. Furthermore, because the slag removal avoids areas where personnel might work, personnel are not required to avoid these areas, ensuring efficiency during personnel operations. This invention solves the technical problem of low slag removal efficiency in existing TBM tunnel slag removal methods, which leads to low TBM tunnel construction efficiency.

[0038] Example of a TBM slag removal robot path planning method considering personnel early warning in the slag removal area:

[0039] The TBM cleaning robot's cleaning path planning method, which considers personnel early warning in the cleaning area, in this embodiment can be based on, for example, Figure 1 The slag removal system shown is implemented as follows: Specifically, the system includes a slag removal robot 2, a vision recognition module, a movement trajectory prediction module, and a path planning module. The vision recognition module acquires information on the distribution of slag in the slag removal area and the status of personnel, including both moving and stationary personnel. The movement trajectory prediction module inputs the status information of moving personnel into a pre-established movement trajectory prediction model to obtain their movement trajectories over a predetermined time period. The path planning module uses the movement trajectories as dynamic obstacles and stationary personnel as static obstacles, employing a path planning algorithm based on the slag distribution information to determine the slag removal robot's slag removal path. The slag removal robot executes the slag removal actions according to the path planning module's output path to perform the slag removal.

[0040] Specifically, the visual recognition module includes a visual sensor 4 installed on the TBM main beam 1. The visual sensor 4 is installed with a downward viewing angle, and needs to be able to acquire image information of the entire slag removal area at the bottom of the tunnel. Based on the acquired image information, a pre-established target detection model is used to identify the personnel status information and the slag distribution information 3 in the slag removal operation area. Specifically, for the slag distribution information, a slag recognition model (a target detection model for identifying slag) is constructed, and the slag depth information is obtained to determine the distribution range and amount of slag at the bottom of the tunnel.

[0041] The number of visual sensors 4 can be one, two or more. When there is more than one visual sensor 4, an image stitching algorithm needs to be configured to integrate the image information obtained by different visual sensors 4 into one image.

[0042] A method for planning the cleaning path of a TBM cleaning robot that considers personnel early warning in the cleaning area, such as... Figure 2 As shown, the method includes:

[0043] S1. Obtain information on the distribution of accumulated slag and the status of personnel in the slag removal operation area; personnel include moving personnel and personnel moving in place.

[0044] It also includes obtaining information on obstacles in the slag removal area.

[0045] Information on slag distribution, personnel status, and obstacles is obtained by acquiring images of the slag removal area. Based on the acquired image information, a pre-established target detection model is used to extract the above information from the images.

[0046] Specifically, regarding the information on the distribution of accumulated debris, a debris recognition model (a target detection model for identifying debris) is constructed, along with methods for obtaining debris depth information. This yields the distribution range and quantity of debris at the bottom of the tunnel. Combined with the debris-cleaning robot's single-cycle cleaning capability and cleaning efficiency, a debris-cleaning target point cluster {G(i)|G1,G2,…Gn,i=1,2,…,n} is generated for the debris-cleaning task. Here, n represents the number of debris points identified by the debris recognition model.

[0047] For obstacle information, obstacles within the knowable range of the slag removal device mainly consist of mechanical structures or hardware such as equipment placed in the slag removal area (excluding personnel). Identified obstacles are marked as follows: ob(m) = {obs(m), obg(m)}, composed of the spatial position obs(m) and the obstacle's center of gravity and angle obg(m). Wherein, the spatial position obs(m) = {obs(x)} i ,y i ,z i), i=1,2,…,m}, the centroid coordinates and angles (such as Euler angles) of obstacle recognition are obg(m)={obg(g i ,θ i ),i=1,2,…,m}, and subsequently, when using spatial path planning algorithms such as A* and RRT, the identified obstacle information is added to the obstacle space.

[0048] For personnel status information, a pre-trained personnel detection model is used to identify the personnel status information in the slag cleaning operation area. Two types of labels, "people" and "hat", are pre-set. After a person is identified, it is determined whether a safety helmet is detected in the area where the person is located. If no safety helmet is detected, a prompt is made through loudspeakers, lights, etc., and the slag cleaning machine operation is paused until a safety helmet is worn normally (i.e., a safety helmet is detected in the area) or there are no personnel in the area where the person is located.

[0049] When a person is detected in the designated area and is wearing a safety helmet (i.e., a safety helmet is detected in the designated area), the distance the person moves per unit time is used to determine whether the person is in a moving state.

[0050] Specifically, the detected person's pose information is (x, y, θ); (x, y) is the identified two-dimensional spatial coordinate system, and θ is the orientation calculated from the previous and next frames; within a unit time Δt, if the following formula is satisfied, the person is considered to be moving, and is designated as the moving person P. iv Conversely, the person is a person who remains in place (P). is Person P will be identified soon. i Divided into {P iv P is}

[0051]

[0052] Among them, (x t y t Let (x, t) be the two-dimensional spatial coordinates of the person at time t. t+1 y t+1 () represents the two-dimensional spatial coordinates of the person at time t+1.

[0053] Specifically, in this embodiment, the person detection model is a YOLOv8 network. In other embodiments, the person detection model may also employ DensePose, RCNN, or other object detection algorithms.

[0054] The debris recognition model and obstacle detection model can also refer to the personnel detection model. The specific detection methods are existing technologies and will not be described in detail in this embodiment.

[0055] To ensure the smooth operation of slag removal tasks while personnel are present in the slag removal workspace, safety attribute spaces are constructed for both moving and stationary personnel. The safety attribute space is a radiating range of activity centered on the personnel's location, taking into account factors such as the personnel's own activity status, behavioral and psychological safety range, and the safe operating distance of the slag removal device. This ensures that the slag removal device will not cause collision injuries to personnel or cause psychological pressure due to close proximity during the automatic slag removal process, thereby realizing an automatic slag removal process in a human-machine coexistence space.

[0056] Using time T as the unit (T being the time statistic for a single target point to perform a slag removal task), for P i {P iv P is Construct a safety attribute space for identifying individuals, providing basic spatial and motion status information for each person.

[0057] Build a time-series model for predicting human behavior and construct a time series set of their location and velocity. Figure 3 This model displays the position and motion state of a person in a world coordinate system. Besides basic motion attributes, the proposed model also requires safety attributes such as tunnel space size, slag removal device design architecture, and human posture. Therefore, the symbolic representation of the person is divided into two parts: one part represents the person's motion attributes, denoted as h. s , where (x i y i θ represents the position of pedestrian i in the world coordinate system. i It is the orientation angle of pedestrian i relative to the world coordinate system, determined by the direction of the distance the person moves per unit time. (v) xi v yi The x and y axes represent the current speeds of pedestrian i along the x and y axes, respectively. The other part represents the safety attribute characteristics of the person, denoted as h. f Here, we only consider two features: tunnel space size and personnel gesture. This is illustrated in the following equation.

[0058] h = [h] s ,h f ]

[0059] h s =[x i ,y i ,θ i ,v xi ,v yi ]

[0060] h f = [tunnel space, gesture]

[0061] like Figure 3 As shown, for position (x i y i ), with an orientation angle of θ i For worker i, a worker body coordinate system x¢o¢y¢ is constructed and aligned with the x-axis. A two-dimensional Gaussian function is used to simulate the basic personal space (security attribute space) around worker i. The security attribute space around worker i is defined by the function f. i (x,y), where the maximum probability density of the two-dimensional Gaussian function is located at the center (x,y). i y i The basic safety attribute space around worker i is established as follows:

[0062]

[0063] θ = arctan[(yy i ) / (xx i )]

[0064] Among them, A p The amplitude is predetermined; and Let θ be the standard deviation of the Gaussian function. i Let A be the orientation angle of person i. From the formula, the shape and size of the staff's comfort space are mainly determined by parameter A. p , and Decision, and and Size can be determined by the human's motion attribute characteristics h s and security attribute characteristics h f The decision is made. Specifically, in this embodiment, for mobile personnel, to personalize the safety attribute space modeling according to each individual's state, a Gaussian function is redesigned to dynamically adjust the individual's state. First, the entire safety attribute space is constructed into the worker's body coordinate system based on the person's orientation angle. Then, the safety attribute space is divided into four parts—front, back, left, and right—according to the four quadrants corresponding to the body coordinate system, denoted by F, B, L, and R, respectively. Four Gaussian functions are used in each of the four quadrants. An asymmetric Gaussian function is generated by combining the four Gaussian functions. The standard deviation of this combined asymmetric Gaussian function in each direction of the body coordinate system is defined as follows:

[0065] The velocity and direction of the person moving at time p;

[0066] σ F Two-dimensional Gaussian function along Standard deviation of direction;

[0067] σ B Two-dimensional Gaussian function along Standard deviation in the opposite direction;

[0068] σ L Two-dimensional Gaussian function along with The standard deviation on the left side in the vertical direction;

[0069] σ R Two-dimensional Gaussian function along with The standard deviation on the right side in the vertical direction.

[0070] Based on the above definition, the safety attribute space of the moving person i at time p is defined. When the vision sensor detects the worker's speed v i ≠0, let

[0071]

[0072] Based on the above formula, a model of the asymmetric safety attribute space of the moving personnel is established. Since the size of the safety attribute space is related to the speed of the moving personnel and the size of the tunnel and the slag removal robot, during the movement, as the speed increases, people will pay more attention to the area in front of their field of vision, and the range of perception will also expand. Therefore, the corresponding front and side spaces also need to be increased accordingly, while the safety space behind will decrease as the attention decreases. Therefore, the variance of the Gaussian function in the four directions is defined as follows.

[0073]

[0074] Where σ is the initial safe space size of the mobile personnel i; r is the mechanism range of the slag removal robot; c is the current size of the TBM tunnel space; v is the moving speed of the mobile personnel i; f F f B f L and f R These are all proportionality coefficients, representing the magnitude of the four standard deviations as a function of speed.

[0075] For personnel operating in place, the safety attribute space is composed of a standard Gaussian function, exhibiting symmetry, and is only related to the initial safety space size and the mechanical range of the slag-clearing robot. Wherein σ F σ B σ L and σ R (Right now and Determine according to the following formula:

[0076]

[0077] Where σ is the initial safe space size for personnel moving in place; r is the range of the slag removal robot's mechanism; and c is the current size of the TBM tunnel space.

[0078] The completed personnel security attribute space is as follows Figure 4 As shown.

[0079] S2. Input the status information of the mobile personnel into the pre-established movement trajectory prediction model to obtain the movement trajectory of the mobile personnel within a set time period.

[0080] The movement trajectory of the moving person obtained by S1 is predicted. Using time as the unit, the trajectory of moving person i is converted into the change of the relative position of moving person i, where the observed trajectory is represented as X = {X1, X2, ..., X...}. i ,…,X N}, where N represents the number of people detected moving within the slag removal operation area, and X i Indicates the movement of person i in the observation time domain T obs Changes in relative position within X. i The position at time t is defined as T obs A given length for observing the sequence; transit time is 1 to T. obs Observed sequences, predict T obs ~T pred The future trajectory of T obs ~T pred The average time period T for a single slag removal task work Same (i.e., a set time period). The predicted trajectory is defined as Y = {Y1, Y2, ..., Y...} i ,…,Y N},

[0081] Considering the complexity of the tunnel environment, simply using location information as a feature input when establishing a moving personnel trajectory prediction model can easily lead to information loss. Therefore, in this embodiment, the model input is defined as a five-element information group. in, This represents the location information of the moving person i at time t. This represents the velocity information of the moving person i at time t. Indicates the direction of movement of person i at time t. The angle between the line connecting the position of person i at time t and the starting point and the horizontal line indicates whether the person's direction of movement has changed. The calculation formula is as follows:

[0082]

[0083] Where, δt To observe the time step of the sequence; Let i be the starting point position of the moving person.

[0084] The mobile personnel trajectory prediction model in this embodiment is established based on security attribute space constraints. The mobile personnel trajectory prediction model employs a generative adversarial network model based on Long Short Term Memory (LSTM), with the network structure as follows: Figure 5 As shown, it consists of a generator module and a group detection module (i.e. Figure 5 The generator module consists of three parts: the GDM (Generator-Decoder Model), the discriminator module, and the group-constrained hierarchical graph attention model (GDM). The generator module continuously learns from the training dataset to fit the random noise distribution to the real data as closely as possible, thus generating data similar to the training dataset. The discriminator module is responsible for judging the authenticity of the input data and feeding this information back to the generator, thereby improving the generator's fitting performance. The training of the two network modules alternates until the data generated by the generator can be indistinguishable from real data and reaches a balance with the discriminator's capabilities. The generator module includes an encoder, a decoder, and a group-constrained hierarchical graph attention model (GDM). Figure 5 (Secure Attribute Spatial Attention Model). The encoder consists of a location encoder and a motion encoder, which are responsible for processing location feature information and dynamic feature information, respectively.

[0085] Specifically, the input to the generator module is a quintuple preprocessed from multiple frames of historical observation trajectories. Each frame contains the observation trajectory of the number of workers in a fixed slag removal area, and the predicted trajectory of these workers is output.

[0086] The encoder in the generator module consists of a position encoder and a motion encoder. The position encoder (i.e.,...) Figure 5 In the FC+LSTM module, the module obtains the position information in the quintuple. A fully connected network (FC) maps the relative position of pedestrian i at time t into a high-dimensional feature vector. Then, the LSTM-based encoder module encodes the relative position changes of each trajectory to obtain hidden state features containing the position information of the entire trajectory.

[0087]

[0088] Where: f E-FC1 W represents the position encoding function consisting of fully connected layers. gmb These are the weights of the positional encoding function. f E-LSTMThis indicates a position encoder composed of LSTM. It is the hidden state at time t, containing motion information of the historical trajectory before time t, W e-lstm These are the corresponding position encoder weights, which are shared among all pedestrians in the LSTM, i.e., W. e-lstm This is shared during the encoding phase. This process is repeated until the observation sequence t = 1, 2, ..., T is obtained. obs All information has been encoded.

[0089] In the motion encoder, the module acquires all the information from the quintuple and uses a multilayer perceptron to encode the obtained information, mapping the position and motion information of worker i at time t into a high-dimensional feature vector.

[0090]

[0091] Among them, W d The weights are those of a multilayer perceptron (MLP).

[0092] Will and the output of the position encoder Perform a splicing operation, and output the result to a group-constrained hierarchical graph attention model through a fully connected layer.

[0093]

[0094] Among them, W E For fully connected layer f E-FC2 The weight.

[0095] Since the staff in the scenario exist in groups (multiple people), the interactions within and between groups have distinct characteristics. Groups exhibit member stability, maintaining their topological structure during movement, while obstacle avoidance maneuvers typically occur between groups. Therefore, the interactions between staff are divided into two levels: intra-group and inter-group, leading to a proposed hierarchical graph attention interaction model that considers group constraints. The input is... The pedestrian interaction relationship adjacency matrix obtained from the group detection module is used as the output as the interaction vector V. i t .

[0096] Group Detection Module: Accurate extraction of temporal features, spatial features, and spatiotemporal dependencies is crucial for constructing worker interaction relationships. The group detection module employs a Transformer-based cross-attention mechanism to extract the temporal, spatial features, and correlations of worker trajectories. To achieve this, the trajectory data of the worker group is first preprocessed and input into a space-time Transformer model. An encoder based on spatial and temporal features is used to model the worker group, extracting the motion state information of each worker at different time steps. Then, a cross-attention mechanism is used to capture the interactions between workers, thus obtaining a similarity score between each worker and other workers. Finally, a decoder is used to transform these similarity scores into an adjacency matrix, where each element of the adjacency matrix represents the adjacency relationship between two workers. This group detection module is as follows: Figure 7 As shown, it consists of an encoder and a decoder.

[0097] The decoder in the generator module is primarily used for generating predicted trajectories. After obtaining the historical encoded information of person i, to account for the uncertainty of person motion, a noise vector z is sampled from the standard normal distribution N(0,1), and then the obtained noise vector... Interaction Feature V i t and t=T obs Encoded hidden state Perform the concatenation to obtain the decoded feature vector at time t.

[0098]

[0099] The concatenated feature vector As a decoder module f D-LSTM The initial hidden state is used as input. When predicting the pedestrian position at each subsequent time t+1, the predicted position Y of the previous time t is first taken as input. i t After passing through a fully connected network f D-FC Transform from coordinate space to feature space to obtain feature vectors Then through f D-LSTM Calculate the current decoding feature state It is worth noting that at the first prediction time t = Tob + 1, the last observation position of the known observation sequence is directly used as the input.

[0100]

[0101] Utilizing the obtained decoder features A multilayer perceptron can be used to directly generate a prediction of the position at time t+1.

[0102]

[0103] Where f dec It is a multilayer perceptron, W d-dec These are the weights corresponding to the multilayer perceptron, Y. i t+1 It is the predicted trajectory that maps decoder features to a multilayer perceptron.

[0104] The discriminator module consists of a motion encoder and a classifier. The motion encoder comprises an LSTM network and fully connected layers, while the classifier is a binary classification model. First, the generated trajectory samples are constructed by combining the predicted trajectory generated by the generator with the observed trajectory. And a sample of real trajectories consisting of observed trajectories and actual future trajectories. The corresponding feature vectors are obtained by inputting them into the motion encoder. and Then, an LSTM network is used to encode the feature vectors to obtain the hidden feature vectors. The input is decoded by a multilayer perceptron. Then, a softmax classifier is used to calculate the output result D. dis Finally, a classification score between 0 and 1 is output. A higher score indicates a greater probability that the input is a real trajectory, while a lower score indicates a greater probability that the input is a generated trajectory.

[0105]

[0106] Among them, W Dis-e-fc These are the parameters of the multilayer perceptron in the motion encoder, responsible for embedding the worker's position into the feature space; W Dis These are the parameters of the LSTM network in the motion encoder of the discriminator.

[0107] S3. Using the moving personnel following the movement trajectory as dynamic obstacles and the stationary personnel as static obstacles, a path planning algorithm is used to plan the path of the slag cleaning robot based on the slag distribution information, so that the slag cleaning robot can avoid dynamic and static obstacles when cleaning slag according to the slag cleaning path.

[0108] Static obstacles during path planning also include obstacle information obtained from S1 detection, ob(m) = {obs(m), obg(m)}.

[0109] Specifically, to address the issue that traditional path planning for slag removal tasks does not consider the movement of workers in the bottom slag removal space over time, the predicted trajectory of personnel in the safety attribute space is considered as a dynamic obstacle. Combined with static obstacles in the environment, a time-series slag removal cost map (including the distribution of slag removal targets and obstacles) is constructed. This allows for the calculation of a trajectory that meets the safety and comfort requirements of workers, as well as the requirements for the execution of automatic slag removal tasks in a human-machine coexistence space, i.e., the slag removal operation path.

[0110] The time-series slag removal cost map includes the distribution of slag accumulation as the slag removal target and the distribution of obstacles that need to be avoided. Among them, obstacles include static obstacles whose positions do not change over time and dynamic obstacles whose positions change over time.

[0111] Taking the traditional A* algorithm as an example, the future T work The safety attribute cost map generated from the predicted trajectory at each time step is integrated into the overall time-series cost map. Then, time-dependent constraints are added to the A* algorithm to finally construct the path planner. The specific process is as follows: Figure 6 As shown.

[0112] In other implementations, the path planning algorithm may also employ the Rapidly Exploring Random Tree algorithm, or other existing path planning algorithms that consider dynamic obstacle avoidance.

[0113] After obtaining the slag removal operation path, it can be transformed into the control of each joint of the automatic slag removal robot by combining the robot's inverse kinematics decomposition, thereby realizing the overall realization of automatic slag removal operation.

[0114] Example of a TBM slag removal system:

[0115] A TBM slag removal system includes a sensing module and a data processing module. The sensing module is used to acquire slag distribution information and personnel status information in the slag removal operation area. The data processing module is used to execute a computer program to implement the steps of the slag removal path planning method for a TBM slag removal robot that considers personnel early warning in the slag removal area, as described above. The specific slag removal path planning method for a TBM slag removal robot that considers personnel early warning in the slag removal area has been described in sufficient detail in the above-described embodiments and will not be repeated here.

[0116] Computer device embodiment:

[0117] A computer device includes a processor for executing a computer program to implement the steps of the TBM slag cleaning robot path planning method considering personnel warnings in the slag cleaning area as described above. The specific slag cleaning robot path planning method considering personnel warnings in the slag cleaning area has been described in sufficient detail in the above-described embodiments and will not be repeated here.

[0118] Specifically, a processor can be a CPU, or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. A processor can also be a processor that supports the Advanced Reduced Instruction Set Machine (ARM) architecture.

[0119] This invention has the following characteristics:

[0120] Based on this invention, it is possible to identify workers in the slag removal area at the bottom of the tunnel using visual sensors. By constructing a safety attribute space and combining it with dynamic prediction of personnel behavior trajectories, a time-series cost map is established to guide the path planning and mechanism control of the automatic slag removal task. This enables the automatic slag removal task to be carried out even when workers and slag removal devices coexist in the same space, while ensuring a safe and comfortable distance between the workers and the equipment, thereby improving the efficiency of automatic slag removal and TBM tunneling.

[0121] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still make modifications to the technical solutions described in the foregoing embodiments without creative effort, or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for planning the cleaning path of a TBM cleaning robot that considers personnel early warning in the cleaning area, characterized in that, The method includes: S1. Obtain information on the distribution of accumulated slag and the status of personnel in the slag removal operation area; the personnel include mobile personnel and personnel moving in place. S2, inputting the state information of the moving personnel into a pre-established moving trajectory prediction model to obtain a time period T in which the moving personnel performs a slag removal task later work moving trajectory within the inner portion The safety attribute space for each person is constructed based on a two-dimensional Gaussian function; the position at time p is (x i , y i The safety attribute space of personnel i) : ; Among them, A p The amplitude is predetermined; ; ; ; Let σ be the direction of the velocity of person i at time p; F σ B σ L and σ R Gaussian function along direction, opposite direction, with The standard deviations of the left and right sides in the vertical direction; For moving people, σ F , σ B , σ L and σ R are determined as follows: ; Wherein, σ is the initial security space size; r is the mechanism range of the slag cleaning robot; c is the current TBM tunnel space size; v is the moving speed; f F , f B , f L , and f R are proportional coefficients; For those who remain in place, ; S3. Using the moving personnel moving according to the moving trajectory as dynamic obstacles and the stationary personnel as static obstacles, a path planning algorithm is used to plan the path of the slag cleaning robot according to the slag distribution information, so that the slag cleaning robot can avoid the dynamic obstacles and static obstacles when cleaning slag according to the slag cleaning path.

2. The TBM slag cleaning robot slag cleaning path planning method considering personnel early warning in the slag cleaning area according to claim 1, characterized in that, Information on the distribution of accumulated slag is obtained by acquiring image information of the slag removal operation area.

3. The TBM slag removal robot slag removal path planning method considering personnel early warning in the slag removal area according to claim 2, characterized in that, Image information of the slag removal operation area is acquired by vision sensors installed on the main beam of the TBM.

4. The TBM slag removal robot slag removal path planning method considering personnel early warning in the slag removal area according to claim 3, characterized in that, When there is more than one visual sensor, an image stitching algorithm is used to integrate the image information acquired by different visual sensors into a single image.

5. The TBM slag removal robot slag removal path planning method considering personnel early warning in the slag removal area according to claim 1, characterized in that, The state information of the moving person input into the pre-established trajectory prediction model includes position information, speed information, direction of movement, and the angle between the line connecting the current position and the starting point and the horizontal line.

6. The descaling path planning method for a TBM descaling robot considering personnel early warning in the descaling area according to any one of claims 1 to 5, characterized in that, The trajectory prediction model is a generative adversarial network based on LSTM.

7. The TBM slag removal robot slag removal path planning method considering personnel early warning in the slag removal area according to claim 1, characterized in that, S1 also includes obtaining obstacle information in the slag removal operation area; correspondingly, in S3, the obstacles are used as static obstacles for path planning.

8. The slag removal path planning method for a TBM slag removal robot considering personnel early warning in the slag removal area according to claim 1, characterized in that, The process of obtaining personnel status information in S1 includes: inputting the image information of the slag removal operation area into the pre-trained personnel detection model to obtain the personnel location information and safety helmet location information in the slag removal operation area; If no safety helmet is detected in the area where personnel were detected, a safety alarm will be triggered and the cleaning operation will be stopped until a safety helmet is detected in the area or no personnel are found in the area.

9. The TBM slag removal robot slag removal path planning method considering personnel early warning in the slag removal area according to claim 8, characterized in that, The process of obtaining personnel status information in S1 also includes: when a safety helmet is detected in the area where a person is detected, if the distance the person moves within a unit of time is less than a preset distance threshold, the person is considered to be a person moving in place; otherwise, the person is considered to be a person moving.

10. The slag removal path planning method for a TBM slag removal robot considering personnel early warning in the slag removal area according to claim 1, characterized in that, The path planning algorithm is as follows: Algorithm or RRT algorithm.

11. A TBM slag removal system, characterized in that, It includes a sensing module and a data processing module; the sensing module is used to acquire information on the distribution of accumulated slag and the status of personnel in the slag removal operation area; the data processing module executes a computer program to implement the following steps of the slag removal path planning method for a TBM slag removal robot that considers personnel early warning in the slag removal area: S1. Obtain information on the distribution of accumulated slag and the status of personnel in the slag removal operation area; the personnel include mobile personnel and personnel moving in place. S2. Input the status information of the mobile personnel into the pre-established movement trajectory prediction model to obtain the time period T for the mobile personnel to perform the slag removal task later. work The movement trajectory within; The safety attribute space for each person is constructed based on a two-dimensional Gaussian function; the position at time p is (x i , y i The safety attribute space of personnel i) : ; Among them, A p The amplitude is predetermined; ; ; ; Let σ be the direction of the velocity of person i at time p; F σ B σ L and σ R Gaussian function along direction, opposite direction, with The standard deviations of the left and right sides in the vertical direction; For mobile personnel, σ F σ B σ L and σ R Determined according to the following formula: ; Where σ is the initial safety space size; r is the range of the cleaning robot's mechanism; c is the current TBM tunnel space size; v is the moving speed; f F f B f L and f R All are proportionality coefficients; For those who remain in place, ; S3. Using the moving personnel moving according to the moving trajectory as dynamic obstacles and the stationary personnel as static obstacles, a path planning algorithm is used to plan the path of the slag cleaning robot according to the slag distribution information, so that the slag cleaning robot can avoid the dynamic obstacles and static obstacles when cleaning slag according to the slag cleaning path.

12. The TBM slag removal system according to claim 11, characterized in that, Information on the distribution of accumulated slag is obtained by acquiring image information of the slag removal operation area.

13. The TBM slag removal system according to claim 12, characterized in that, Image information of the slag removal operation area is acquired by vision sensors installed on the main beam of the TBM.

14. The TBM slag removal system according to claim 13, characterized in that, When there is more than one visual sensor, an image stitching algorithm is used to integrate the image information acquired by different visual sensors into a single image.

15. The TBM slag removal system according to claim 11, characterized in that, The state information of the moving person input into the pre-established trajectory prediction model includes position information, speed information, direction of movement, and the angle between the line connecting the current position and the starting point and the horizontal line.

16. The TBM slag removal system according to any one of claims 11 to 15, characterized in that, The trajectory prediction model is a generative adversarial network based on LSTM.

17. The TBM slag removal system according to claim 11, characterized in that, S1 also includes obtaining obstacle information in the slag removal operation area; correspondingly, in S3, the obstacles are used as static obstacles for path planning.

18. The TBM slag removal system according to claim 11, characterized in that, The process of obtaining personnel status information in S1 includes: inputting the image information of the slag removal operation area into the pre-trained personnel detection model to obtain the personnel location information and safety helmet location information in the slag removal operation area; If no safety helmet is detected in the area where personnel were detected, a safety alarm will be triggered and the cleaning operation will be stopped until a safety helmet is detected in the area or no personnel are found in the area.

19. The TBM slag removal system according to claim 18, characterized in that, The process of obtaining personnel status information in S1 also includes: when a safety helmet is detected in the area where a person is detected, if the distance the person moves within a unit of time is less than a preset distance threshold, the person is considered to be a person moving in place; otherwise, the person is considered to be a person moving.

20. The TBM slag removal system according to claim 11, characterized in that, The path planning algorithm is as follows: Algorithm or RRT algorithm.

21. A computer device comprising a processor, characterized in that, The processor is used to execute a computer program to implement the following steps of the TBM slag removal robot slag removal path planning method that takes into account personnel early warning in the slag removal area: S1. Obtain information on the distribution of accumulated slag and the status of personnel in the slag removal operation area; the personnel include mobile personnel and personnel moving in place. S2. Input the status information of the mobile personnel into the pre-established movement trajectory prediction model to obtain the time period T for the mobile personnel to perform the slag removal task later. work The movement trajectory within; The safety attribute space for each person is constructed based on a two-dimensional Gaussian function; the position at time p is (x i , y i The safety attribute space of personnel i) : ; Among them, A p The amplitude is predetermined; ; ; ; Let σ be the direction of the velocity of person i at time p; F σ B σ L and σ R Gaussian function along direction, opposite direction, with The standard deviations of the left and right sides in the vertical direction; For mobile personnel, σ F σ B σ L and σ R Determined according to the following formula: ; Where σ is the initial safety space size; r is the range of the cleaning robot's mechanism; c is the current TBM tunnel space size; v is the moving speed; f F f B f L and f R All are proportionality coefficients; For those who remain in place, ; S3. Using the moving personnel moving according to the moving trajectory as dynamic obstacles and the stationary personnel as static obstacles, a path planning algorithm is used to plan the path of the slag cleaning robot according to the slag distribution information, so that the slag cleaning robot can avoid the dynamic obstacles and static obstacles when cleaning slag according to the slag cleaning path.

22. The computer apparatus according to claim 21, characterized in that, Information on the distribution of accumulated slag is obtained by acquiring image information of the slag removal operation area.

23. The computer apparatus according to claim 22, characterized in that, Image information of the slag removal operation area is acquired by vision sensors installed on the main beam of the TBM.

24. The computer apparatus according to claim 23, characterized in that, When there is more than one visual sensor, an image stitching algorithm is used to integrate the image information acquired by different visual sensors into a single image.

25. The computer apparatus according to claim 21, characterized in that, The state information of the moving person input into the pre-established trajectory prediction model includes position information, speed information, direction of movement, and the angle between the line connecting the current position and the starting point and the horizontal line.

26. The computer apparatus according to any one of claims 21 to 25, characterized in that, The trajectory prediction model is a generative adversarial network based on LSTM.

27. The computer apparatus according to claim 21, characterized in that, S1 also includes obtaining obstacle information in the slag removal operation area; correspondingly, in S3, the obstacles are used as static obstacles for path planning.

28. The computer apparatus according to claim 21, characterized in that, The process of obtaining personnel status information in S1 includes: inputting the image information of the slag removal operation area into the pre-trained personnel detection model to obtain the personnel location information and safety helmet location information in the slag removal operation area; If no safety helmet is detected in the area where personnel were detected, a safety alarm will be triggered and the cleaning operation will be stopped until a safety helmet is detected in the area or no personnel are found in the area.

29. The computer apparatus according to claim 28, characterized in that, The process of obtaining personnel status information in S1 also includes: when a safety helmet is detected in the area where a person is detected, if the distance the person moves within a unit of time is less than a preset distance threshold, the person is considered to be a person moving in place; otherwise, the person is considered to be a person moving.

30. The computer device according to claim 21, characterized in that, The path planning algorithm is as follows: Algorithm or RRT algorithm.

Citation Information

Patent Citations

  • Tunnel bottom accumulated slag cleaning system suitable for TBM construction and starting method of tunnel bottom accumulated slag cleaning system

    CN116901064A

  • Slag cleaning robot and TBM accumulated slag cleaning system

    CN117888918A

  • Dynamic obstacle avoidance system and method for animal house cleaning robot based on laser sensing

    CN118760195A