Wheeled mobile platform terrain detection method

Abstract

The invention discloses a wheeled mobile platform terrain detection method, comprising: acquiring a terrain classification data set, the data set comprising a tactile image stream corresponding to tactile sequence data collected in the traveling process of a wheeled mobile platform and pre-labeled terrain category information; training the initial classification model by using a tactile image stream in the terrain classification data set to obtain a terrain classification model; and inputting a to-be-detected tactile image flow into the terrain classification model to obtain a terrain detection result of the mobile platform. The terrain type of the wheel type mobile platform in the advancing process is effectively detected through the wheel type mobile platform terrain detection method, the trolley can avoid the terrain prone to sinking and bumping, damage is prevented, and the task success rate is increased.

Description

[Technical field]

[0001] The invention relates to a terrain detection method for a wheeled mobile platform, and belongs to the fields of terrain detection technology, tactile perception technology and artificial intelligence technology. [Background technology]

[0002] Wheeled mobile platforms are widely used in various complex tasks, and terrain factors have a great impact on task execution. Terrain recognition can provide key information for subsequent decision-making, allowing wheeled mobile platforms to avoid terrain that easily causes them to sink and bump, thereby improving the success rate of the task. Therefore, terrain detection methods are of great significance to the task execution of wheeled mobile platforms.

[0003] Existing terrain detection methods for wheeled mobile platforms are based on visual perception and vibration perception. However, visual perception is easily restricted by factors such as lighting conditions and air particles. When the lighting conditions for outdoor operations are poor and there are many air particles, the quality of the acquired terrain images is poor, which leads to unreliable final terrain detection results. Vibration perception cannot directly reflect the surface characteristics. It indirectly infers the surface properties by obtaining the vibration information generated when the wheeled mobile platform contacts the ground. The mapping relationship with the terrain category is difficult to clarify. When the spatial characteristics of the terrain are relatively complex, the local surface features are not given enough attention, resulting in a decrease in the overall detection accuracy. Therefore, this patent proposes a wheeled mobile platform detection method based on touch. [Summary of the invention]

[0004] In view of this, the present invention proposes a terrain detection method for a wheeled mobile platform for robot grasping operations.

[0005] The present invention provides a terrain detection method for a wheeled mobile platform, comprising:

[0006] Acquire a terrain classification data set, wherein the data set includes a tactile image stream corresponding to the tactile sequence data collected during the movement of the wheeled mobile platform, and pre-labeled terrain category information;

[0007] The initial classification model is trained using the tactile image stream in the terrain classification dataset to obtain a terrain classification model;

[0008] The tactile image stream to be detected is input into the terrain classification model to obtain the terrain detection result of the mobile platform.

[0009] In the above method, the step of obtaining a terrain classification dataset includes:

[0010] A plurality of terrains with different surface shapes and hardness are selected to obtain a terrain classification data set. For each type of terrain, a wheeled mobile platform is placed on the terrain, and the wheeled mobile platform is controlled to move at a constant speed within a fixed data collection period to collect a tactile information sequence.

[0011] Set the cutoff threshold x s and time step T. In each acquisition cycle, from the moment when the acquisition value of any acquisition point of the capacitive flexible tactile sensor first exceeds the interception threshold, all tactile information within the subsequent T steps is intercepted to form a tactile information sequence, which constitutes a single sample;

[0012] The tactile information of each step is encoded into an image to obtain a tactile image stream.

[0013] In the above method, the initial classification model is trained using the tactile image stream in the terrain classification dataset to obtain a terrain classification model, including

[0014] Inputting the tactile image stream into a tag generation module to obtain a time tag and a space tag;

[0015] Input the time and space tags into the multi-head spatiotemporal attention module to obtain terrain information;

[0016] Based on the terrain information and the pre-labeled terrain information, determine whether the current classification model has reached the preset convergence condition of the loss function value. If so, use the current model as the terrain classification model; if not, continue to execute the above training process until the initial classification model reaches the preset convergence condition.

[0017] In the above method, the pre-labeled terrain category information includes:

[0018] The terrain category corresponding to the tactile image stream.

[0019] In the above method, the tag generation module includes

[0020] Using convolution operation, the tactile image stream in the sample is taken as input, and the spatiotemporal mark is output as the initial tactile spatiotemporal feature information;

[0021] Using position embedding and dimension reconstruction operations, it takes spatiotemporal tags as input and outputs time tags and spatial tags.

[0022] In the above method, the multi-head spatiotemporal attention module includes

[0023] The attention encoder takes the time tag and the space tag as input and outputs the spatiotemporal feature vector as the tactile spatiotemporal feature information;

[0024] The classification head takes tactile spatiotemporal feature information as input and outputs terrain detection results.

[0025] It can be seen from the above technical solutions that the embodiments of the present invention have the following beneficial effects:

[0026] The present invention uses a model to realize the terrain detection of the wheeled mobile platform, obtains the tactile image stream through the tactile information sequence image representation method, and obtains the terrain recognition result of the wheeled mobile platform during its movement through the terrain classification model. It effectively improves the perception ability of the wheeled mobile platform during its movement, can effectively detect the terrain category during the movement of the wheeled mobile platform, enables the vehicle to avoid the terrain that is easy to sink and bump, prevents damage, and improves the mission success rate.

Brief Description of the Drawings

[0027] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It is obvious that the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creativity and labor.

[0028] Figure 1 It is a schematic flow chart of a terrain detection method for a wheeled mobile platform provided in an embodiment of the present invention;

[0029] Figure 2 Schematic diagram of a terrain classification model in an embodiment of the present invention. [Specific implementation method]

[0030] In order to better understand the technical solution of the present invention, the embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0031] It should be clear that the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] The present invention provides a method for detecting terrain on a wheeled mobile platform. Figure 1 , which is a flow chart of a terrain detection method for a wheeled mobile platform provided in an embodiment of the present invention, such as Figure 1 As shown, the method comprises the following steps:

[0033] Step 101 : obtaining a terrain classification data set, wherein the data set includes a tactile image stream and pre-labeled terrain category information; wherein the labeled category terrain information includes the terrain category corresponding to the tactile image stream.

[0034] Several terrains with different surface shapes and hardnesses are selected to obtain terrain classification data sets. For each type of terrain, a wheeled mobile platform is placed on it and controlled to move at a constant speed within a fixed data collection period to collect tactile information sequences.

[0035] Set the cutoff threshold xs and time step T. In each acquisition cycle, from the moment when the acquisition value of any acquisition point of the capacitive flexible tactile sensor first exceeds the interception threshold, all tactile information within the subsequent T steps is intercepted to form a tactile information sequence, which constitutes a single sample;

[0036] The arbitrary acquisition point of the capacitive flexible tactile sensor includes any one of the L capacitance acquisition units of the capacitive flexible tactile sensor. The tactile information sequence is a capacitance value data within T steps intercepted by setting the interception threshold and the time step, a total of T×L capacitance value data.

[0037] The tactile information of each step is encoded into an image to obtain a tactile image stream. Specifically, for the capacitance value data [x 1 ,x 2 ,x 3 ,...,x L ], and use the method of filling zero values ​​to expand its dimension to a square number L+k that is larger than L and closest to the value of L. Then the tactile information within a single step is:

[0038] X=[x 1 ,x 2 ,x 3 ,...,x L ,p 1 ,p 2 ,p 3 ,...,p k ]

[0039] Among them, p m =0, m=1,2,3,...,k. Then the capacitance data is normalized and mapped to the (-1,1) interval:

[0040]

[0041] in, is the tactile information element after normalization;

[0042] Then the normalized tactile information Mapping into polar coordinate space:

[0043]

[0044] The tactile information element Encoded as radius r, element number n i It is encoded as an angle φ, and N is a constant for normalizing the scale of the polar coordinate system;

[0045] Then project the polar coordinate tactile information into the matrix:

[0046]

[0047] Here I represents the unit row vector, i and j are the serial numbers of the tactile information elements;

[0048] The GASF value is mapped into a pseudo-color image to achieve the conversion of tactile information to image. The tactile information sequence of T steps in a single sample is converted into a size of image stream.

[0049] Step 102: Use the tactile image stream in the terrain classification data set to train the initial classification model to obtain a terrain classification model.

[0050] Please refer to Figure 2 , which is a schematic diagram of the terrain detection process in an embodiment of the present invention.

[0051] The tactile image stream is input into a tag generation module to obtain a time tag and a space tag. The time tag and the space tag are input into a multi-head spatiotemporal attention module to obtain terrain information.

[0052] The tag generation module includes using convolution operations to transform the The tactile image stream is taken as input, and n frames of images are sampled from the sample T. t Frame, each tactile image frame is divided into n h ×n w blocks, you can output n t ×n h ×n w spatiotemporal markers as the initial tactile spatiotemporal feature information. h and n w They are the number of spatiotemporal markers in the height direction and the width direction when marking the image;

[0053] Using position embedding and dimension reconstruction operations, we take spatiotemporal tags as input and output n for each sample. t time stamp and n h ×n w A space marker.

[0054] The multi-head spatiotemporal attention module includes an attention encoder and a classification head. The attention encoder takes time tags and spatial tags as input and outputs a spatiotemporal feature vector as tactile spatiotemporal feature information; the classification head takes tactile spatiotemporal feature information as input and outputs terrain information.

[0055] The multi-head spatiotemporal attention module includes: using a linear layer to expand the feature dimension d of the time tag and the space tag to 3d, and adjusting the dimension accordingly according to the spatial or temporal attention, and assigning it to the key, value and query, thereby constructing the key, value and query of the spatial attention respectively. and temporal attention keys, values, and queries Then the spatial attention head and the temporal attention head are used for the spatial input and the temporal input respectively, and the attention mechanism uses dot product attention:

[0056]

[0057] Among them, Q, K, V are the input key, value and query respectively, Y is the output of the attention head, and Softmax is the normalized exponential function. Then the spatial attention head and the temporal attention head are respectively subjected to residual connection operations. Then the output Y of the spatial attention head is s And the output Y of the temporal attention head t Through linear projection combination, the tactile spatiotemporal feature information Y is output o :

[0058] Y o =Concat(Y s ,Y t )W O

[0059] Among them, W O is the coefficient matrix, and Concat is the vector concatenation operation.

[0060] Finally, the tactile spatiotemporal feature information is input into a classification head composed of a fully connected layer, and the classification head outputs terrain information.

[0061] The data shapes of the input and output of the main modules of the terrain classification model are shown in Table 1.

[0062]

[0063] Table 1 Model parameters

[0064] According to the terrain information and the pre-marked terrain information, it is determined whether the current classification model reaches the convergence condition of the preset loss function value. If yes, the current model is used as the terrain classification model; if not,

[0065] The above training process is continued until the initial classification model reaches a preset convergence condition.

[0066] Among them, the loss function is a multi-category cross entropy loss function:

[0067]

[0068] Given N samples of C terrain categories, y i,c is the label of sample i corresponding to terrain category C; p i,c When input sample i is given, the network confidence output corresponding to category C.

[0069] Among them, a multi-category cross entropy loss function is used to calculate the terrain category and the pre-labeled terrain category. When the error value calculated by the loss function is less than a preset threshold, or the number of iterations reaches a predetermined value, it can be considered that the initial classification model has reached a preset convergence condition.

[0070] Step 103: input the tactile image stream to be detected into a terrain classification model to obtain a terrain detection result of the mobile platform.

[0071] The tactile image stream is input into a tag generation module to obtain a time tag and a space tag. The time tag and the space tag are input into a multi-head spatiotemporal attention module to obtain a terrain detection result.

[0072] According to the above method provided by the embodiment of the present invention, the initial terrain classification model is trained using a partial data set of the acquired training data set, and the accuracy of the terrain classification model is tested using a partial data set of the acquired training data set and compared with the existing method. The evaluation method adopts the widely used accuracy.

[0073] The accuracy is calculated as follows:

[0074]

[0075] Among them, TP is the number of samples correctly predicted as positive by the terrain classification model, FP is the number of samples of the negative category incorrectly identified as positive by the terrain classification model, FN is the number of samples incorrectly predicted as negative by the terrain classification model, and TN is the number of samples of the negative category correctly identified as negative by the terrain classification model.

[0076] The results are shown in Table 2:

[0077] Table 2 Comparison between the present invention and the existing method

[0078]

[0079] It can be seen from Table 2 that the terrain recognition method proposed in the present invention can effectively realize terrain detection during the movement of the wheeled mobile platform, with an accuracy rate of 91.9%, which is better than the existing method.

[0080] The technical solution of the embodiment of the present invention has the following beneficial effects:

[0081] In the technical solution of the embodiment of the present invention, a model is used to realize the terrain detection of the wheeled mobile platform, a tactile image stream is obtained through the tactile information sequence image representation method, and the terrain recognition result of the wheeled mobile platform during its movement is obtained through the terrain classification model. The perception ability of the wheeled mobile platform during its movement is effectively improved, and the terrain category of the wheeled mobile platform during its movement can be effectively detected, so that the vehicle can avoid the terrain that is easy to sink and bump, prevent damage, and improve the success rate of the mission.

[0082] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

[0083] The contents not described in detail in the specification of the present invention belong to the common knowledge of those skilled in the art.

Claims

1. A method for detecting terrain on a wheeled mobile platform, characterized in that: The method comprises: Acquire a terrain classification dataset, wherein the dataset includes a tactile image stream and pre-labeled terrain category information; The initial classification model is trained using the tactile image stream in the terrain classification dataset to obtain a terrain classification model; The tactile image stream to be detected is input into the terrain classification model to obtain the terrain detection result of the mobile platform.

2. The method according to claim 1, characterized in that The terrain classification dataset includes: A plurality of terrains with different surface shapes and hardness are selected to obtain a terrain classification data set. For each type of terrain, a wheeled mobile platform is placed on the terrain, and the wheeled mobile platform is controlled to move at a constant speed within a fixed data collection period to collect a tactile information sequence. Set the cutoff threshold x s and time step T. In each acquisition cycle, from the moment when the acquisition value of any acquisition point of the capacitive flexible tactile sensor first exceeds the interception threshold, all tactile information within the subsequent T steps is intercepted to form a tactile information sequence, which constitutes a single sample; The tactile information of each step is encoded into an image to obtain a tactile image stream.

3. The method according to claim 1, characterized in that The method of training the initial classification model using the tactile image stream in the terrain classification data set to obtain the terrain classification model includes: Inputting the tactile image stream into a tag generation module to obtain a time tag and a space tag; Input the time and space tags into the multi-head spatiotemporal attention module to obtain terrain information; According to the terrain information and the pre-labeled terrain category information, it is determined whether the current classification model has reached the preset convergence condition of the loss function value. If so, the current model is used as the terrain classification model; if not, the training is repeated until the initial classification model reaches the preset convergence condition.

4. The method according to claim 1, characterized in that: The pre-marked terrain category information includes: The terrain category corresponding to the tactile image stream.

5. The method according to claim 2, characterized in that: Any collection points of the capacitive flexible tactile sensor include: Any one of the L capacitance collection units of the capacitive flexible tactile sensor.

6. The method according to claim 2, characterized in that The tactile information sequence includes: The capacitance value data within T steps intercepted by setting the interception threshold and the time step is T×L capacitance value data in total.

7. The method according to claim 2, characterized in that The tactile information of each step is encoded as an image including: For the capacitance value data collected within a single step [x1, x2, x3, ..., x L ], and use the method of filling zero values ​​to expand its dimension to a square number L+k that is larger than L and closest to the value of L. Then the tactile information within a single step is: X=[x1,x2,x3,...,x L ,p1,p2,p3,...,p k ] Among them, p m =0, m=1,2,3,...,k, and then normalize the capacitance data and map it to the (-1,1) interval: in, is the normalized tactile information element; Then the normalized tactile information Mapping into polar coordinate space: The tactile information element Encoded as radius r, element number n i It is encoded as an angle φ, and N is a constant for normalizing the scale of the polar coordinate system; Then project the polar coordinate tactile information into the matrix: Here I represents the unit row vector, i and j are the serial numbers of the tactile information elements; The GASF value is mapped into a pseudo-color image to achieve the conversion of tactile information to image. The tactile information sequence of T steps in a single sample is converted into a size of image stream.

8. The method according to claim 3, characterized in that The tag generation module comprises: Using the convolution operation, the sample The tactile image stream is taken as input, and n frames of images are sampled from the sample T. t Frame, each tactile image frame is divided into n h ×n w blocks, you can output n t ×n h ×n w spatiotemporal markers as the initial tactile spatiotemporal feature information, where n h and n w They are the number of spatiotemporal markers in the height direction and the width direction when marking the image; Using position embedding and dimension reconstruction operations, we take spatiotemporal tags as input and output n for each sample. t time stamp and n h ×n w A space marker.

9. The method according to claim 3, characterized in that: The loss function includes: Multi-class cross entropy loss function: Given N samples of C terrain categories, y i,c is the label of sample i corresponding to terrain category C; p i,c When input sample i is given, the network confidence output corresponding to category C.

10. The method according to claim 3, characterized in that: The multi-head spatiotemporal attention module includes: The feature dimension d of temporal and spatial tags is expanded to 3D using a linear layer, and the dimensions are adjusted accordingly according to the spatial attention, and they are assigned to keys, values, and queries to construct the keys, values, and queries of spatial attention. And adjust the dimensions accordingly according to the time attention and assign them to keys, values, and queries to construct the keys, values, and queries of time attention. Then spatial attention and temporal attention are used on the spatial input and temporal input respectively, and the attention mechanism uses dot product attention: Among them, Q, K, V are the input key, value and query respectively, Y is the output of the attention head, Softmax is the normalized exponential function, and then the spatial attention head and the temporal attention head are respectively subjected to residual connection operations, and the output Y of the spatial attention head is then s And the output Y of the temporal attention head t Through linear projection combination, the tactile spatiotemporal feature information Y is output o : AND o =Concat(And s ,AND t )W O Among them, W O is the coefficient matrix, Concat is the vector concatenation operation; Finally, the tactile spatiotemporal feature information is input into a classification head composed of a fully connected layer, and the classification head outputs terrain information.

Images