A method and device for quickly positioning an unmanned vehicle based on store sign recognition assistance

Abstract

The present invention relates to the field of intelligent positioning, and in particular to a method and device for rapid positioning of an unmanned vehicle based on store sign recognition assistance. The method comprises: obtaining environmental perception data of the unmanned vehicle to be positioned; inputting the environmental perception data into a constructed end-to-end OCR network based on the Transformer architecture to obtain a text recognition result; performing similarity calculation on the text recognition result and the constructed feature database to obtain the position range of the unmanned vehicle; determining the initial position of the unmanned vehicle according to the position range of the unmanned vehicle and the constructed three-dimensional point cloud environment map; obtaining the precise positioning of the unmanned vehicle according to the initial position of the unmanned vehicle and a particle filtering algorithm. The present invention aims at the problems existing in the positioning of unmanned vehicles in large shopping malls. By combining the end-to-end OCR technology based on the Transformer architecture, the feature database matching technology and the particle filtering algorithm without relying on GNSS, the present invention realizes efficient and precise positioning of unmanned vehicles in shopping mall environments.

Description

Technical Field

[0001] The present invention relates to the field of intelligent positioning technology, and in particular to a method and device for quickly positioning an unmanned vehicle based on store sign recognition assistance. Background Art

[0002] With the rapid development of computer technology, Internet technology, intelligent manufacturing technology and other high-tech technologies, unmanned vehicle technology is becoming one of the hottest research directions in the field of automation. Unmanned vehicles have played an important role in exploring unknown environments, shopping mall navigation, logistics distribution, intelligent inspection and other fields. For unmanned vehicles, accurate estimation of their own position is an important guarantee for completing tasks, especially the problem of precise positioning in complex environments.

[0003] The existing unmanned vehicle positioning technology mainly relies on the Global Navigation Satellite System (GNSS), and its positioning accuracy is greatly affected by sensor accuracy and GNSS failures. In particular, when there is no satellite signal, such as complex terrain, scene occlusion, and electronic interference, there is a lack of absolute positioning means, making it difficult to guarantee long-term, high-precision navigation needs. In complex indoor environments such as large shopping malls and airports, conventional positioning methods cannot provide accurate location information due to problems such as signal occlusion and multipath effects.

[0004] In addition, the environmental information inside the mall is complex and diverse, including logos, store signs, wall paintings, etc. How to accurately identify the text information in the scene environment and extract this information for rapid positioning of the unmanned vehicle has become a key issue. Traditional optical character recognition (OCR) technology is usually defined as an encoder-decoder problem, mainly using convolutional neural networks (CNN) as encoders for image understanding and recurrent neural networks (RNN) or long short-term memory networks (LSTM) as decoders for text generation. In the image understanding stage, the processing method of CNN is structurally static, that is, the convolution kernel is fixed throughout the processing process, and it cannot dynamically adjust its processing process according to different input data. Especially when processing sequence data, CNN usually requires multi-layer stacking and large-size convolution kernels to try to capture longer-range dependencies, and has poor adaptability and flexibility. In the text generation stage, RNN and LSTM must process sequence data in order because the output of each step depends in part on the output of the previous step, which limits the training and inference speed of the model. In addition, when actually processing long sequences, the model's ability to capture long-distance dependencies and understand context still faces certain challenges. Summary of the invention

[0005] In order to solve the technical problem of poor real-time precision positioning of unmanned vehicles in some existing large indoor scenes such as shopping malls, airports and other GNSS-denied environments, the embodiment of the present invention provides a method and device for rapid positioning of unmanned vehicles based on store sign recognition assistance. The technical solution is as follows:

[0006] On the one hand, a method for rapid positioning of an unmanned vehicle based on store sign recognition assistance is provided, the method being implemented by an unmanned vehicle rapid positioning device, the method comprising:

[0007] S1. Obtain environmental perception data of the unmanned vehicle to be located.

[0008] S2. Input the environmental perception data into the constructed end-to-end OCR network based on the Transformer architecture to obtain the text recognition results.

[0009] S3. Perform similarity calculation on the text recognition results and the constructed feature database to obtain the position range of the unmanned vehicle; determine the initial position of the unmanned vehicle based on the position range of the unmanned vehicle and the constructed three-dimensional point cloud environment map.

[0010] S4. According to the initial position of the unmanned vehicle and the particle filter algorithm, the precise positioning of the unmanned vehicle is obtained.

[0011] Optionally, before S2, it also includes:

[0012] Establish unmanned vehicle kinematic model and unmanned vehicle observation model.

[0013] Among them, the unmanned vehicle motion model is shown in the following formula (1):

[0014] (1)

[0015] In the formula, Indicates that unmanned vehicles are The posture of the moment, Indicates movement The horizontal coordinate of the unmanned vehicle after time, Indicates movement The vertical coordinate of the unmanned vehicle after time, Indicates movement The perspective of the driverless car after time, Indicates that unmanned vehicles are The posture of the moment, Indicates that unmanned vehicles are The horizontal axis of time, Indicates that unmanned vehicles are The vertical coordinate of time, Indicates that unmanned vehicles are The angle of time, Indicates that unmanned vehicles are The control input at the moment, Indicates the time, Indicates noise.

[0016] The unmanned vehicle observation model is shown in the following formula (2):

[0017] (2)

[0018] In the formula, represents the measurement probability, express The measurement of the time sensor, Indicates a map, Indicates that in a measurement The number of internal measurements, Indicates the maximum measuring distance of the sensor. Mean ,variance The univariate normal distribution of It indicates the first position observed by the unmanned vehicle at the current position. Features, represents the standard deviation of the measurement model noise, Indicated by and After ray casting calculation, represents the noise parameter inherent to the measurement model.

[0019] Optionally, the end-to-end OCR network based on the Transformer architecture includes an encoder and a decoder.

[0020] In S2, the environmental perception data is input into the built end-to-end OCR network based on the Transformer architecture to obtain the text recognition results, including:

[0021] S21. Obtain image data from the environmental perception data, resize the image data, decompose the resized image into multiple image blocks, perform block embedding, position embedding, and image feature extraction on the multiple image blocks, and obtain an encoder output.

[0022] S22. Input the encoder output into the decoder to obtain the text recognition result.

[0023] Optionally, the feature database in S3 is as shown in the following formula (3):

[0024] (3)

[0025] In the formula, express The descriptor formed after scanning The position of the unmanned vehicle corresponding to the descriptor A collection of Represents the descriptor, i.e. the result of text recognition. represents a 3D column vector space, Indicates the number of scans.

[0026] Optionally, the process of constructing the 3D point cloud environment map in S3 includes:

[0027] S31. Establish a point cloud map of the three-dimensional environment.

[0028] S32: Fusing the point cloud map of the three-dimensional environment and the feature database to obtain a three-dimensional point cloud environment map containing semantic information.

[0029] Optionally, establishing a point cloud map of a three-dimensional environment in S31 includes:

[0030] S311. Scan the environment through laser radar to obtain point cloud data.

[0031] S312. Measure the acceleration and angular velocity information of the unmanned vehicle through an inertial measurement unit, and then obtain the posture and motion state of the unmanned vehicle.

[0032] S313, pre-processing and data synchronization are performed on the point cloud data, the posture of the unmanned vehicle, and the motion state of the unmanned vehicle to obtain processed data, and a point cloud map of the three-dimensional environment is established based on the processed data.

[0033] Optionally, the step of fusing the point cloud map of the three-dimensional environment and the feature database in S32 includes:

[0034] S321, traversing the voxels in the point cloud map of the three-dimensional environment, obtaining the mean of the point cloud in the voxels, and obtaining the coordinate information corresponding to the mean.

[0035] S322. Traverse the unmanned vehicle positions in the feature database to obtain coordinate information corresponding to the unmanned vehicle positions.

[0036] S323. Calculate the distance between the coordinate information corresponding to the mean and the coordinate information corresponding to the position of the unmanned vehicle, and set the descriptor corresponding to the voxel according to the distance.

[0037] On the other hand, a device for quickly positioning an unmanned vehicle based on shop sign recognition assistance is provided, and the device is applied to a method for quickly positioning an unmanned vehicle based on shop sign recognition assistance, and the device comprises:

[0038] The acquisition module is used to obtain the environmental perception data of the unmanned vehicle to be located.

[0039] The text recognition module is used to input environmental perception data into the built end-to-end OCR network based on the Transformer architecture to obtain text recognition results.

[0040] The initial position determination module is used to calculate the similarity between the text recognition results and the constructed feature database to obtain the position range of the unmanned vehicle; based on the position range of the unmanned vehicle and the constructed three-dimensional point cloud environment map, the initial position of the unmanned vehicle is determined.

[0041] The output module is used to obtain the precise positioning of the unmanned vehicle based on the initial position of the unmanned vehicle and the particle filter algorithm.

[0042] Optionally, the text recognition module is further used to:

[0043] Establish unmanned vehicle kinematic model and unmanned vehicle observation model.

[0044] Among them, the unmanned vehicle motion model is shown in the following formula (1):

[0045] (1)

[0046] In the formula, Indicates that unmanned vehicles are The posture of the moment, Indicates movement The horizontal coordinate of the unmanned vehicle after time, Indicates movement The vertical coordinate of the unmanned vehicle after time, Indicates movement The perspective of the driverless car after time, Indicates that unmanned vehicles are The posture of the moment, Indicates that unmanned vehicles are The horizontal axis of time, Indicates that unmanned vehicles are The vertical coordinate of time, Indicates that unmanned vehicles are The angle of time, Indicates that unmanned vehicles are The control input at the moment, Indicates the time, Indicates noise.

[0047] The unmanned vehicle observation model is shown in the following formula (2):

[0048] (2)

[0049] In the formula, represents the measurement probability, express The measurement of the time sensor, Indicates a map, Indicates that in a measurement The number of internal measurements, Indicates the maximum measuring distance of the sensor. Mean ,variance The univariate normal distribution of It indicates the first position observed by the unmanned vehicle at the current position. Features, represents the standard deviation of the measurement model noise, Indicated by and After ray casting calculation, represents the noise parameter inherent to the measurement model.

[0050] Optionally, the end-to-end OCR network based on the Transformer architecture includes an encoder and a decoder.

[0051] The text recognition module is further used to:

[0052] S21. Obtain image data from the environmental perception data, resize the image data, decompose the resized image into multiple image blocks, perform block embedding, position embedding, and image feature extraction on the multiple image blocks, and obtain an encoder output.

[0053] S22. Input the encoder output into the decoder to obtain the text recognition result.

[0054] Optionally, the feature database is as shown in the following formula (3):

[0055] (3)

[0056] In the formula, express The descriptor formed after scanning The position of the unmanned vehicle corresponding to the descriptor A collection of Represents the descriptor, i.e. the result of text recognition. represents a 3D column vector space, Indicates the number of scans.

[0057] Optionally, the initial position determination module is further used to:

[0058] S31. Establish a point cloud map of the three-dimensional environment.

[0059] S32: Fusing the point cloud map of the three-dimensional environment and the feature database to obtain a three-dimensional point cloud environment map containing semantic information.

[0060] Optionally, the initial position determination module is further used to:

[0061] S311. Scan the environment through laser radar to obtain point cloud data.

[0062] S312. Measure the acceleration and angular velocity information of the unmanned vehicle through an inertial measurement unit, and then obtain the posture and motion state of the unmanned vehicle.

[0063] S313, pre-processing and data synchronization are performed on the point cloud data, the posture of the unmanned vehicle, and the motion state of the unmanned vehicle to obtain processed data, and a point cloud map of the three-dimensional environment is established based on the processed data.

[0064] Optionally, the initial position determination module is further used to:

[0065] S321, traversing the voxels in the point cloud map of the three-dimensional environment, obtaining the mean of the point cloud in the voxels, and obtaining the coordinate information corresponding to the mean.

[0066] S322. Traverse the unmanned vehicle positions in the feature database to obtain coordinate information corresponding to the unmanned vehicle positions.

[0067] S323. Calculate the distance between the coordinate information corresponding to the mean and the coordinate information corresponding to the position of the unmanned vehicle, and set the descriptor corresponding to the voxel according to the distance.

[0068] On the other hand, a device for rapid positioning of an unmanned vehicle is provided, comprising: a processor; a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, any one of the above-mentioned methods for rapid positioning of an unmanned vehicle assisted by store sign recognition is implemented.

[0069] On the other hand, a computer-readable storage medium is provided, in which at least one instruction is stored. The at least one instruction is loaded and executed by a processor to implement any one of the above-mentioned methods for rapid positioning of unmanned vehicles based on the assistance of store sign recognition.

[0070] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0071] In the embodiment of the present invention, the end-to-end OCR network based on the Transformer architecture is more concise and efficient, without the need for complex pre-processing or post-processing, and avoids the complexity of multiple independent modules in traditional methods. The self-attention mechanism of the model enables the model to parallelize processing and effectively capture long-distance dependencies in the sequence, thus showing higher accuracy and real-time performance in semantic understanding and text recognition tasks.

[0072] The method of constructing a descriptor matching feature database can obtain a relatively accurate initial position estimate of the unmanned vehicle without relying on the GNSS system, and quickly index the local map of the global map of the large shopping mall environment stored on the server, greatly saving time.

[0073] The particle filter algorithm has strong nonlinear processing capabilities and can continue to provide reliable estimates when the measurement noise is large or the environment changes, and has strong robustness. In addition, the particle swarm-based filtering algorithm is easy to integrate prior knowledge and effectively handle multimodal distribution. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0075] Figure 1 This is a flow chart of a method for rapid positioning of an unmanned vehicle based on store sign recognition assistance provided by an embodiment of the present invention;

[0076] Figure 2 This is an overall structural diagram of an unmanned vehicle rapid positioning method based on store sign recognition assistance provided by an embodiment of the present invention;

[0077] Figure 3 is an OCR network structure diagram provided by an embodiment of the present invention;

[0078] Figure 4 is a structural diagram of a self-attention mechanism provided by an embodiment of the present invention;

[0079] Figure 5 It is a multi-head attention structure diagram provided by an embodiment of the present invention;

[0080] Figure 6 It is a flowchart of constructing a three-dimensional environment point cloud map provided by an embodiment of the present invention;

[0081] Figure 7 is a flow chart of constructing a feature database provided by an embodiment of the present invention;

[0082] Figure 8 is a flow chart of a particle filtering algorithm provided by an embodiment of the present invention;

[0083] Fig. 9 This is a block diagram of an unmanned vehicle rapid positioning device based on store sign recognition assistance provided by an embodiment of the present invention;

[0084] Fig.10 It is a structural schematic diagram of an unmanned vehicle rapid positioning device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0085] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0086] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0087] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same.

[0088] In the embodiments of the present invention, sometimes a subscript such as W1 may be written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are the same.

[0089] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0090] An embodiment of the present invention provides a method for rapid positioning of an unmanned vehicle based on the assistance of store sign recognition. The method can be implemented by an unmanned vehicle rapid positioning device, which can be a terminal or a server. The present invention aims to identify the text in the signs, store signs, and wall paintings in the shopping mall and match them with the feature database through an end-to-end OCR network based on the Transformer architecture without relying on the GNSS system. First, the approximate location range of the unmanned vehicle is located, and this information is quickly indexed into the local three-dimensional map data corresponding to the location of the unmanned vehicle in the stored large-scale three-dimensional point cloud environmental map data, so as to achieve the purpose of rapid positioning. On this basis, the particle filtering algorithm is applied to realize the precise positioning of the unmanned vehicle based on the three-dimensional environmental map. Figure 1 , Figure 2 The flowchart of the unmanned vehicle rapid positioning method based on store sign recognition assistance is shown in the figure. The processing flow of the method may include the following steps:

[0091] S1. Obtain environmental perception data of the unmanned vehicle to be located.

[0092] In a feasible implementation, the environmental perception data may include scene images scanned by the unmanned vehicle, etc.

[0093] Optionally, before S2, it also includes:

[0094] Establish unmanned vehicle kinematic model and unmanned vehicle observation model.

[0095] In a feasible implementation, it is assumed that the unmanned vehicle is in an ideal state (no noise), The control input at time is , express The speed of unmanned vehicles at all times, express The angular velocity of the unmanned vehicle at time Motion on a circle with a radius of .

[0096] make Indicates that unmanned vehicles are The position of the moment, and in If the inner velocity remains constant, the center coordinate of the circle for:

[0097] (1)

[0098] sports After a period of time, the driverless car is located , and there are:

[0099] (2)

[0100] However, the movement of the unmanned vehicle is always affected by noise. Therefore, the actual speed is different from the measured speed. Therefore, the noise is taken into account in the unmanned vehicle speed model, and the error variable is introduced and the actual speed is assumed to be given by the following formula:

[0101] (3)

[0102] in, The mean is 0 and the variance is The error variable, represents the real speed of the unmanned vehicle under the influence of noise, Represents the true angular velocity of the unmanned vehicle under the influence of noise. The true velocity is equal to the given velocity plus the error noise. Parameters is the error parameter specific to the unmanned vehicle. It is a normal distribution, and its probability density function is as follows:

[0103] (4)

[0104] exist After executing the motion command The final pose can be calculated as follows:

[0105] (5)

[0106] However, since the radius of the arc segment and the distance moved are also affected by noise, the trajectory is not a circle. Assume that the unmanned vehicle rotates when it reaches the final position. , ,in , and is the additional rotation noise variance and specific robot parameters. So the motion model of the unmanned vehicle is:

[0107] (6)

[0108] Write the above formula as:

[0109] (7)

[0110] In the formula, Indicates that unmanned vehicles are The posture of the moment, Indicates movement The horizontal coordinate of the unmanned vehicle after time, Indicates movement The vertical coordinate of the unmanned vehicle after time, Indicates movement The perspective of the driverless car after time, Indicates that unmanned vehicles are The posture of the moment, Indicates that unmanned vehicles are The horizontal axis of time, Indicates that unmanned vehicles are The vertical coordinate of time, Indicates that unmanned vehicles are The angle of time, Indicates that unmanned vehicles are The control input at the moment, Indicates the time, Indicates noise.

[0111] Furthermore, the observation model of the unmanned vehicle is established as follows: express The measurement of the time sensor, Indicates that in a measurement The number of internal measurements is , is an independent measurement, assuming that each measurement is independent of each other, then It can be obtained by multiplying the likelihood of each single measurement:

[0112] (8)

[0113] Map is a list of objects in the environment and their properties:

[0114] (9)

[0115] is the total number of objects in the environment, Specifies an attribute.

[0116] Measuring probability Given by:

[0117] (10)

[0118] in, Depend on and After ray casting calculation; is the mean ,variance The univariate normal distribution of is the noise parameter inherent in the measurement model, namely:

[0119] (11)

[0120] Normalization factor for:

[0121] (12)

[0122] Therefore, the observation model of the unmanned vehicle can be expressed as follows:

[0123] (13)

[0124] In the formula, represents the measurement probability, express The measurement of the time sensor, Indicates a map, Indicates that in a measurement The number of internal measurements, Indicates the maximum measuring distance of the sensor. Mean ,variance The univariate normal distribution of It indicates the first position observed by the unmanned vehicle at the current position. Features, represents the standard deviation of the measurement model noise, Indicated by and After ray casting calculation, represents the noise parameter inherent to the measurement model.

[0125] S2. Input the environmental perception data into the constructed end-to-end OCR network based on the Transformer architecture to obtain the text recognition results.

[0126] Optionally, the end-to-end OCR network based on the Transformer architecture includes an encoder and a decoder.

[0127] The above step S2 may include the following steps S21-S22:

[0128] S21. Obtain image data from the environmental perception data, resize the image data, decompose the resized image into multiple image blocks, perform block embedding, position embedding, and image feature extraction on the multiple image blocks, and obtain an encoder output.

[0129] S22. Input the encoder output into the decoder to obtain the text recognition result.

[0130] In a feasible implementation, in order to improve the accuracy of text recognition, an end-to-end OCR network based on the Transformer architecture is designed and trained. The network is built using the Transformer architecture, including an image transformer for extracting visual features and a text transformer for language modeling. The purpose of the encoder is to obtain the representation of the image block, and the purpose of the decoder is to generate a word sequence under the guidance of visual features and previous predictions. That is, the image to be inferred is input into the OCR network, and after the encoder and decoder, the text recognition result corresponding to the text information in the image is finally output. Figure 3 This is the structural diagram of the OCR network.

[0131] Specifically, the encoder receives an input image , and resize it to a fixed size . Decompose the input image into image blocks, each of which has a size of , while ensuring that the width W and height H of the adjusted image are divisible by P. The image block is then flattened and linearly projected as dimensional vector, namely, patch embedding. is the hidden size of all layers of Transformer. Compared with traditional RNN and LSTM networks, since Transformer uses global information, it is also necessary to use position embedding to represent the position of the image block in the image. After extracting image features, the encoder sends the output to the decoder responsible for text prediction. The decoder predicts the final text recognition result through iteration.

[0132] Furthermore, the encoder is mainly composed of Multi-Head Attention and Feed Forward Network (FFN). The structure of the Self-Attention mechanism is as follows Figure 4 As shown, by calculating the query and key The dot product between them is used to measure the similarity between them, and this similarity is used as the weight to weight the value MatMul represents matrix multiplication, and the Scale operation is used to scale the dot product result.

[0133] The SoftMax function is defined as follows:

[0134] (14)

[0135] in For the The output of the node, is the number of output nodes.

[0136] From the above structure, we know the output of the self-attention mechanism:

[0137] (15)

[0138] in For the matrix Dimension.

[0139] In order to make the model pay attention to different aspects of information, the model is divided into multiple heads to form multiple subspaces. Multi-Head Attention is formed by combining multiple self-attention mechanisms, such as Figure 5 As shown in the figure, Linear is the linear layer, h is the number of self-attention mechanisms, and the Concat layer is used to realize tensor concatenation.

[0140] The calculation formula of the multi-head attention mechanism is as follows:

[0141] (16)

[0142] (17)

[0143] (18)

[0144] in , , , is the parameter matrix.

[0145] The feedforward neural network FFN consists of two fully connected layers. The activation function used in the first layer is Relu, and the second layer does not use an activation function. The corresponding formula is as follows:

[0146] (19)

[0147] in , is the parameter matrix, , For bias.

[0148] Furthermore, the decoder is composed of Masked Multi-Head Attention, Multi-Head Attention, and Feedforward Neural Network (FFN). The decoder inserts "encoder-decoder attention" between the multi-head self-attention and feedforward network to assign different attention to the output of the encoder. In the encoder-decoder self-attention module, the key Sum comes from the encoder output, and the query from the decoder input. In addition, the decoder uses masked multi-head attention to prevent itself from obtaining more information than the prediction during training. That is, by masking access to future words during the self-attention calculation, the causality of the generation process is ensured.

[0149] Model training: In order to improve the performance of the model and reduce the training time, the encoder and decoder are not trained from scratch, but initialized using models that have been pre-trained on large-scale data. The encoder is initialized using the DeiT model, and the decoder is initialized using the RoBERTa model.

[0150] The training process is divided into two different stages according to the dataset used. In the first stage, in order to ensure the quality and comprehensiveness of the dataset, document pages are extracted from PDF files publicly available on the Internet, and the documents should cover both Chinese and English corpora. They are converted into page images and text lines are extracted from the cropped images to build a large dataset, and the OCR model is trained on this basis. The second stage is initialized on the model trained in the first stage. For the specific task of text recognition in large shopping malls to assist unmanned vehicle positioning, a dataset in a large shopping mall is constructed, which contains scene logos, store signs, wall paintings, etc., and a separate model is trained for this specific dataset for text recognition in large shopping malls.

[0151] S3. Perform similarity calculation on the text recognition results and the constructed feature database to obtain the position range of the unmanned vehicle; determine the initial position of the unmanned vehicle based on the position range of the unmanned vehicle and the constructed three-dimensional point cloud environment map.

[0152] In a feasible implementation, a three-dimensional environment semantic point cloud map including store sign information is constructed based on semantic SLAM technology.

[0153] In an embodiment of the present invention, a point cloud map of a three-dimensional environment is first established, then a scene descriptor and a feature database are constructed, and finally the feature database is fused with the established point cloud map as semantic information to obtain a semantic point cloud map of the three-dimensional environment containing store sign information.

[0154] Optionally, the process of constructing the three-dimensional point cloud environment map in S3 may include S31- S32:

[0155] S31. Establish a point cloud map of the three-dimensional environment.

[0156] Optionally, the above step S31 may include the following steps S311-S313:

[0157] S311. Scan the environment through laser radar to obtain point cloud data.

[0158] S312. Measure the acceleration and angular velocity information of the unmanned vehicle through an inertial measurement unit, and then obtain the posture and motion state of the unmanned vehicle.

[0159] S313, pre-processing and data synchronization are performed on the point cloud data, the posture of the unmanned vehicle, and the motion state of the unmanned vehicle to obtain processed data, and a point cloud map of the three-dimensional environment is established based on the processed data.

[0160] The process of constructing a 3D environment point cloud map is as follows: Figure 6 As shown. Data collection is completed by Lidar and Inertial Measurement Unit (IMU). Lidar scans the environment and obtains point cloud data. At the same time, IMU obtains the attitude and motion state of the vehicle by measuring the acceleration and angular velocity information of the unmanned vehicle. In the preprocessing stage, the acquired radar point cloud data is noise filtered to remove ground points and abnormal points and retain key feature points. IMU data is integrated to obtain speed and position information. Due to the different data acquisition frequencies of radar and IMU, the data of both need to be synchronized to the same timestamp. Then, the Normal Distributions Transform (NDT) algorithm is used to coarsely align the point cloud data, align the point cloud data of different perspectives or times to the same coordinate system, and use IMU data for fine alignment and pose estimation. In the pose optimization stage, graph optimization is used to eliminate ghosting caused by errors. According to the estimated pose, all registered point clouds are fused into a global map. Finally, the 3D environment point cloud map is exported.

[0161] Furthermore, the point cloud matching uses the NDT algorithm, which can handle large-scale point cloud data and has good robustness to noise and missing data. In addition, since it is a probability-based method, it can provide uncertainty estimates of the registration results. The algorithm flow is as follows:

[0162] 1. Divide the target point cloud space into several voxels at a certain resolution ;

[0163] 2. For each point cloud in the voxel, calculate its mean and the covariance matrix , define a multivariate normal distribution with these parameters;

[0164] 3. Calculate the position of the point in the source point cloud in the voxel of the target point cloud and establish the residual between the point and the voxel;

[0165] 4. Use the Gauss-Newton method to iterate the estimated pose and optimize the pose estimate.

[0166] Let the point in the source point cloud be go through , After transformation, the voxel middle, The statistical indicators are , , then the residual in the grid is:

[0167] (20)

[0168] Formulate a weighted least squares problem:

[0169] (twenty one)

[0170] The weighted least squares Gauss-Newton method is:

[0171] (twenty two)

[0172] In the formula, is the increment of each iteration, is the Jacobian matrix of the residuals with respect to the independent variables.

[0173] After the optimization process is completed, the optimal transformation obtained is the registration result between the source point cloud and the target point cloud.

[0174] Furthermore, the scene descriptor and feature database are constructed as follows:

[0175] Due to the different distances and angles of the camera, different scenes are different, resulting in different descriptors. For each scan data of the camera, it is input into the OCR network and the corresponding descriptor of the scene is generated, and the position of the unmanned vehicle at that time is recorded, so as to establish the connection between the descriptor and the position of the unmanned vehicle, and finally form a feature database of the mall map.

[0176] This step is modeled as follows: Assume that the unmanned vehicle The scene is scanned once, and the RGB image information recorded by the camera is ,Will Input into the OCR network, and the corresponding text output set is , Included All text information in As a descriptor. At the same time, the position of the unmanned vehicle at that moment is recorded , thus building a descriptor Location of unmanned vehicle In order to ensure that the initial position estimate of the unmanned vehicle can be accurately obtained in the subsequent positioning stage, so as to quickly index the sub-map under the global map of the shopping mall, the database built should ensure that the descriptor covers the signs and store signs in the shopping mall. Finally, the feature database set of the shopping mall map is obtained:

[0177] (twenty three)

[0178] In the formula, The element is The descriptor information formed after scanning The position of the unmanned vehicle corresponding to the descriptor The feature database will be used in the subsequent positioning process. The feature database construction process is as follows: Figure 7 As shown, Represents the descriptor, i.e. the result of text recognition. represents a 3D column vector space, Indicates the number of scans.

[0179] S32: Fusing the point cloud map of the three-dimensional environment and the feature database to obtain a three-dimensional point cloud environment map containing semantic information.

[0180] Optionally, fusing the point cloud map of the three-dimensional environment and the feature database in S32 may include S321-S323:

[0181] S321, traversing the voxels in the point cloud map of the three-dimensional environment, obtaining the mean of the point cloud in the voxels, and obtaining the coordinate information corresponding to the mean.

[0182] S322. Traverse the unmanned vehicle positions in the feature database to obtain coordinate information corresponding to the unmanned vehicle positions.

[0183] S323. Calculate the distance between the coordinate information corresponding to the mean and the coordinate information corresponding to the position of the unmanned vehicle, and set the descriptor corresponding to the voxel according to the distance.

[0184] In a feasible implementation, the steps of fusing the feature database with the point cloud map are as follows: traverse the voxels of the three-dimensional environment point cloud map And get the corresponding mean , record the coordinate information corresponding to the mean map point . Traverse the database In , record the location Corresponding coordinate information .

[0185] Define distance:

[0186] (twenty four)

[0187] Pick ,but Voxel To location The closest distance to the voxel, so the semantic label of the voxel is set to the descriptor .

[0188] This step realizes the fusion of the feature database and the point cloud map, and finally establishes a three-dimensional environmental semantic point cloud map containing store sign information.

[0189] Furthermore, the similarity between the scene descriptor and the database is calculated to infer the initial position estimate of the unmanned vehicle.

[0190] Specifically, this step is the positioning link. The camera scans the scene environment and inputs it into the OCR network. The network output (scene descriptor) is matched with the feature database to calculate the similarity to infer the initial position estimate of the unmanned vehicle and quickly index the sub-map of the global map stored on the server.

[0191] Jaccard similarity is used to measure the similarity between the scene descriptor and the database. Jaccard similarity is a statistical method used to measure the similarity between two sets. It is defined as the size of the intersection of two sets divided by the size of the union. That is, given two sets , , define Jaccard similarity:

[0192] (25)

[0193] here, Representing a collection and collection The number of elements in the intersection, Representing a collection and collection The number of elements in the union of . If two sets are completely disjoint (i.e. they have no elements in common), then their Jaccard similarity is 0. If two sets are identical (i.e. all of their elements are identical), then their Jaccard similarity is 1.

[0194] Suppose the scene environment scanned at the initial moment is output by the OCR network as the scene descriptor ,calculate and database Each descriptor in Jaccard similarity:

[0195] (26)

[0196] Set the initial position of the unmanned vehicle The probability distribution of ,definition:

[0197] (27)

[0198] Then we have:

[0199] (28)

[0200] Since the text collection The associated unmanned vehicle position is , so use the associated location information To initialize, Then the descriptor Matching with the 3D environment semantic point cloud map, indexing the semantic labels in the global map The local point cloud map composed of voxels is used for point cloud matching.

[0201] S4. According to the initial position of the unmanned vehicle and the particle filter algorithm, the precise positioning of the unmanned vehicle is obtained.

[0202] In a feasible implementation, accurate position and posture estimation of the unmanned vehicle is achieved based on a particle filtering algorithm.

[0203] Specifically, the LiDAR collects point cloud data in real time and matches it with the indexed local point cloud map. Accurate pose estimation based on particle filtering is achieved by fusing LiDAR and IMU data.

[0204] The basic idea of ​​particle filter-based positioning is as follows: the particle filter first initializes a set of particles, each particle represents the possible position of the unmanned vehicle. When the unmanned vehicle moves, the particle filter will update the position of each particle according to the motion model of the unmanned vehicle. When the unmanned vehicle obtains new observation data, the particle filter will update the weight of each particle based on this data. The weight of the particle indicates the consistency between the position represented by the particle and the observation data. Finally, the particle filter will perform a resampling operation. In this process, particles with high weights have a greater probability of being selected, while particles with low weights have a greater probability of being discarded. In this way, the particle collection will become more and more concentrated near the actual position of the unmanned vehicle. The above process will be repeated continuously, allowing the particle filter to track and update the position of the unmanned vehicle in real time. The specific process of the particle filter algorithm is as follows Figure 8 As shown:

[0205] Initialize the particle swarm: Assume that the initial estimated position of the unmanned vehicle is , from step 3 we know ,exist The initial particle swarm is generated by sampling using Gaussian distribution nearby, and a weight is assigned to each particle. dimensional random variable The probability density function of the Gaussian distribution is:

[0206] (29)

[0207] in, is the covariance matrix, is the mean vector.

[0208] Assume the sampling noise is , and remember for The largest variance across all dimensions, is the total number of particles sampled, and the particles obtained after sampling are recorded as , the weight of the particle is , the calculation formula is as follows:

[0209] (30)

[0210] in For the sampling particles and The normalized weight of the particle is:

[0211] (31)

[0212] Then the set of initialized particle swarm is:

[0213] (32)

[0214] State prediction: The particle swarm set at the moment is:

[0215] (33)

[0216] From the kinematic model of the unmanned vehicle, we know Particle swarm at time: , the initial estimate exist The position at the moment is .

[0217] Weight update: When the unmanned vehicle obtains new observation data, the particle filter will update the weight of each particle based on this data. The weight of the particle indicates the consistency between the position represented by the particle and the observation data.

[0218] set up Sensor posture at all times The observed value is , Observed value .

[0219] Then the updated particle weight is:

[0220] (34)

[0221] in is the sensor measurement noise, and the normalized weight expression is: ,therefore The particle swarm set at the moment is:

[0222] (35)

[0223] Resampling: In the update step of the particle filter, the weight of each particle is updated according to its consistency with the observed data. Generally, only a few particles' predictions are consistent with the observed data, and the weights of these particles become very large, while the weights of other particles become very small. Over time, the weights of most particles will approach zero, and these particles will make almost no contribution in representing the posterior probability distribution. Therefore, it is necessary to determine whether resampling is needed.

[0224] set up is the weight of the particle before resampling, is the weight of the particle after resampling. When , the particle swarm has particle degradation, and the particle swarm needs to be resampled. The total number of particles after resampling remains unchanged.

[0225] The resampling method is as follows: Calculate the cumulative probability .

[0226] Let the random variable obey Uniform distribution on the interval, for of sample observations, for each random number , if the smallest Make , then let , .

[0227] when hour, .

[0228] According to the above method, particles with relatively large weights will be replicated multiple times, and particles that contribute very little to the calculation of the posterior probability will be eliminated, thereby avoiding the problem of weight degradation and improving the performance of the particle filter. Since the previous step obtains a relatively accurate initial position of the unmanned vehicle, the particle filter algorithm can converge quickly and track and update the precise position of the unmanned vehicle in real time.

[0229] The present invention establishes a kinematic model and an observation model for an unmanned vehicle, trains an end-to-end OCR network based on a Transformer architecture to improve the accuracy of text recognition and establishes a feature database for a shopping mall map. The OCR technology is used to assist in obtaining the approximate position of the unmanned vehicle. Finally, the particle filter technology is applied to estimate the precise position and posture of the unmanned vehicle by matching the real-time collected environmental point cloud data with the constructed three-dimensional environmental point cloud data. The method is computationally efficient, can achieve good initial position estimation and fast indexing of local maps, and improves the speed and accuracy of unmanned vehicle positioning.

[0230] With the help of information such as store signs and advertising signs in rich scenes, based on optical character recognition real-time detection technology, the approximate range of the unmanned vehicle can be quickly determined, and the local three-dimensional map data corresponding to the location of the unmanned vehicle can be indexed. Furthermore, particle filtering technology is used to achieve rapid and precise positioning of the unmanned vehicle based on the three-dimensional point cloud map.

[0231] In the embodiment of the present invention, the end-to-end OCR network based on the Transformer architecture is more concise and efficient, without the need for complex pre-processing or post-processing, and avoids the complexity of multiple independent modules in traditional methods. The self-attention mechanism of the model enables the model to parallelize processing and effectively capture long-distance dependencies in the sequence, thus showing higher accuracy and real-time performance in semantic understanding and text recognition tasks.

[0232] The method of constructing a descriptor matching feature database can obtain a relatively accurate initial position estimate of the unmanned vehicle without relying on the GNSS system, and quickly index the local map of the global map of the large shopping mall environment stored on the server, greatly saving time.

[0233] The particle filter algorithm has strong nonlinear processing capabilities and can continue to provide reliable estimates when the measurement noise is large or the environment changes, and has strong robustness. In addition, the particle swarm-based filtering algorithm is easy to integrate prior knowledge and effectively handle multimodal distribution.

[0234] Fig. 9 The block diagram of a device for rapid positioning of an unmanned vehicle based on shop sign recognition assistance according to an exemplary embodiment is shown. The device is used for a method for rapid positioning of an unmanned vehicle based on shop sign recognition assistance. Fig. 9 The device includes an acquisition module 310, a text recognition module 320, an initial position determination module 330 and an output module 340. Among them:

[0235] The acquisition module 310 is used to acquire the environmental perception data of the unmanned vehicle to be located.

[0236] The text recognition module 320 is used to input the environmental perception data into the constructed end-to-end OCR network based on the Transformer architecture to obtain the text recognition result.

[0237] The initial position determination module 330 is used to perform similarity calculation on the text recognition results and the constructed feature database to obtain the position range of the unmanned vehicle; and determine the initial position of the unmanned vehicle based on the position range of the unmanned vehicle and the constructed three-dimensional point cloud environment map.

[0238] The output module 340 is used to obtain the precise positioning of the unmanned vehicle according to the initial position of the unmanned vehicle and the particle filter algorithm.

[0239] In the embodiment of the present invention, the end-to-end OCR network based on the Transformer architecture is more concise and efficient, without the need for complex pre-processing or post-processing, and avoids the complexity of multiple independent modules in traditional methods. The self-attention mechanism of the model enables the model to parallelize processing and effectively capture long-distance dependencies in the sequence, thus showing higher accuracy and real-time performance in semantic understanding and text recognition tasks.

[0240] The method of constructing a descriptor matching feature database can obtain a relatively accurate initial position estimate of the unmanned vehicle without relying on the GNSS system, and quickly index the local map of the global map of the large shopping mall environment stored on the server, greatly saving time.

[0241] The particle filter algorithm has strong nonlinear processing capabilities and can continue to provide reliable estimates when the measurement noise is large or the environment changes, and has strong robustness. In addition, the particle swarm-based filtering algorithm is easy to integrate prior knowledge and effectively handle multimodal distribution.

[0242] Fig.10 is a schematic diagram of the structure of an unmanned vehicle rapid positioning device provided by an embodiment of the present invention, such as Fig.10 As shown, the unmanned vehicle rapid positioning device may include the above Fig. 9 The unmanned vehicle rapid positioning device based on shop sign recognition assistance is shown. Optionally, the unmanned vehicle rapid positioning device 410 may include a first processor 2001.

[0243] Optionally, the unmanned vehicle rapid positioning device 410 may also include a memory 2002 and a transceiver 2003 .

[0244] The first processor 2001, the memory 2002 and the transceiver 2003 may be connected via a communication bus.

[0245] Combine the following Fig.10 The components of the unmanned vehicle rapid positioning device 410 are specifically introduced:

[0246] The first processor 2001 is the control center of the unmanned vehicle rapid positioning device 410, which can be a processor or a general term for multiple processing elements. For example, the first processor 2001 is one or more central processing units (CPUs), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (field programmable gate arrays, FPGAs).

[0247] Optionally, the first processor 2001 can perform various functions of the unmanned vehicle rapid positioning device 410 by running or executing a software program stored in the memory 2002 and calling data stored in the memory 2002.

[0248] In a specific implementation, as an embodiment, the first processor 2001 may include one or more CPUs, such as Fig.10 CPU0 and CPU1 are shown in FIG.

[0249] In a specific implementation, as an embodiment, the unmanned vehicle rapid positioning device 410 may also include multiple processors, such as Fig.10 The first processor 2001 and the second processor 2004 are shown in FIG. Each of these processors may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0250] The memory 2002 is used to store the software program for executing the solution of the present invention, and is controlled to be executed by the first processor 2001. The specific implementation method can refer to the above method embodiment, which will not be repeated here.

[0251] Optionally, the memory 2002 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 2002 may be integrated with the first processor 2001, or may exist independently and access the first processor 2001 through the interface circuit ( Fig.10 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0252] The transceiver 2003 is used to communicate with a network device or a terminal device.

[0253] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Fig.10 The receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

[0254] Optionally, the transceiver 2003 may be integrated with the first processor 2001, or may exist independently and be connected to the first processor 2001 through the interface circuit ( Fig.10 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0255] It should be noted that Fig.10 The structure of the unmanned vehicle rapid positioning device 410 shown in the figure does not constitute a limitation on the router. The actual knowledge structure recognition device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0256] In addition, the technical effects of the unmanned vehicle rapid positioning device 410 can refer to the technical effects of the unmanned vehicle rapid positioning method based on store sign recognition assistance described in the above method embodiment, and will not be repeated here.

[0257] It should be understood that the first processor 2001 in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0258] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0259] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0260] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0261] In the present invention, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0262] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0263] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0264] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0265] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0266] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0267] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0268] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0269] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for rapid positioning of unmanned vehicles based on store sign recognition assistance, characterized in that: The method comprises: S1. Obtain environmental perception data of the unmanned vehicle to be located; S2, inputting the environmental perception data into the constructed end-to-end OCR network based on the Transformer architecture to obtain a text recognition result; S3, determining the initial position of the unmanned vehicle according to the text recognition result and the constructed three-dimensional point cloud environment map; S4, obtaining accurate positioning of the unmanned vehicle according to the initial position of the unmanned vehicle and the particle filter algorithm; The process of constructing the three-dimensional point cloud environment map in S3 includes: S31, establishing a point cloud map of the three-dimensional environment; S32, constructing a scene descriptor and a feature database, and fusing the feature database as semantic information with the constructed point cloud map to obtain a three-dimensional point cloud environment map containing semantic information; The step of establishing a point cloud map of a three-dimensional environment in S31 includes: S311, scanning the environment through laser radar to obtain point cloud data; S312, measuring the acceleration and angular velocity information of the unmanned vehicle through an inertial measurement unit, thereby obtaining the posture and motion state of the unmanned vehicle; S313, preprocessing and data synchronization of the point cloud data, the posture of the unmanned vehicle, and the motion state of the unmanned vehicle to obtain processed data, and establishing a point cloud map of the three-dimensional environment according to the processed data; The step of constructing the scene descriptor and feature database in S32 includes: Assume that the unmanned vehicle scans the scene for the kth time, and the RGB image information recorded by the camera is p k , p k Input to the OCR network, and the corresponding text output set is A k , A k Contains p k All the text information in A k As a descriptor; and record the position α corresponding to the unmanned vehicle at that moment k =(x k y k θ k ) T , thus establishing the descriptor A k and the position of the unmanned vehicle α k In order to ensure that the initial position estimate of the unmanned vehicle can be accurately obtained in the subsequent positioning stage, so as to quickly index the sub-map under the global map of the shopping mall, the database built should ensure that the descriptor covers the signs and store signs in the shopping mall; finally, the feature database set of the shopping mall map is obtained: Where D represents the descriptor A formed after n scans k And the position of the unmanned vehicle corresponding to the descriptor α k A collection of k Represents the descriptor, i.e. the result of text recognition. represents the 3D column vector space, n represents the number of scans; The step of fusing the point cloud map of the three-dimensional environment and the feature database in S32 includes: S321, traversing the voxels in the point cloud map of the three-dimensional environment, obtaining the mean of the point cloud in the voxels, and obtaining the coordinate information corresponding to the mean; S322, traversing the unmanned vehicle positions in the feature database to obtain coordinate information corresponding to the unmanned vehicle positions; S323, calculating the distance between the coordinate information corresponding to the mean and the coordinate information corresponding to the position of the unmanned vehicle, and setting the descriptor corresponding to the voxel according to the distance; The S3 further includes: The scene descriptor is matched with the feature database to calculate the similarity to infer the initial position of the unmanned vehicle, including: Jaccard similarity is used to measure the similarity between the scene descriptor and the database. Given two sets P and Q, Jaccard similarity is defined as: Here, |P∩Q| represents the number of elements in the intersection of set P and set Q, |P∪Q| represents the number of elements in the union of set P and set Q, J(P,Q)∈[0,1]; Assume that the scene descriptor output by the OCR network at the initial moment is B0, and calculate the value of B0 and each descriptor A in the database D. i Jaccard similarity: Assume the probability distribution of the initial position x0 of the unmanned vehicle is P(x0|z0,m), and define: Where z0 represents the sensor measurement at time t = 0, and m represents the map; Then we have: Since the descriptor A M The associated unmanned vehicle position is α m , so the associated position information α is used m Initialize, that is, let x0 = α m , then descriptor A M Match with the 3D environment semantic point cloud map and index the semantic label A in the global map M The local point cloud map composed of voxels is used for point cloud matching; The step S4 obtains accurate positioning of the unmanned vehicle according to the initial position of the unmanned vehicle and the particle filter algorithm, including: The lidar collects point cloud data in real time and matches it with the indexed local point cloud map. By fusing the lidar and IMU data, accurate pose estimation of the unmanned vehicle based on the particle filter algorithm is achieved.

2. The unmanned vehicle rapid positioning method based on store sign recognition assistance according to claim 1 is characterized in that: Before the S2, it also includes: Establish unmanned vehicle kinematic model and unmanned vehicle observation model; The unmanned vehicle motion model is as shown in the following formula (6): x t =f(x t-1 ,u t )+ε (6) In the formula, x t =(x′y′θ′) T represents the position of the unmanned vehicle at time t, x′ represents the horizontal coordinate of the unmanned vehicle after moving for Δt time, y′ represents the vertical coordinate of the unmanned vehicle after moving for Δt time, θ′ represents the angle of the unmanned vehicle after moving for Δt time, and x t-1 =(xyθ) T represents the position of the unmanned vehicle at time t-1, x represents the horizontal coordinate of the unmanned vehicle at time t-1, y represents the vertical coordinate of the unmanned vehicle at time t-1, θ represents the angle of the unmanned vehicle at time t-1, and u t represents the control input of the unmanned vehicle at time t, t represents the time, and ε represents the noise; The unmanned vehicle observation model is shown in the following formula (7): In the formula, P(z t |x t ,m) represents the measurement probability, z t represents the sensor measurement at time t, m represents the map, and K represents the measurement at a time z t Number of internal measurements, z max Indicates the maximum measuring distance of the sensor. Mean variance The univariate normal distribution of represents the kth feature observed by the unmanned vehicle at the current position, σ hit represents the standard deviation of the measurement model noise, Indicated by x t and m are calculated by ray casting, represents the noise parameter inherent to the measurement model.

3. The unmanned vehicle rapid positioning method based on store sign recognition assistance according to claim 1 is characterized in that: The end-to-end OCR network based on the Transformer architecture includes an encoder and a decoder; The step S2 of inputting the environmental perception data into the constructed end-to-end OCR network based on the Transformer architecture to obtain the text recognition result includes: S21, acquiring image data in the environmental perception data, adjusting the size of the image data, decomposing the resized image into a plurality of image blocks, performing block embedding, position embedding, and image feature extraction on the plurality of image blocks, and obtaining an encoder output; S22. Input the encoder output into the decoder to obtain a text recognition result.

4. A device for quickly positioning an unmanned vehicle based on shop sign recognition assistance, the device for quickly positioning an unmanned vehicle based on shop sign recognition assistance is used to implement the method for quickly positioning an unmanned vehicle based on shop sign recognition assistance as claimed in any one of claims 1 to 3, characterized in that: The device comprises: An acquisition module is used to acquire environmental perception data of the unmanned vehicle to be located; A text recognition module, used to input the environmental perception data into the constructed end-to-end OCR network based on the Transformer architecture to obtain a text recognition result; An initial position determination module is used to perform similarity calculation on the text recognition result and the constructed feature database to obtain the position range of the unmanned vehicle; and determine the initial position of the unmanned vehicle according to the position range of the unmanned vehicle and the constructed three-dimensional point cloud environment map; The output module is used to obtain the precise positioning of the unmanned vehicle according to the initial position of the unmanned vehicle and the particle filtering algorithm.

5. An unmanned vehicle rapid positioning device, characterized in that: The unmanned vehicle rapid positioning device comprises: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 3 is implemented.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, which can be called by a processor to execute the method according to any one of claims 1 to 3.

Images