High-precision positioning system and method based on multi-sensor fusion

Abstract

The present application relates to the field of intelligent positioning, and specifically discloses a high-precision positioning system and method based on multi-sensor fusion, which improves the positioning accuracy of bucket wheel excavators by adopting a neural network model based on deep learning, integrating encoder positioning and Gray bus positioning, so as to more accurately control the operating posture of the bucket wheel excavator.

Description

Technical Field

[0001] The present application relates to the field of intelligent positioning, and more specifically, to a high-precision positioning system and method based on multi-sensor fusion. Background Art

[0002] To achieve unmanned operation of large-scale stacker-reclaimers, the first challenge is high-precision positioning. Different types of stacker-reclaimers require different positioning points. For example, a cantilever bucket wheel excavator requires positioning points for the machine's travel, boom pitch, and boom rotation. Only by accurately measuring the machine's travel, boom pitch, and boom rotation angles can the machine's operating posture be accurately controlled.

[0003] The mainstream positioning technologies currently on the market include encoder positioning and Gray bus positioning. Each positioning technology has its own characteristics, such as different positioning accuracy, coverage, data transmission methods, and prices.

[0004] Therefore, a high-precision positioning solution based on multi-sensor fusion is expected. Summary of the Invention

[0005] To address the above technical issues, the present application is proposed. The embodiments of the present application provide a high-precision positioning system and method based on multi-sensor fusion, which improves the positioning accuracy of bucket wheel excavators by adopting a neural network model based on deep learning, integrating encoder positioning and Gray bus positioning, and thus more accurately controlling the operating posture of the bucket wheel excavator.

[0006] According to one aspect of the present application, a high-precision positioning system based on multi-sensor fusion is provided, which includes:

[0007] A multi-sensor positioning data acquisition module is used to acquire a plurality of positioning coded digital signals collected by the positioning encoder at a plurality of time points within a predetermined time period and an absolute value Gray code provided by the Gray bus locator at the plurality of time points;

[0008] a first positioning data feature extraction module, configured to arrange the multiple positioning coded digital signals at the multiple time points into a positioning coding input vector according to the time dimension and then pass the vector through a first multi-scale neighborhood feature extraction module to obtain a multi-scale positioning coding feature vector;

[0009] A second positioning data feature extraction module is configured to arrange the absolute value Gray codes of the multiple time points according to the time dimension into an absolute value Gray code input vector and then pass it through a second multi-scale neighborhood feature extraction module to obtain a multi-scale Gray code feature vector;

[0010] A feature enhancement module is used to perform feature-level data enhancement on the multi-scale positioning coding feature vector and the multi-scale Gray code feature vector based on a Gaussian density map to obtain a positioning coding feature matrix and a Gray code feature matrix;

[0011] a contribution calculation module, configured to pass the positioning code feature matrix and the Gray code feature matrix through a pre-classifier to obtain a first probability value and a second probability value;

[0012] a feature fusion module, configured to fuse the multi-scale positioning coding feature vector and the multi-scale Gray code feature vector based on the first probability value and the second probability value to obtain a decoding feature vector; and

[0013] The positioning result generating module is used to decode and regress the decoded feature vector through a decoder to obtain a decoded value, and the decoded value is used to represent the rotation angle of the cantilever of the bucket wheel machine at the current time point.

[0014] In the above-mentioned high-precision positioning system based on multi-sensor fusion, the first multi-scale neighborhood feature extraction module includes: a first convolutional layer, a second convolutional layer parallel to the first convolutional layer, and a first multi-scale feature fusion layer connected to the first convolutional layer and the second convolutional layer, wherein the first convolutional layer uses a one-dimensional convolution kernel with a first length, and the second convolutional layer uses a one-dimensional convolution kernel with a second length.

[0015] In the above-mentioned high-precision positioning system based on multi-sensor fusion, the first positioning data feature extraction module includes: a first neighborhood-scale feature extraction unit, configured to input the positioning code input vector into the first convolution layer of the first multi-scale neighborhood feature extraction module to obtain a first neighborhood-scale positioning code feature vector, wherein the first convolution layer has a first one-dimensional convolution kernel of a first length; a second neighborhood-scale feature extraction unit, configured to input the positioning code input vector into the second convolution layer of the first multi-scale neighborhood feature extraction module to obtain a second neighborhood-scale positioning code feature vector, wherein the second convolution layer has a second one-dimensional convolution kernel of a second length, the first length being different from the second length; and a multi-scale feature fusion unit, configured to concatenate the first neighborhood-scale positioning code feature vector and the second neighborhood-scale positioning code feature vector to obtain the multi-scale positioning code feature vector. The first neighborhood-scale feature extraction unit is further configured to: use the first convolution layer of the first multi-scale neighborhood feature extraction module to perform one-dimensional convolution encoding on the positioning code input vector using the following formula to obtain a first neighborhood-scale positioning code feature vector; wherein the formula is:

[0016]

[0017] Wherein, a is the width of the first convolution kernel in the x direction, F(a) is the first convolution kernel parameter vector, G(xa) is the local vector matrix operated with the convolution kernel function, w is the size of the first convolution kernel, and X represents the positioning coding input vector; and the second neighborhood scale feature extraction unit is further used to: use the second convolution layer of the first multi-scale neighborhood feature extraction module to perform one-dimensional convolution coding on the positioning coding input vector using the following formula to obtain the second neighborhood scale positioning coding feature vector; wherein, the formula is:

[0018]

[0019] Wherein, b is the width of the second convolution kernel in the x direction, F(b) is the second convolution kernel parameter vector, G(xb) is the local vector matrix operated with the convolution kernel function, m is the size of the second convolution kernel, and X represents the positioning encoding input vector.

[0020] In the above-mentioned high-precision positioning system based on multi-sensor fusion, the second multi-scale neighborhood feature extraction module includes: a third convolutional layer, a fourth convolutional layer parallel to the third convolutional layer, and a second multi-scale feature fusion layer connected to the third convolutional layer and the fourth convolutional layer, wherein the third convolutional layer uses a one-dimensional convolution kernel with a third length, and the fourth convolutional layer uses a one-dimensional convolution kernel with a fourth length.

[0021] In the above-mentioned high-precision positioning system based on multi-sensor fusion, the feature enhancement module includes: a Gaussian density map construction unit, which is used to construct a Gaussian density map of the multi-scale positioning coding feature vector and the multi-scale Gray code feature vector to obtain a first Gaussian density map and a second Gaussian density map, wherein the mean vectors of the first Gaussian density map and the second Gaussian density map are respectively the multi-scale positioning coding feature vector and the multi-scale Gray code feature vector, and the covariance matrix of the first Gaussian density map and the second Gaussian density map are respectively the variance between the eigenvalues ​​of the corresponding two positions in the multi-scale positioning coding feature vector and the variance between the eigenvalues ​​of the corresponding two positions in the multi-scale Gray code feature vector; a Gaussian discretization unit, which is used to discretize the Gaussian distribution of each position in the first Gaussian density map and the second Gaussian density map to obtain a positioning coding feature matrix and a Gray code feature matrix.

[0022] In the above-mentioned high-precision positioning system based on multi-sensor fusion, the contribution calculation module is further used to: use the pre-classifier to pass the positioning code feature matrix and the Gray code feature matrix through the pre-classifier respectively to obtain a first probability value and a second probability value according to the following formula; wherein the formula is:

[0023] O=softmax{(Wn , B n ):...:(W1,B1)|Project(F)}, wherein Project(F) represents the projection of the positioning coding feature matrix and the Gray code feature matrix into a vector, W1 to W n is the weight matrix of each fully connected layer, B1 to B n Represents the bias vector of each fully connected layer.

[0024] In the above-mentioned high-precision positioning system based on multi-sensor fusion, the feature fusion module is further used to perform Hilbert space constraints of the vector modulus basis on the multi-scale positioning coding feature vector and the multi-scale Gray code feature vector to obtain the decoding feature vector, wherein the formula is:

[0025]

[0026] Where V1 and V2 represent the multi-scale positioning coding feature vector and the multi-scale Gray code feature vector respectively, V d represents the decoded feature vector, ||·||2 represents the two-norm of the vector, Cov 1D Represents a one-dimensional convolution operation, that is, the convolution operator (||V1||2, ||V2||2, V1V2 T ) for the eigenvector Perform one-dimensional convolution, represents a dot-add operation on the feature vector, and p1 and p2 are the first probability value and the second probability value.

[0027] In the above-mentioned high-precision positioning system based on multi-sensor fusion, the positioning result generation module is further used to: use the decoder to perform decoding regression on the decoded feature vector using the following formula to obtain a decoded value representing the rotation angle of the boom of the bucket wheel excavator at the current time point; wherein the formula is: Where X represents the decoded feature vector, Y is the decoded value, and W is the weight matrix. Represents matrix multiplication.

[0028] According to another aspect of the present application, a high-precision positioning method based on multi-sensor fusion is provided, which includes:

[0029] Acquire a plurality of positioning coded digital signals collected by the positioning encoder at a plurality of time points within a predetermined time period and an absolute value Gray code provided by the Gray bus locator at the plurality of time points;

[0030] Arranging the multiple positioning coding digital signals at the multiple time points according to the time dimension into a positioning coding input vector and then passing it through a first multi-scale neighborhood feature extraction module to obtain a multi-scale positioning coding feature vector;

[0031] Arranging the absolute value Gray codes of the multiple time points according to the time dimension into an absolute value Gray code input vector and then passing it through a second multi-scale neighborhood feature extraction module to obtain a multi-scale Gray code feature vector;

[0032] Based on the Gaussian density map, feature-level data enhancement is performed on the multi-scale positioning coding feature vector and the multi-scale Gray code feature vector to obtain a positioning coding feature matrix and a Gray code feature matrix;

[0033] Passing the positioning code feature matrix and the Gray code feature matrix through a pre-classifier to obtain a first probability value and a second probability value respectively;

[0034] Based on the first probability value and the second probability value, fusing the multi-scale positioning encoding feature vector and the multi-scale Gray code feature vector to obtain a decoding feature vector; and

[0035] The decoded feature vector is decoded and regressed by a decoder to obtain a decoded value, and the decoded value is used to represent the rotation angle of the cantilever of the bucket wheel machine at the current time point.

[0036] According to another aspect of the present application, an electronic device is provided, comprising: a processor; and a memory, wherein computer program instructions are stored in the memory, and when the computer program instructions are executed by the processor, the processor executes the high-precision positioning method based on multi-sensor fusion as described above.

[0037] According to another aspect of the present application, a computer-readable medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the high-precision positioning method based on multi-sensor fusion as described above.

[0038] Compared with the existing technology, the present application provides a high-precision positioning system and method based on multi-sensor fusion, which improves the positioning accuracy of the bucket wheel machine by adopting a neural network model based on deep learning, integrating encoder positioning and Gray bus positioning, so as to more accurately control the operating posture of the bucket wheel machine. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The above and other purposes, features, and advantages of the present application will become more apparent through a more detailed description of the embodiments of the present application in conjunction with the accompanying drawings. The accompanying drawings are intended to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the drawings, the same reference numerals generally represent the same components or steps.

[0040] Figure 1is a block diagram of a high-precision positioning system based on multi-sensor fusion according to an embodiment of the present application;

[0041] Figure 2 This is a system architecture diagram of a high-precision positioning system based on multi-sensor fusion according to an embodiment of the present application;

[0042] Figure 3 4 is a block diagram of a first positioning data feature extraction module in a high-precision positioning system based on multi-sensor fusion according to an embodiment of the present application;

[0043] Figure 4 1 is a block diagram of a feature enhancement module in a high-precision positioning system based on multi-sensor fusion according to an embodiment of the present application;

[0044] Figure 5 Flowchart of a high-precision positioning method based on multi-sensor fusion according to an embodiment of the present application;

[0045] Figure 6 is a block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0046] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described herein.

[0047] Application Overview

[0048] As mentioned in the previous background technology, the mainstream positioning technologies currently on the market include encoder positioning and Gray busbar positioning. Each positioning technology has its own characteristics, such as different positioning accuracy. Therefore, in order to improve the positioning accuracy of the bucket wheel excavator and to more accurately control the operating posture of the bucket wheel excavator, a high-precision positioning solution based on multi-sensor fusion is expected.

[0049] Specifically, the technical solution of this application acquires multiple positioning coded digital signals collected by a positioning encoder at multiple time points within a predetermined time period, along with the absolute value Gray codes provided by a Gray busbar positioner at these multiple time points. In other words, the technical solution of this application attempts to improve the positioning accuracy of a bucket wheel excavator by combining encoder positioning and Gray busbar positioning to more accurately control the operating posture of the bucket wheel excavator. Here, the positioning encoder and Gray busbar positioner are used to locate the rotation angle of the bucket wheel excavator's boom as an example.

[0050] Next, the multiple positioning coded digital signals at the multiple time points are arranged according to the time dimension into a positioning coding input vector, which is then passed through a first multi-scale neighborhood feature extraction module to obtain a multi-scale positioning coding feature vector. Specifically, at the data structure level, a time series vector of encoder positioning data is constructed to obtain the positioning coding input vector. The positioning coding input vector is then subjected to multi-scale one-dimensional convolutional coding using the first multi-scale neighborhood feature extraction module, which includes multiple parallel one-dimensional convolutional layers, to capture high-dimensional implicit correlation feature representations of correlations between encoder positioning data within different time spans in the positioning coding input vector.

[0051] In a specific example of the present application, the first multi-scale neighborhood feature extraction module includes a first convolution layer and a second convolution layer in parallel, and a first multi-scale feature fusion layer connected to the first convolution layer and the second convolution layer. In particular, during the encoding process of the first multi-scale neighborhood feature extraction module, the first convolution layer and the second convolution layer respectively use one-dimensional convolution kernels with different scales to perform multi-scale one-dimensional convolution encoding on the positioning coding input vector. Here, the one-dimensional convolution kernels of different scales correspond to the discrete distribution of encoder positioning data in different time spans in the positioning coding input vector. Therefore, after the one-dimensional convolution encoding by the first convolution layer and the second convolution layer, the high-dimensional implicit correlation feature representation of the correlation between the encoder positioning data in different time spans in the positioning coding input vector can be effectively extracted, and then the high-dimensional implicit correlation feature representation of the correlation between the encoder positioning data in different time spans in the positioning coding input vector is feature fused by the first multi-scale feature fusion layer to obtain the multi-scale positioning coding feature vector.

[0052] With respect to the absolute value Gray codes of the multiple time points collected by the Gray bus locator, in the technical solution of the present application, they are processed in the same manner, that is, the absolute value Gray codes of the multiple time points are arranged according to the time dimension as absolute value Gray code input vectors and then passed through a second multi-scale neighborhood feature extraction module to obtain a multi-scale Gray code feature vector. In particular, in the technical solution of the present application, the second multi-scale neighborhood feature extraction module has the same network structure as the first multi-scale neighborhood feature extraction module, but the second multi-scale neighborhood feature extraction module can use one-dimensional convolution kernels with different scales to improve the data adaptability of the encoding, thereby improving the accuracy of feature extraction.

[0053] In particular, in the technical solution of the present application, considering that the multi-scale positioning coding feature vector is obtained from the discrete distribution of the positioning coding digital signal, and the multi-scale Gray code feature vector is obtained from the discrete distribution of the absolute value Gray code, since the data of the discrete distribution of the positioning coding digital signal and the discrete distribution of the absolute value Gray code are relatively sparse (that is, the number of the multiple time points is relatively thin), therefore, in order to improve the richness and accuracy of feature expression, in the technical solution of the present application, feature-level data enhancement is performed on the multi-scale positioning coding feature vector and the multi-scale Gray code feature vector based on the Gaussian density map to obtain a positioning coding feature matrix and a Gray code feature matrix.

[0054] Those skilled in the art should know that Gaussian density maps are often used in the field of deep learning and are used for target learning distribution of deep learning. Therefore, in the technical solution of the present application, the Gaussian density map can be used as the expression learning target of the multi-scale positioning coding feature vector and the multi-scale Gray code feature vector to perform feature-level data enhancement on the multi-scale positioning coding feature vector and the multi-scale Gray code feature vector to obtain the positioning coding feature matrix and the Gray code feature matrix.

[0055] Specifically, a Gaussian density map of the multi-scale positioning coding feature vector and the multi-scale Gray code feature vector is first constructed to obtain a first Gaussian density map and a second Gaussian density map, wherein the mean vector of the first Gaussian density map is the multi-scale positioning coding feature vector, and the value of each position in the covariance matrix of the first Gaussian density map is the variance between the eigenvalues ​​of the corresponding two positions in the multi-scale positioning coding feature vector; the mean vector of the second Gaussian density map is the multi-scale Gray code feature vector, and the eigenvalue of each position in the covariance matrix of the second Gaussian density map is the variance between the eigenvalues ​​of the corresponding two positions in the multi-scale Gray code feature vector. Then, the Gaussian distribution of each position of the first Gaussian density map and the second Gaussian density map are Gaussian discretized to obtain the positioning coding feature matrix and the Gray code feature matrix.

[0056] In particular, in the technical solution of the present application, when performing precise positioning through multiple sensors, the key is to determine the contribution of the positioning data of each sensor to the final positioning data. Here, the pre-classification idea is used to determine the adaptive weight based on the distribution characteristics of the data itself. Specifically, the positioning coding feature matrix and the Gray code feature matrix are respectively passed through a pre-classifier to obtain a first probability value and a second probability value. Then, based on the first probability value and the second probability value, the multi-scale positioning coding feature vector and the multi-scale Gray code feature vector are fused to obtain a decoding feature vector. For example, in a specific example of the present application, the first probability value and the second probability value are used as weights to calculate the position-weighted sum between the multi-scale positioning coding feature vector and the multi-scale Gray code feature vector to obtain the decoding feature vector. In this way, when performing feature fusion, the contribution of the positioning data of each sensor to the final positioning data is effectively integrated at the data layer to improve the accuracy of decoding and regressing the decoding feature vector through the decoder, that is, to improve the positioning accuracy.

[0057] In particular, in the technical solution of the present application, when the position-weighted sum of the multi-scale positioning coding feature vector and the multi-scale Gray code feature vector is calculated using the first probability value and the second probability value as weights to obtain the decoded feature vector, although the multi-scale positioning coding feature vector and the multi-scale Gray code feature vector are time-series multi-scale neighborhood correlation features of the positioning coding digital signal and the absolute value Gray code, respectively, in the case of heterogeneous source data, there is still inconsistency in the feature distribution between them, resulting in poor convergence of the overall feature distribution of the decoded feature vector obtained after weighted fusion, that is, poor fitting effect of the decoder. On the other hand, when the first probability value and the second probability value are set as weights for explicit association, the correlation between the eigenvalues ​​of the decoded feature vector may increase, which will also reduce the accuracy of the decoding result of the decoded feature vector.

[0058] Therefore, it is preferred to first perform the Hilbert space constraint of the vector modulus basis on the multi-scale positioning encoding feature vector, for example, V1, and the multi-scale Gray code feature vector, for example, V2, to obtain the decoding feature vector V d , expressed as:

[0059]

[0060] Cov 1D Represents a one-dimensional convolution operation, that is, the convolution operator (||V1||2, ||V2||2, V1V2 T ) for the eigenvector Perform one-dimensional convolution, represents a dot-add operation on the feature vector, and p1 and p2 are the first probability value and the second probability value.

[0061] Here, the multi-scale positioning coding feature vector V1 and the multi-scale Gray code feature vector V2 to be fused are constrained by the convolution operator in the Hilbert space that defines the inner product of the vector sum modulus and the vector, so that the fused decoding feature vector V d The feature distribution of is limited to a finite closed domain in the Hilbert space based on the modulus of the vector, and improves the decoding feature vector V d The orthogonality between the basis dimensions of the high-dimensional manifold of the feature distribution is achieved, thereby achieving sparse correlation between eigenvalues ​​while maintaining the overall convergence of the feature distribution. In this way, the decoding feature vector V d The fitting effect of the decoder and the accuracy of the decoding results.

[0062] Based on this, the present application proposes a high-precision positioning system based on multi-sensor fusion, which includes: a multi-sensor positioning data acquisition module, which is used to obtain multiple positioning coded digital signals collected by the positioning encoder at multiple time points within a predetermined time period and the absolute value Gray code provided by the Gray bus locator at the multiple time points; a first positioning data feature extraction module, which is used to arrange the multiple positioning coded digital signals at the multiple time points into a positioning code input vector according to the time dimension and then pass it through a first multi-scale neighborhood feature extraction module to obtain a multi-scale positioning code feature vector; a second positioning data feature extraction module, which is used to arrange the absolute value Gray code of the multiple time points into an absolute value Gray code input vector according to the time dimension and then pass it through a second multi-scale neighborhood feature extraction module to obtain a multi-scale Gray code A feature vector; a feature enhancement module for performing feature-level data enhancement on the multi-scale positioning coding feature vector and the multi-scale Gray code feature vector based on a Gaussian density map to obtain a positioning coding feature matrix and a Gray code feature matrix; a contribution calculation module for passing the positioning coding feature matrix and the Gray code feature matrix through a pre-classifier to obtain a first probability value and a second probability value; a feature fusion module for fusing the multi-scale positioning coding feature vector and the multi-scale Gray code feature vector based on the first probability value and the second probability value to obtain a decoded feature vector; and a positioning result generation module for decoding and regressing the decoded feature vector through a decoder to obtain a decoded value, wherein the decoded value is used to represent the rotation angle of the cantilever of the bucket wheel excavator at the current time point.

[0063] After introducing the basic principles of the present application, various non-limiting embodiments of the present application will be described in detail with reference to the accompanying drawings.

[0064] Exemplary Systems

[0065] Figure 1 FIG is a block diagram of a high-precision positioning system based on multi-sensor fusion according to an embodiment of the present application. Figure 1 As shown, the high-precision positioning system 300 based on multi-sensor fusion according to an embodiment of the present application includes: a multi-sensor positioning data acquisition module 310; a first positioning data feature extraction module 320; a second positioning data feature extraction module 330; a feature enhancement module 340; a contribution calculation module 350; a feature fusion module 360; and a positioning result generation module 370.

[0066] Among them, the multi-sensor positioning data acquisition module 310 is used to obtain multiple positioning coded digital signals collected by the positioning encoder at multiple time points within a predetermined time period and the absolute value Gray code provided by the Gray bus locator at the multiple time points; the first positioning data feature extraction module 320 is used to arrange the multiple positioning coded digital signals at the multiple time points into a positioning code input vector according to the time dimension and then pass it through the first multi-scale neighborhood feature extraction module to obtain a multi-scale positioning code feature vector; the second positioning data feature extraction module 330 is used to arrange the absolute value Gray code of the multiple time points into an absolute value Gray code input vector according to the time dimension and then pass it through the second multi-scale neighborhood feature extraction module to obtain a multi-scale Gray code feature vector; the feature enhancement module 340, The module 350 is used to perform feature-level data enhancement on the multi-scale positioning coding feature vector and the multi-scale Gray code feature vector based on the Gaussian density map to obtain a positioning coding feature matrix and a Gray code feature matrix; the contribution calculation module 350 is used to pass the positioning coding feature matrix and the Gray code feature matrix through a pre-classifier to obtain a first probability value and a second probability value; the feature fusion module 360 ​​is used to fuse the multi-scale positioning coding feature vector and the multi-scale Gray code feature vector based on the first probability value and the second probability value to obtain a decoded feature vector; and the positioning result generation module 370 is used to decode and regress the decoded feature vector through a decoder to obtain a decoded value, and the decoded value is used to represent the rotation angle of the boom of the bucket wheel excavator at the current time point.

[0067] Figure 2 FIG is a system architecture diagram of a high-precision positioning system based on multi-sensor fusion according to an embodiment of the present application. Figure 2As shown, in this network architecture, first, the multi-sensor positioning data acquisition module 310 acquires a plurality of positioning coded digital signals acquired by the positioning encoder at a plurality of time points within a predetermined time period and the absolute value Gray code provided by the Gray bus locator at the plurality of time points; then, the first positioning data feature extraction module 320 arranges the plurality of positioning coded digital signals acquired by the multi-sensor positioning data acquisition module 310 at a plurality of time points into a positioning code input vector according to the time dimension and then passes through the first multi-scale neighborhood feature extraction module to obtain a multi-scale positioning code feature vector; the second positioning data feature extraction module 330 arranges the absolute value Gray code of the plurality of time points acquired by the multi-sensor positioning data acquisition module 310 into an absolute value Gray code input vector according to the time dimension and then passes through the second multi-scale neighborhood feature extraction module to obtain a multi-scale Gray code feature vector; then, the feature enhancement module 3 40 Based on the Gaussian density map, feature-level data enhancement is performed on the multi-scale positioning coding feature vector obtained by the first positioning data feature extraction module 320 and the multi-scale Gray code feature vector obtained by the second positioning data feature extraction module 330 to obtain a positioning coding feature matrix and a Gray code feature matrix; the contribution calculation module 350 passes the positioning coding feature matrix and the Gray code feature matrix obtained by the feature enhancement module 340 through a pre-classifier to obtain a first probability value and a second probability value; the feature fusion module 360 ​​fuses the multi-scale positioning coding feature vector and the multi-scale Gray code feature vector based on the first probability value and the second probability value to obtain a decoding feature vector; further, the positioning result generation module 370 decodes and regresses the decoded feature vector through a decoder to obtain a decoding value, and the decoding value is used to represent the rotation angle of the cantilever of the bucket wheel excavator at the current time point.

[0068] Specifically, during the operation of the high-precision positioning system 300 based on multi-sensor fusion, the multi-sensor positioning data acquisition module 310 is used to obtain a plurality of positioning coded digital signals collected by the positioning encoder at a plurality of time points within a predetermined time period and the absolute value Gray code provided by the Gray bus locator at the plurality of time points. Specifically, in the technical solution of the present application, a plurality of positioning coded digital signals collected by the positioning encoder at a plurality of time points within a predetermined time period and the absolute value Gray code provided by the Gray bus locator at the plurality of time points are obtained. That is, in the technical solution of the present application, an attempt is made to improve the positioning accuracy of the bucket wheel machine by combining encoder positioning and Gray bus positioning, so as to more accurately control the operating posture of the bucket wheel machine. Here, the positioning encoder and the Gray bus locator are used to position the rotation angle of the boom of the bucket wheel machine as an example.

[0069] Specifically, during the operation of the high-precision positioning system 300 based on multi-sensor fusion, the first positioning data feature extraction module 320 is configured to arrange the multiple positioning coded digital signals at the multiple time points according to the time dimension into a positioning coding input vector, and then pass the result through the first multi-scale neighborhood feature extraction module to obtain a multi-scale positioning coding feature vector. That is, after the multiple positioning coded digital signals at the multiple time points are arranged according to the time dimension into a positioning coding input vector, the result is passed through the first multi-scale neighborhood feature extraction module to obtain a multi-scale positioning coding feature vector. That is, at the data structure level, a time series vector of the encoder positioning data is constructed to obtain the positioning coding input vector, and the first multi-scale neighborhood feature extraction module, which includes multiple parallel one-dimensional convolutional layers, performs multi-scale one-dimensional convolutional coding on the positioning coding input vector to capture the high-dimensional implicit correlation feature representation of the correlation between the encoder positioning data within different time spans in the positioning coding input vector. More specifically, in a specific example of the present application, the first multi-scale neighborhood feature extraction module includes a parallel first convolutional layer and a second convolutional layer, as well as a first multi-scale feature fusion layer connected to the first convolutional layer and the second convolutional layer. In particular, in the encoding process of the first multi-scale neighborhood feature extraction module, the first convolution layer and the second convolution layer respectively use one-dimensional convolution kernels with different scales to perform multi-scale one-dimensional convolution encoding on the positioning coding input vector. Here, the one-dimensional convolution kernels of different scales correspond to the discrete distribution of encoder positioning data in different time spans in the positioning coding input vector. Therefore, after the one-dimensional convolution encoding by the first convolution layer and the second convolution layer, the high-dimensional implicit correlation feature representation of the correlation between the encoder positioning data in different time spans in the positioning coding input vector can be effectively extracted, and then the high-dimensional implicit correlation feature representation of the correlation between the encoder positioning data in different time spans in the positioning coding input vector is feature fused through the first multi-scale feature fusion layer to obtain the multi-scale positioning coding feature vector.

[0070] Figure 3 FIG. 1 is a block diagram of a first positioning data feature extraction module in a high-precision positioning system based on multi-sensor fusion according to an embodiment of the present application. Figure 3As shown, the first positioning data feature extraction module 320 includes: a first neighborhood scale feature extraction unit 321, which is used to input the positioning code input vector into the first convolution layer of the first multi-scale neighborhood feature extraction module to obtain a first neighborhood scale positioning code feature vector, wherein the first convolution layer has a first one-dimensional convolution kernel of a first length; a second neighborhood scale feature extraction unit 322, which is used to input the positioning code input vector into the second convolution layer of the first multi-scale neighborhood feature extraction module to obtain a second neighborhood scale positioning code feature vector, wherein the second convolution layer has a second one-dimensional convolution kernel of a second length, and the first length is different from the second length; and a multi-scale feature fusion unit 323, which is used to cascade the first neighborhood scale positioning code feature vector and the second neighborhood scale positioning code feature vector to obtain the multi-scale positioning code feature vector. The first neighborhood scale feature extraction unit is further used to: use the first convolution layer of the first multi-scale neighborhood feature extraction module to perform one-dimensional convolution encoding on the positioning code input vector according to the following formula to obtain the first neighborhood scale positioning code feature vector; wherein the formula is:

[0071]

[0072] Wherein, a is the width of the first convolution kernel in the x direction, F(a) is the first convolution kernel parameter vector, G(xa) is the local vector matrix operated with the convolution kernel function, w is the size of the first convolution kernel, and X represents the positioning coding input vector; and the second neighborhood scale feature extraction unit is further used to: use the second convolution layer of the first multi-scale neighborhood feature extraction module to perform one-dimensional convolution coding on the positioning coding input vector using the following formula to obtain the second neighborhood scale positioning coding feature vector; wherein, the formula is:

[0073]

[0074] Wherein, b is the width of the second convolution kernel in the x direction, F(b) is the second convolution kernel parameter vector, G(xb) is the local vector matrix operated with the convolution kernel function, m is the size of the second convolution kernel, and X represents the positioning coding input vector; more specifically, the cascading of the first neighborhood scale positioning coding feature vector and the second neighborhood scale positioning coding feature vector to obtain the multi-scale positioning coding feature vector includes: cascading the first neighborhood scale positioning coding feature vector and the second neighborhood scale positioning coding feature vector according to the following formula to obtain the multi-scale positioning coding feature vector; wherein, the formula is:

[0075] V1=Concat[V a , V b ]

[0076] Among them, V a represents the first neighborhood scale positioning encoding feature vector, V b represents the second neighborhood scale positioning coding feature vector, Concat[·,·] represents the cascade function, and V1 represents the multi-scale positioning coding feature vector.

[0077] Specifically, during the operation of the high-precision positioning system 300 based on multi-sensor fusion, the second positioning data feature extraction module 330 is used to arrange the absolute value Gray codes of the multiple time points into an absolute value Gray code input vector according to the time dimension and then pass it through the second multi-scale neighborhood feature extraction module to obtain a multi-scale Gray code feature vector. That is, for the absolute value Gray codes of the multiple time points collected by the Gray bus locator, in the technical solution of the present application, they are processed in the same way, that is, the absolute value Gray codes of the multiple time points are arranged into an absolute value Gray code input vector according to the time dimension and then pass it through the second multi-scale neighborhood feature extraction module to obtain a multi-scale Gray code feature vector. In particular, in the technical solution of the present application, the second multi-scale neighborhood feature extraction module has the same network structure as the first multi-scale neighborhood feature extraction module, but the second multi-scale neighborhood feature extraction module can use one-dimensional convolution kernels with different scales to improve the data adaptability of the encoding, thereby improving the accuracy of feature extraction. More specifically, the second multi-scale neighborhood feature extraction module includes: a third convolutional layer, a fourth convolutional layer parallel to the third convolutional layer, and a second multi-scale feature fusion layer connected to the third convolutional layer and the fourth convolutional layer, wherein the third convolutional layer uses a one-dimensional convolution kernel with a third length, and the fourth convolutional layer uses a one-dimensional convolution kernel with a fourth length.

[0078] Specifically, during the operation of the high-precision positioning system 300 based on multi-sensor fusion, the feature enhancement module 340 is used to perform feature-level data enhancement on the multi-scale positioning coding feature vector and the multi-scale Gray code feature vector based on the Gaussian density map to obtain a positioning coding feature matrix and a Gray code feature matrix. In the technical solution of the present application, considering that the multi-scale positioning coding feature vector is obtained from the discrete distribution of the positioning coding digital signal, and the multi-scale Gray code feature vector is obtained from the discrete distribution of the absolute value Gray code, because the data of the discrete distribution of the positioning coding digital signal and the discrete distribution of the absolute value Gray code are relatively sparse (that is, the number of the multiple time points is relatively thin), therefore, in order to improve the richness and accuracy of feature expression, in the technical solution of the present application, based on the Gaussian density map, the multi-scale positioning coding feature vector and the multi-scale Gray code feature vector are respectively subjected to feature-level data enhancement to obtain a positioning coding feature matrix and a Gray code feature matrix. It should be understood that Gaussian density maps are often used in the field of deep learning and are used for target learning distribution of deep learning. Therefore, in the technical solution of the present application, the Gaussian density map can be used as the expression learning target of the multi-scale positioning coding feature vector and the multi-scale Gray code feature vector to perform feature-level data enhancement on the multi-scale positioning coding feature vector and the multi-scale Gray code feature vector to obtain the positioning coding feature matrix and the Gray code feature matrix. Specifically, a Gaussian density map of the multi-scale positioning coding feature vector and the multi-scale Gray code feature vector is first constructed to obtain a first Gaussian density map and a second Gaussian density map, wherein the mean vector of the first Gaussian density map is the multi-scale positioning coding feature vector, the value of each position in the covariance matrix of the first Gaussian density map is the variance between the eigenvalues ​​of the corresponding two positions in the multi-scale positioning coding feature vector, the mean vector of the second Gaussian density map is the multi-scale Gray code feature vector, and the eigenvalue of each position in the covariance matrix of the second Gaussian density map is the variance between the eigenvalues ​​of the corresponding two positions in the multi-scale Gray code feature vector. Next, Gaussian discretization is performed on the Gaussian distributions of each position of the first Gaussian density map and the second Gaussian density map to obtain the positioning code feature matrix and the Gray code feature matrix.

[0079] Figure 4 FIG is a block diagram of a feature enhancement module in a high-precision positioning system based on multi-sensor fusion according to an embodiment of the present application. Figure 4As shown, the feature enhancement module 340 includes: a Gaussian density map construction unit 341, which is used to construct a Gaussian density map of the multi-scale positioning coding feature vector and the multi-scale Gray code feature vector to obtain a first Gaussian density map and a second Gaussian density map, wherein the mean vectors of the first Gaussian density map and the second Gaussian density map are respectively the multi-scale positioning coding feature vector and the multi-scale Gray code feature vector, and the covariance matrix of the first Gaussian density map and the second Gaussian density map are respectively the variance between the eigenvalues ​​of the corresponding two positions in the multi-scale positioning coding feature vector and the variance between the eigenvalues ​​of the corresponding two positions in the multi-scale Gray code feature vector; a Gaussian discretization unit 342, which is used to discretize the Gaussian distribution of each position in the first Gaussian density map and the second Gaussian density map to obtain a positioning coding feature matrix and a Gray code feature matrix.

[0080] Specifically, during the operation of the high-precision positioning system 300 based on multi-sensor fusion, the contribution calculation module 350 is used to pass the positioning coding feature matrix and the Gray code feature matrix through a pre-classifier to obtain a first probability value and a second probability value. It should be understood that when performing precise positioning through multiple sensors, the key is to determine the contribution of the positioning data of each sensor to the final positioning data. Here, the adaptive weight is determined based on the distribution characteristics of the data itself through the idea of ​​pre-classification. Specifically, the positioning coding feature matrix and the Gray code feature matrix are passed through a pre-classifier to obtain a first probability value and a second probability value. In a specific example of the present application, the passing of the positioning coding feature matrix and the Gray code feature matrix through a pre-classifier to obtain a first probability value and a second probability value includes: using the pre-classifier to pass the positioning coding feature matrix and the Gray code feature matrix through a pre-classifier to obtain a first probability value and a second probability value according to the following formula; wherein, the formula is: O=softmax{(W n ,B n ):…:(W1,B1)|Project(F)}, where Project(F) represents the projection of the positioning coding feature matrix and the Gray code feature matrix into vectors, W1 to W n is the weight matrix of each fully connected layer, B1 to B n Represents the bias vector of each fully connected layer.

[0081] Specifically, during the operation of the high-precision positioning system 300 based on multi-sensor fusion, the feature fusion module 360 ​​is used to fuse the multi-scale positioning coding feature vector and the multi-scale Gray code feature vector based on the first probability value and the second probability value to obtain a decoding feature vector. That is, based on the first probability value and the second probability value, the multi-scale positioning coding feature vector and the multi-scale Gray code feature vector are fused to obtain a decoding feature vector. For example, in a specific example of the present application, the first probability value and the second probability value are used as weights to calculate the position-weighted sum between the multi-scale positioning coding feature vector and the multi-scale Gray code feature vector to obtain the decoding feature vector. In this way, when performing feature fusion, the contribution of the positioning data of each sensor to the final positioning data is effectively integrated at the data layer to improve the accuracy of decoding and regressing the decoding feature vector through the decoder, that is, to improve the positioning accuracy. In particular, in the technical solution of the present application, when the position-weighted sum of the multi-scale positioning coding feature vector and the multi-scale Gray code feature vector is calculated using the first probability value and the second probability value as weights to obtain the decoding feature vector, although the multi-scale positioning coding feature vector and the multi-scale Gray code feature vector are respectively the time-series multi-scale neighborhood correlation features of the positioning coding digital signal and the absolute value Gray code, in the case of heterogeneous source data, there is still inconsistency in the feature distribution between them, resulting in poor convergence of the overall feature distribution of the decoding feature vector obtained after weighted fusion, that is, the fitting effect of the decoder is poor. On the other hand, when the first probability value and the second probability value are set as weights for explicit association, the correlation between the eigenvalues ​​of the decoding feature vector may increase, which will also reduce the accuracy of the decoding result of the decoding feature vector. Therefore, it is preferred to first perform the Hilbert space constraint of the vector modulus basis on the multi-scale positioning coding feature vector, for example, denoted as V1, and the multi-scale Gray code feature vector, for example, denoted as V2 to obtain the decoding feature vector V d , expressed as:

[0082]

[0083] Where V1 and V2 represent the multi-scale positioning coding feature vector and the multi-scale Gray code feature vector respectively, V d represents the decoded feature vector, ‖·‖2 represents the two-norm of the vector, Cov 1D Represents a one-dimensional convolution operation, that is, the convolution operator (‖V1‖2,‖V2‖2,V1V2 T ) for the eigenvector Perform one-dimensional convolution, Here, the multi-scale positioning coding feature vector V1 and the multi-scale Gray code feature vector V2 to be fused are constrained by the convolution operator in the Hilbert space that defines the vector sum modulus and vector inner product, so that the fused decoded feature vector V d The feature distribution of is limited to a finite closed domain in the Hilbert space based on the modulus of the vector, and improves the decoding feature vector V d The orthogonality between the basis dimensions of the high-dimensional manifold of the feature distribution is achieved, thereby achieving sparse correlation between eigenvalues ​​while maintaining the overall convergence of the feature distribution. In this way, the decoding feature vector V d The fitting effect of the decoder and the accuracy of the decoding results.

[0084] Specifically, during the operation of the high-precision positioning system 300 based on multi-sensor fusion, the positioning result generation module 370 is used to decode and regress the decoded feature vector through a decoder to obtain a decoded value, and the decoded value is used to represent the rotation angle of the boom of the bucket wheel excavator at the current time point. That is, the first probability value and the second probability value are used as weights to calculate the position-weighted sum between the multi-scale positioning coding feature vector and the multi-scale Gray code feature vector to obtain the decoded feature vector. In a specific example of the present application, the decoder is used to decode and regress the decoded feature vector using the following formula to obtain a decoded value for representing the rotation angle of the boom of the bucket wheel excavator at the current time point; wherein, the formula is: Where X represents the decoded feature vector, Y is the decoded value, and W is the weight matrix. Represents matrix multiplication.

[0085] In summary, a high-precision positioning system 300 based on multi-sensor fusion according to an embodiment of the present application is explained, which improves the positioning accuracy of the bucket wheel machine by adopting a neural network model based on deep learning, integrating encoder positioning and Gray bus positioning, so as to more accurately control the operating posture of the bucket wheel machine.

[0086] As described above, the high-precision positioning system based on multi-sensor fusion according to the embodiment of the present application can be implemented in various terminal devices. In one example, the high-precision positioning system 300 based on multi-sensor fusion according to the embodiment of the present application can be integrated into the terminal device as a software module and / or hardware module. For example, the high-precision positioning system 300 based on multi-sensor fusion can be a software module in the operating system of the terminal device, or can be an application developed for the terminal device; of course, the high-precision positioning system 300 based on multi-sensor fusion can also be one of the many hardware modules of the terminal device.

[0087] Alternatively, in another example, the high-precision positioning system 300 based on multi-sensor fusion and the terminal device may also be separate devices, and the high-precision positioning system 300 based on multi-sensor fusion may be connected to the terminal device through a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

[0088] Exemplary Methods

[0089] Figure 5 Flowchart of a high-precision positioning method based on multi-sensor fusion according to an embodiment of the present application. Figure 5 As shown, according to the embodiment of the present application, the high-precision positioning method based on multi-sensor fusion includes the following steps: S110, obtaining a plurality of positioning coded digital signals collected by a positioning encoder at a plurality of time points within a predetermined time period and an absolute value Gray code provided by a Gray bus locator at the plurality of time points; S120, arranging the plurality of positioning coded digital signals at the plurality of time points into a positioning code input vector according to the time dimension and then passing it through a first multi-scale neighborhood feature extraction module to obtain a multi-scale positioning code feature vector; S130, arranging the absolute value Gray codes at the plurality of time points into an absolute value Gray code input vector according to the time dimension and then passing it through a second multi-scale neighborhood feature extraction module to obtain a multi-scale Gray code feature vector ; S140, based on the Gaussian density map, feature-level data enhancement is performed on the multi-scale positioning coding feature vector and the multi-scale Gray code feature vector to obtain a positioning coding feature matrix and a Gray code feature matrix; S150, the positioning coding feature matrix and the Gray code feature matrix are respectively passed through a pre-classifier to obtain a first probability value and a second probability value; S160, based on the first probability value and the second probability value, the multi-scale positioning coding feature vector and the multi-scale Gray code feature vector are fused to obtain a decoding feature vector; and, S170, the decoded feature vector is decoded and regressed through a decoder to obtain a decoding value, and the decoding value is used to represent the rotation angle of the cantilever of the bucket wheel excavator at the current time point.

[0090] In one example, in the high-precision positioning method based on multi-sensor fusion, step S120 includes: inputting the positioning code input vector into the first convolution layer of the first multi-scale neighborhood feature extraction module to obtain a first neighborhood-scale positioning code feature vector, wherein the first convolution layer has a first one-dimensional convolution kernel of a first length; inputting the positioning code input vector into the second convolution layer of the first multi-scale neighborhood feature extraction module to obtain a second neighborhood-scale positioning code feature vector, wherein the second convolution layer has a second one-dimensional convolution kernel of a second length, the first length being different from the second length; and concatenating the first neighborhood-scale positioning code feature vector and the second neighborhood-scale positioning code feature vector to obtain the multi-scale positioning code feature vector. The first multi-scale neighborhood feature extraction module includes: a first convolution layer, a second convolution layer parallel to the first convolution layer, and a first multi-scale feature fusion layer connected to the first and second convolution layers, wherein the first convolution layer uses a one-dimensional convolution kernel of a first length, and the second convolution layer uses a one-dimensional convolution kernel of a second length. More specifically, the first neighborhood-scale feature extraction unit is further configured to: use the first convolution layer of the first multi-scale neighborhood feature extraction module to perform one-dimensional convolution encoding on the positioning encoding input vector using the following formula to obtain a first neighborhood-scale positioning encoding feature vector; wherein the formula is:

[0091]

[0092] Wherein, a is the width of the first convolution kernel in the x direction, F(a) is the first convolution kernel parameter vector, G(xa) is the local vector matrix operated with the convolution kernel function, w is the size of the first convolution kernel, and X represents the positioning coding input vector; and the second neighborhood scale feature extraction unit is further used to: use the second convolution layer of the first multi-scale neighborhood feature extraction module to perform one-dimensional convolution coding on the positioning coding input vector using the following formula to obtain the second neighborhood scale positioning coding feature vector; wherein, the formula is:

[0093]

[0094] Wherein, b is the width of the second convolution kernel in the x direction, F(b) is the second convolution kernel parameter vector, G(xb) is the local vector matrix operated with the convolution kernel function, m is the size of the second convolution kernel, and X represents the positioning encoding input vector.

[0095] In one example, in the above-mentioned high-precision positioning method based on multi-sensor fusion, the step S130 includes: the second multi-scale neighborhood feature extraction module, including: a third convolution layer, a fourth convolution layer parallel to the third convolution layer, and a second multi-scale feature fusion layer connected to the third convolution layer and the fourth convolution layer, wherein the third convolution layer uses a one-dimensional convolution kernel with a third length, and the fourth convolution layer uses a one-dimensional convolution kernel with a fourth length.

[0096] In one example, in the above-mentioned high-precision positioning method based on multi-sensor fusion, the step S140 includes: constructing a Gaussian density map of the multi-scale positioning coding feature vector and the multi-scale Gray code feature vector to obtain a first Gaussian density map and a second Gaussian density map, wherein the mean vectors of the first Gaussian density map and the second Gaussian density map are respectively the multi-scale positioning coding feature vector and the multi-scale Gray code feature vector, and the covariance matrix of the first Gaussian density map and the second Gaussian density map are respectively the variance between the eigenvalues ​​of the corresponding two positions in the multi-scale positioning coding feature vector and the variance between the eigenvalues ​​of the corresponding two positions in the multi-scale Gray code feature vector; discretizing the Gaussian distribution of each position in the first Gaussian density map and the second Gaussian density map to obtain a positioning coding feature matrix and a Gray code feature matrix.

[0097] In one example, in the above-mentioned high-precision positioning method based on multi-sensor fusion, step S150 includes: using the pre-classifier to pass the positioning code feature matrix and the Gray code feature matrix through the pre-classifier respectively to obtain a first probability value and a second probability value according to the following formula; wherein the formula is:

[0098] O=softmax{(W b ,B n ):…:(W1,B1)|Project(F)}, where Project(F) means

[0099] Project the positioning coding feature matrix and the Gray code feature matrix into vectors, W1 to W n is the weight matrix of each fully connected layer, B1 to B n Represents the bias vector of each fully connected layer.

[0100] In one example, in the above-mentioned high-precision positioning method based on multi-sensor fusion, step S160 includes: performing a Hilbert space constraint of a vector modulus basis on the multi-scale positioning coding feature vector and the multi-scale Gray code feature vector using the following formula to obtain the decoding feature vector, wherein the formula is:

[0101]

[0102] Where V1 and V2 represent the multi-scale positioning coding feature vector and the multi-scale Gray code feature vector respectively, V d represents the decoded feature vector, ‖·‖2 represents the two-norm of the vector, Cov 1D Represents a one-dimensional convolution operation, that is, the convolution operator (‖V1‖2,‖V2‖2,V1V2 T ) for the eigenvector Perform one-dimensional convolution, represents a dot-add operation on the feature vector, and p1 and p2 are the first probability value and the second probability value.

[0103] In one example, in the high-precision positioning method based on multi-sensor fusion, step S170 includes: using the decoder to perform decoding regression on the decoded feature vector using the following formula to obtain a decoded value representing the rotation angle of the boom of the bucket wheel machine at the current time point; wherein the formula is: Where X represents the decoded feature vector, Y is the decoded value, and W is the weight matrix. Represents matrix multiplication.

[0104] In summary, according to the embodiment of the present application, a high-precision positioning method based on multi-sensor fusion is explained, which improves the positioning accuracy of the bucket wheel machine by adopting a neural network model based on deep learning, integrating encoder positioning and Gray bus positioning, so as to more accurately control the operating posture of the bucket wheel machine.

[0105] Exemplary electronic devices

[0106] Below, reference Figure 6 To describe the electronic device according to the embodiment of the present application.

[0107] Figure 6 The figure shows a block diagram of an electronic device according to an embodiment of the present application.

[0108] like Figure 6 As shown, the electronic device 10 includes one or more processors 11 and a memory 12 .

[0109] The processor 11 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

[0110] The memory 12 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. The non-volatile memory may include, for example, read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may execute the program instructions to implement the functions of the high-precision positioning system based on multi-sensor fusion of the various embodiments of the present application described above and / or other desired functions. Various contents such as decoded feature vectors may also be stored in the computer-readable storage medium.

[0111] In one example, the electronic device 10 may further include an input device 13 and an output device 14 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0112] The input device 13 may include, for example, a keyboard, a mouse, and the like.

[0113] The output device 14 can output various information to the outside, including decoded values, etc. The output device 14 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0114] Of course, to simplify, Figure 6 Only some of the components related to the present application in the electronic device 10 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, the electronic device 10 may further include any other appropriate components according to specific application scenarios.

[0115] Exemplary computer program products and computer-readable storage media

[0116] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to perform the steps in the functions of the high-precision positioning method based on multi-sensor fusion according to various embodiments of the present application described in the above "Exemplary System" section of this specification.

[0117] The computer program product may be written in any combination of one or more programming languages ​​to implement the program code for performing the operations of the embodiments of the present application, including object-oriented programming languages ​​such as Java, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0118] In addition, an embodiment of the present application may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enable the processor to perform the steps in the functions of the high-precision positioning method based on multi-sensor fusion according to various embodiments of the present application described in the above "Exemplary System" section of this specification.

[0119] The computer-readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can, for example, include but is not limited to a system, device or component of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0120] The basic principles of the present application have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, and effects mentioned in this application are merely illustrative and not restrictive, and it should not be assumed that these advantages, strengths, and effects are required of each embodiment of this application. In addition, the specific details disclosed above are merely illustrative and facilitating understanding, and are not restrictive. The above details do not limit this application to necessarily being implemented using the above specific details.

[0121] The block diagrams of the devices, devices, equipment, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, devices, equipment, and systems can be connected, arranged, or configured in any manner. Words such as "include," "comprise," "have," and the like are open-ended words, meaning "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or" and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0122] It should also be noted that in the apparatus, device, and method of the present application, each component or each step can be decomposed and / or recombined, and such decomposition and / or recombination should be regarded as equivalent solutions of the present application.

[0123] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0124] The above description has been provided for the purpose of illustration and description. Furthermore, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although a number of example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A high-precision positioning system based on multi-sensor fusion, characterized in that: include: A multi-sensor positioning data acquisition module is used to acquire a plurality of positioning coded digital signals collected by the positioning encoder at a plurality of time points within a predetermined time period and an absolute value Gray code provided by the Gray bus locator at the plurality of time points; a first positioning data feature extraction module, configured to arrange the multiple positioning coded digital signals at the multiple time points into a positioning coding input vector according to the time dimension and then pass the vector through a first multi-scale neighborhood feature extraction module to obtain a multi-scale positioning coding feature vector; A second positioning data feature extraction module is configured to arrange the absolute value Gray codes of the multiple time points according to the time dimension into an absolute value Gray code input vector and then pass it through a second multi-scale neighborhood feature extraction module to obtain a multi-scale Gray code feature vector; A feature enhancement module is used to perform feature-level data enhancement on the multi-scale positioning coding feature vector and the multi-scale Gray code feature vector based on a Gaussian density map to obtain a positioning coding feature matrix and a Gray code feature matrix; a contribution calculation module, configured to pass the positioning code feature matrix and the Gray code feature matrix through a pre-classifier to obtain a first probability value and a second probability value; a feature fusion module, configured to fuse the multi-scale positioning coding feature vector and the multi-scale Gray code feature vector based on the first probability value and the second probability value to obtain a decoding feature vector; as well as The positioning result generating module is used to decode and regress the decoded feature vector through a decoder to obtain a decoded value, and the decoded value is used to represent the rotation angle of the cantilever of the bucket wheel machine at the current time point.

2. The high-precision positioning system based on multi-sensor fusion according to claim 1, characterized in that: The first multi-scale neighborhood feature extraction module includes: a first convolutional layer, a second convolutional layer parallel to the first convolutional layer, and a first multi-scale feature fusion layer connected to the first convolutional layer and the second convolutional layer, wherein the first convolutional layer uses a one-dimensional convolution kernel with a first length, and the second convolutional layer uses a one-dimensional convolution kernel with a second length.

3. The high-precision positioning system based on multi-sensor fusion according to claim 2, characterized in that: The first positioning data feature extraction module includes: a first neighborhood-scale feature extraction unit, configured to input the positioning code input vector into a first convolutional layer of the first multi-scale neighborhood feature extraction module to obtain a first neighborhood-scale positioning code feature vector, wherein the first convolutional layer has a first one-dimensional convolution kernel of a first length; a second neighborhood-scale feature extraction unit, configured to input the location encoding input vector into a second convolutional layer of the first multi-scale neighborhood feature extraction module to obtain a second neighborhood-scale location encoding feature vector, wherein the second convolutional layer has a second one-dimensional convolution kernel of a second length, and the first length is different from the second length; and a multi-scale feature fusion unit, configured to concatenate the first neighborhood scale positioning coding feature vector and the second neighborhood scale positioning coding feature vector to obtain the multi-scale positioning coding feature vector; The first neighborhood scale feature extraction unit is further configured to: use the first convolution layer of the first multi-scale neighborhood feature extraction module to perform one-dimensional convolution encoding on the positioning code input vector using the following formula to obtain a first neighborhood scale positioning code feature vector; Wherein, the formula is: Wherein, a is the width of the first convolution kernel in the x direction, F(a) is the first convolution kernel parameter vector, G(xa) is the local vector matrix operated with the convolution kernel function, w is the size of the first convolution kernel, and X represents the positioning code input vector; and The second neighborhood scale feature extraction unit is further configured to: use the second convolution layer of the first multi-scale neighborhood feature extraction module to perform one-dimensional convolution encoding on the positioning code input vector using the following formula to obtain the second neighborhood scale positioning code feature vector; Wherein, the formula is: Wherein, b is the width of the second convolution kernel in the x direction, F(b) is the second convolution kernel parameter vector, G(xb) is the local vector matrix operated with the convolution kernel function, m is the size of the second convolution kernel, and X represents the positioning encoding input vector.

4. The high-precision positioning system based on multi-sensor fusion according to claim 3, characterized in that: The second multi-scale neighborhood feature extraction module includes: a third convolutional layer, a fourth convolutional layer parallel to the third convolutional layer, and a second multi-scale feature fusion layer connected to the third convolutional layer and the fourth convolutional layer, wherein the third convolutional layer uses a one-dimensional convolution kernel with a third length, and the fourth convolutional layer uses a one-dimensional convolution kernel with a fourth length.

5. The high-precision positioning system based on multi-sensor fusion according to claim 4, characterized in that: The feature enhancement module includes: a Gaussian density map construction unit, configured to construct a Gaussian density map of the multi-scale positioning coding feature vector and the multi-scale Gray code feature vector to obtain a first Gaussian density map and a second Gaussian density map, wherein the mean vectors of the first Gaussian density map and the second Gaussian density map are respectively the multi-scale positioning coding feature vector and the multi-scale Gray code feature vector, and the covariance matrices of the first Gaussian density map and the second Gaussian density map are respectively the variance between the eigenvalues ​​of the corresponding two positions in the multi-scale positioning coding feature vector and the variance between the eigenvalues ​​of the corresponding two positions in the multi-scale Gray code feature vector; The Gaussian discretization unit is used to discretize the Gaussian distribution of each position in the first Gaussian density map and the second Gaussian density map to obtain a positioning coding feature matrix and a Gray code feature matrix.

6. The high-precision positioning system based on multi-sensor fusion according to claim 5, characterized in that: The contribution calculation module is further configured to: use the pre-classifier to pass the positioning code feature matrix and the Gray code feature matrix through the pre-classifier respectively to obtain a first probability value and a second probability value according to the following formula; wherein the formula is: O=softmax{(W n , B n ):...:(W1,B1)|Project(F)}, wherein Project(F) represents the projection of the positioning coding feature matrix and the Gray code feature matrix into a vector, W1 to W n is the weight matrix of each fully connected layer, B1 to B n Represents the bias vector of each fully connected layer.

7. The high-precision positioning system based on multi-sensor fusion according to claim 6, characterized in that: The feature fusion module is further configured to perform a Hilbert space constraint of a vector modulus basis on the multi-scale positioning coding feature vector and the multi-scale Gray code feature vector using the following formula to obtain the decoding feature vector, wherein the formula is: Where V1 and V2 represent the multi-scale positioning coding feature vector and the multi-scale Gray code feature vector respectively, V d represents the decoded feature vector, ||·||2 represents the two-norm of the vector, Cov 1D Represents a one-dimensional convolution operation, that is, the convolution operator (||V1||2, ||V2||2, V1V2 T ) for the eigenvector Perform one-dimensional convolution, represents a dot-add operation on the feature vector, and p1 and p2 are the first probability value and the second probability value.

8. The high-precision positioning system based on multi-sensor fusion according to claim 7, characterized in that: The positioning result generating module is further used to: use the decoder to perform decoding regression on the decoding feature vector according to the following formula to obtain a decoding value representing the rotation angle of the boom of the bucket wheel machine at the current time point; Wherein, the formula is: Where X represents the decoded feature vector, Y is the decoded value, and W is the weight matrix. Represents matrix multiplication.

9. A high-precision positioning method based on multi-sensor fusion, characterized in that: include: Acquire a plurality of positioning coded digital signals collected by the positioning encoder at a plurality of time points within a predetermined time period and an absolute value Gray code provided by the Gray bus locator at the plurality of time points; Arranging the multiple positioning coding digital signals at the multiple time points according to the time dimension into a positioning coding input vector and then passing it through a first multi-scale neighborhood feature extraction module to obtain a multi-scale positioning coding feature vector; Arranging the absolute value Gray codes of the multiple time points according to the time dimension into an absolute value Gray code input vector and then passing it through a second multi-scale neighborhood feature extraction module to obtain a multi-scale Gray code feature vector; Based on the Gaussian density map, feature-level data enhancement is performed on the multi-scale positioning coding feature vector and the multi-scale Gray code feature vector to obtain a positioning coding feature matrix and a Gray code feature matrix; Passing the positioning code feature matrix and the Gray code feature matrix through a pre-classifier to obtain a first probability value and a second probability value respectively; Based on the first probability value and the second probability value, fusing the multi-scale positioning coding feature vector and the multi-scale Gray code feature vector to obtain a decoding feature vector; as well as The decoded feature vector is decoded and regressed by a decoder to obtain a decoded value, and the decoded value is used to represent the rotation angle of the cantilever of the bucket wheel machine at the current time point.

10. The high-precision positioning method based on multi-sensor fusion according to claim 9, characterized in that: The decoding feature vector is decoded and regressed by a decoder to obtain a decoded value, wherein the decoded value is used to represent the rotation angle of the cantilever of the bucket wheel machine at the current time point, including: using the decoder to decode and regress the decoding feature vector according to the following formula to obtain a decoded value representing the rotation angle of the cantilever of the bucket wheel machine at the current time point; Wherein, the formula is: Where X represents the decoded feature vector, Y is the decoded value, and W is the weight matrix. Represents matrix multiplication.

Images