Deep learning positioning method and device based on pixelated magnetic nail information

Abstract

The present invention discloses a deep learning positioning method and device based on pixelated magnetic nail information, which relates to the field of magnetic nail positioning. The method comprises obtaining permanent magnet magnetic nail information of a point to be positioned, wherein the permanent magnet magnetic nail information includes three-axis magnetic field strength data collected by a magnetic sensor; pixelating the permanent magnet magnetic nail information to obtain three-channel pixel data corresponding to the three-axis magnetic field strength data; inputting the pixel data corresponding to the permanent magnet magnetic nail information of the point to be positioned into a trained deep learning neural network to obtain the position and posture of the permanent magnet of the point to be positioned. By converting the optimization problem of the traditional magnetic dipole model into a gradient regression problem of deep learning, the algorithm is solved by using a three-color image processing method based on the magnetic field strength data collected by a magnetic sensor array, the computational cost of magnetic nail positioning is reduced, the operation rate is increased, the position accuracy and posture accuracy of magnetic nail information positioning are improved, and the problem of complex calculation faced by the traditional magnetic nail positioning method is solved.

Description

Technical Field

[0001] The present invention relates to the field of magnetic nail positioning, and in particular to a deep learning positioning method and device based on pixelated magnetic nail information. Background Art

[0002] Magnetic strips and magnetic nails are important information sources for autonomous guided vehicles (AGVs) to perceive global positions. Autonomous guided vehicles (AGVs) based on magnetic strips have been widely used in smart warehouses and smart factories. However, this navigation solution requires large-scale installation of magnetic strips, which is costly. At the same time, the magnetic strips are prone to wear after long-term operation. Lost magnetic strips will cause AGV failure and loss of control. Compared with the magnetic strip navigation solution, the magnetic nail navigation solution is a lower-cost and low-maintenance solution. By burying permanent magnets underground, wear and maintenance costs can be avoided. Magnetic nail navigation has the advantages of high positioning accuracy, no restrictions on sensor field of view and measurement distance, environmental interference resistance, low cost and low maintenance. However, the traditional magnetic nail positioning algorithm requires the calculation of multivariate high-order equations, and the computational efficiency is low. Therefore, the promotion of autonomous guided robots based on magnetic nails is limited. The challenge of magnetic nail positioning is to quickly and accurately obtain high-precision positioning position and posture.

[0003] Among the existing patents, in the invention patent with application number CN202110984805.4 and patent name “A method and device for locating magnetic markers based on a magnetic dipole model”, the magnetic dipole model is simplified into a univariate quartic equation to solve the two-dimensional position of the magnetic nail. This method requires a huge amount of calculation and is therefore not suitable for real-time magnetic nail navigation. In the invention patent with application number CN202110943111.6 and patent name “A navigation method and dispatching system for a guided vehicle based on magnetic nails”[], the position coordinates of the magnetic nails are obtained by weighted fusion of the posture information of multiple magnetic sensors. This solution has poor magnetic nail positioning accuracy. In the invention patent with application number CN201811060863.2 and patent name "AGV navigation travel deviation correction method based on BP network" [authorization announcement number: CN109506650B], RFID electronic tags and magnetic nails are used to obtain AGV inertial navigation driving deviation correction. This scheme uses magnetic nail information to compensate for the missing data of inertial navigation data. However, this scheme cannot obtain the accurate coordinates of the magnetic nail under a single magnetic nail sensor.

[0004] Among the existing magnetic nail positioning methods, the most common method is the Levenberg-Marquardt (LM) optimization algorithm. However, the LM algorithm requires an initial value at the beginning of iteration and search. If the initial value is far from the true value, the accuracy and efficiency of the LM algorithm will be greatly affected. One solution to provide an initial value is to deploy an intelligent optimization algorithm with global convergence capability. Lv Bowen et al. [B. Lv, Y. Qin, H. Dai, and S. Su, “Improvinglocalization success rate of three magnetic targets using individual memory-based WO-LM algorithm,” IEEE Sensors Journal, vol. 21, no. 19, pp. 21750-21758, Oct. 2021.] proposed a combination algorithm based on the whale optimization algorithm (WOA) and the LM algorithm to solve the problem of obtaining the initial value of the LM algorithm. However, due to the complex cost function, the optimization algorithm will incur computational costs. Dai Houde and others studied an analytical method based on a simplified magnetic dipole model [H. Dai, P. Guo, S. Su, S.Song, S. Zhao, and S. Cheng, “A simplified magnetic positioning approachbased on analytical method and data fusion for automated guided vehicles,” IEEE / ASME Transactions on Mechatronics, early access, Sep. 2021.], which can derive the computational expression for the two-dimensional position of a single magnetic pin. This analytical method reduces the computational complexity, but it cannot calculate the orientation of the magnet. Summary of the invention

[0005] In view of the above-mentioned problems such as difficulty in quickly and accurately obtaining high-precision positioning position and posture, the purpose of the embodiments of the present application is to propose a deep learning positioning method and device based on pixelated magnetic nail information to solve the technical problems mentioned in the above background technology section.

[0006] In a first aspect, the present invention provides a deep learning positioning method based on pixelated magnetic nail information, comprising the following steps:

[0007] S1, obtaining permanent magnet nail information, the permanent magnet nail information includes three-axis magnetic field strength data collected by the magnetic sensor;

[0008] S2, converting the permanent magnet nail information into pixels to obtain three-channel pixel data corresponding to the three-axis magnetic field intensity data;

[0009] S3, obtaining the permanent magnet pin information on the calibration plate, obtaining the pixel data corresponding to the permanent magnet pin information on the calibration plate through steps S1-S2, combining the position and posture of the permanent magnet on the calibration plate to form training data, using the training data to train the deep learning neural network to obtain a trained deep learning neural network;

[0010] S4, obtaining the permanent magnet pin information of the point to be positioned, obtaining the pixel data corresponding to the permanent magnet pin information of the point to be positioned through steps S1-S2, inputting the pixel data corresponding to the permanent magnet pin information of the point to be positioned into the trained deep learning neural network, and obtaining the position and posture of the permanent magnet of the point to be positioned.

[0011] As a preference, l The three-axis magnetic field intensity vector B measured by the magnetic sensor l for:

[0012] ;

[0013] Among them, the attitude vector of the permanent magnet is H0=( m , n , p ), m 2 + n 2 + p 2 = 1;

[0014] No. l The three-axis magnetic field intensity data collected by the magnetic sensor is the magnetic field intensity vector B l Magnetic field strength data in XYZ axis ( B lx , B ly , B lz ); B T is a constant related to the material of the permanent magnet pin;

[0015] The center position of the permanent magnet is ( a , b , c ); No. l The position of the magnetic sensor is ( x l , y l , zl );P l is the center position of the permanent magnet ( a , b , c ) to l The position of the magnetic sensor ( x l , y l , z l ) vector; R l P l The Euclidean distance length, .

[0016] Preferably, step S2 specifically includes:

[0017] The magnetic sensor array is composed of N magnetic sensors. The three-axis magnetic field strength data collected by the magnetic sensor array is:

[0018] ;

[0019] in, N is the number of magnetic sensors;

[0020] The three-axis magnetic field strength data is converted into 3 i OK j Column matrix form:

[0021] ;

[0022] According to the 3 i OK j The three-channel pixel data is calculated in matrix form:

[0023] .

[0024] Preferably, the deep learning neural network includes convolutional neural network (CNN), BP network (BPNN), long short-term memory artificial neural network (LSTM), and residual neural network (ResNet).

[0025] Preferably, the deep learning neural network includes an input layer, a hidden layer and an output layer, and the pixel data of the three channels are input into the input layer, and the hidden layer n Layer Network L n No. i Neurons n i The input result of z i (n) for:

[0026] ;

[0027] in, z i (n) It is n Layer Network L n No. i Neurons n i Input result of; [ w 1i (n) , w 2i (n) , w 3i (n) ] yes To n Layer Network L n No. i Neurons n i The weight of b 1i (n) is the deviation value b 1 to n Layer Network L n No. i Neurons n i The weight of

[0028] No. n Layer Network L n No. i Neurons n i The output of a i (n) for:

[0029] ;

[0030] in, z i (n) It is n Layer Network L n No. i Neurons n i The input result of f ( z i (n) ) is the activation function of the neural network;

[0031] Output of a deep learning neural network z i (n) for:

[0032] ;

[0033] in,[ a 1i (n-1) , a 2i (n-1) , a 3i (n-1) ] is the n -1 layer of network neurons n 1, n 2 and n Output result of 3; w 1i (n) , w 2i (n) , w 3i (n) ] yes[ a 1i (n-1) , a 2i (n-1) , a 3i (n-1) ] to n Layer Network L n The weight of the output result; b ni (n) is the deviation value b n The weight of the output result.

[0034] Preferably, the hidden layer is the first layer of the network L 1, contains 3 neurons, the output layer is the second layer network L 2, contains 1 neuron; the first layer of the network L The first neuron of 1 n Input result of 1 z 1 (1) for:

[0035] ;

[0036] in, [ w 11 (1) , w21 (1) , w 31 (1) ] yes To Layer 1 Network L The first neuron of 1 n A weight of 1; b 11 (1) is the deviation value b 1 to Layer 1 Network L The first neuron of 1 n A weight of 1;

[0037] Layer 1 Network L 1 The second neuron n Input result of 2 z 2 (1) for:

[0038] ;

[0039] in, z 2 (1) It is a layer 1 network L The second neuron of 1 n 2 input result; [ w 12 (1) , w 22 (1) , w 32 (1) ] yes To Layer 1 Network L The second neuron of 1 n A weight of 2; b 12 (1) is the deviation value b 1 to Layer 1 Network L The second neuron of 1 n A weight of 2;

[0040] Layer 1 Network L 1Third neuron n Input result of 3 z 3 (1) for:

[0041] ;

[0042] in, z 3 (1) It is a layer 1 network L The third neuron of 1 n 3 input result; [w 13 (1) , w 23 (1) , w 33 (1) ] yes To Layer 1 Network L The third neuron of 1 n A weight of 3; b 13 (1) is the deviation value b 1 to Layer 1 Network L The third neuron of 1 n A weight of 3;

[0043] z 1 (1) , z 2 (1) and z 3 (1) Converted into matrix form, layer 1 network L 1 Input Matrix Z 1 is:

[0044] ;

[0045] Output of the second layer of deep learning neural network z 1 (2) for:

[0046] ;

[0047] in, [ a 11 (1) , a 21 (1) , a 31 (1) ] is a layer 1 network L 1 of 3 neurons n 1, n 2 and n Output result of 3; w 11 (2) , w 21 (2) , w 31 (2) ] yes[ a 11 (1) , a21 (1) , a 31 (1) ] to the Layer 2 network L The weight of the neuron 2; b 21 (2) is the deviation value b 2 to Layer 2 Network L The weight of the neuron 2;

[0048] Output of a deep learning neural network Y T This is the output result of the last layer of neural network:

[0049] ;

[0050] Among them, n The output of the layer network is the output of the neural network.

[0051] As a preferred loss function used in the deep learning neural network training process E for:

[0052] ;

[0053] in, n is the number of layers of the neural network; z i (n) It is n Layer Network L n No. i Neurons n i The input result.

[0054] In a second aspect, the present invention provides a deep learning positioning device based on pixelated magnetic nail information, comprising:

[0055] A data acquisition module is configured to acquire permanent magnet nail information, wherein the permanent magnet nail information includes three-axis magnetic field intensity data collected by a magnetic sensor;

[0056] The data conversion module is configured to perform pixel conversion on the permanent magnet nail information to obtain three-channel pixel data corresponding to the three-axis magnetic field intensity data;

[0057] A training module is configured to obtain information about permanent magnet pins on a calibration plate, obtain pixel data corresponding to the information about permanent magnet pins on the calibration plate by executing a data acquisition module and a data conversion module, and form training data in combination with the position and posture of the permanent magnet on the calibration plate, and use the training data to train a deep learning neural network to obtain a trained deep learning neural network;

[0058] The positioning module is configured to obtain the permanent magnet nail information of the point to be positioned, and execute the data acquisition module and the data conversion module to obtain the pixel data corresponding to the permanent magnet nail information of the point to be positioned, and input the pixel data corresponding to the permanent magnet nail information of the point to be positioned into the trained deep learning neural network to obtain the position and posture of the permanent magnet of the point to be positioned.

[0059] In a third aspect, the present invention provides an electronic device comprising one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner in the first aspect.

[0060] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any implementation manner in the first aspect.

[0061] Compared with the prior art, the present invention has the following beneficial effects:

[0062] (1) The deep learning positioning method based on pixelated magnetic nail information proposed in the present invention converts the three-axis magnetic field strength data of the permanent magnet magnetic nail information into three-channel pixel data, so that the magnetic nail timing signal is input into the deep learning neural network in the form of pixel image.

[0063] (2) The deep learning positioning method based on pixelated magnetic nail information proposed in the present invention can convert the magnetic field strength data collected by the magnetic sensor array into pixel data by pixelation, and further input the pixel data into the deep learning neural network for positioning, so as to obtain the position and posture of the point to be positioned, thereby reducing the computational cost of magnetic nail positioning and improving the operation rate.

[0064] (3) The deep learning positioning method based on pixelated magnetic nail information proposed in the present invention solves the computational complexity problem faced by traditional magnetic nail positioning methods by transforming the optimization problem of the traditional magnetic dipole model into a deep learning gradient regression problem. It can effectively improve the position accuracy and attitude accuracy of magnetic nail information positioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0066] Figure 1 is an exemplary device architecture diagram to which an embodiment of the present application may be applied;

[0067] Figure 2 A schematic diagram of a process of a deep learning positioning method based on pixelated magnetic nail information according to an embodiment of the present application;

[0068] Figure 3 A schematic diagram of the correspondence between magnetic field intensity data and pixel image converted from pixel data of a deep learning positioning method based on pixelated magnetic nail information according to an embodiment of the present application;

[0069] Figure 4 A schematic diagram of the structure of a deep learning neural network of a deep learning positioning method based on pixelated magnetic nail information according to an embodiment of the present application;

[0070] Figure 5 A schematic diagram of a deep learning positioning device based on pixelated magnetic nail information according to an embodiment of the present application;

[0071] Figure 6 It is a schematic diagram of the structure of a computer device suitable for implementing the electronic device of the embodiment of the present application. DETAILED DESCRIPTION

[0072] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0073] Figure 1 An exemplary device architecture 100 is shown to which the deep learning positioning method based on pixelated magnetic nail information or the deep learning positioning device based on pixelated magnetic nail information of the embodiments of the present application can be applied.

[0074] like Figure 1As shown, the device architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0075] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various applications, such as data processing applications, file processing applications, etc., can be installed on terminal devices 101, 102, 103.

[0076] Terminal devices 101, 102, 103 may be hardware or software. When terminal devices 101, 102, 103 are hardware, they may be various electronic devices, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, etc. When terminal devices 101, 102, 103 are software, they may be installed in the electronic devices listed above. They may be implemented as multiple software or software modules (for example, software or software modules for providing distributed services), or as a single software or software module. No specific limitation is made here.

[0077] The server 105 may be a server that provides various services, such as a background data processing server that processes files or data uploaded by the terminal devices 101, 102, and 103. The background data processing server may process the acquired files or data and generate processing results.

[0078] It should be noted that the deep learning positioning method based on pixelated magnetic nail information provided in the embodiment of the present application can be executed by the server 105, or by the terminal devices 101, 102, and 103. Accordingly, the deep learning positioning device based on pixelated magnetic nail information can be set in the server 105, or in the terminal devices 101, 102, and 103.

[0079] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided as required. In the case where the processed data does not need to be obtained remotely, the above device architecture may not include a network, but only a server or a terminal device.

[0080] Figure 2 A deep learning positioning method based on pixelated magnetic nail information provided by an embodiment of the present application is shown, comprising the following steps:

[0081] S1, obtaining permanent magnet pin information, the permanent magnet pin information includes three-axis magnetic field strength data collected by the magnetic sensor.

[0082] In a specific embodiment, l The three-axis magnetic field intensity vector B measured by the magnetic sensor l for:

[0083] ;

[0084] Among them, the attitude vector of the permanent magnet is H0=( m , n , p ), m 2 + n 2 + p 2 = 1;

[0085] No. l The three-axis magnetic field intensity data collected by the magnetic sensor is the magnetic field intensity vector B l Magnetic field strength data in XYZ axis ( B lx , B ly , B lz ); B T is a constant related to the material of the permanent magnet pin;

[0086] The center position of the permanent magnet is ( a , b , c ); No. l The position of the magnetic sensor is ( x l , y l , z l );P l is the center position of the permanent magnet ( a , b , c ) to l The position of the magnetic sensor ( x l , y l , z l ) vector; R l P l The Euclidean distance length, .

[0087] Specifically, a magnetic sensor array composed of several magnetic sensors can be used to collect permanent magnet magnetic nail information, where the permanent magnet is a magnetic marker, including but not limited to a magnetic block, a magnet, etc.; the magnetic nail is used as a positioning marker. The specific step of collecting permanent magnet magnetic nail information is to obtain the magnetic field strength data of the three axes XYZ of each magnetic sensor through a three-dimensional magnetic sensor array.

[0088] S2, performing pixel conversion on the permanent magnet pin information to obtain three-channel pixel data corresponding to the three-axis magnetic field intensity data.

[0089] In a specific embodiment, step S2 specifically includes:

[0090] The magnetic field strength data of the three axes are:

[0091] ;

[0092] in, N is the number of magnetic sensors;

[0093] The three-axis magnetic field strength data is converted into 3 i OK j Column matrix form:

[0094] ;

[0095] According to the 3 i OK j The three-channel pixel data is calculated in matrix form:

[0096] .

[0097] Specifically, the permanent magnet information measured by the pixelated magnetic sensor array converts the three-axis magnetic field strength data of each magnetic sensor into three-channel pixel data. Figure 3 , convert the permanent magnet nail information into three-channel pixel data to form a pixel image. The color and pixel value of each pixel in the pixel image correspond to the magnetic field intensity data measured by the magnetic sensor. The pixel value of the black pixel is (0, 0, 0), and the pixel value of the white pixel is (255, 255,255). The larger the magnetic field intensity data, the lighter the color of the pixel of the corresponding pixel image. Figure 3 Indicates 3 groups The corresponding pixel image, (a), (b), (c) below it represent the corresponding positions and poses.

[0098] In one embodiment, the magnetic field strength data of 48 channels of the three-axis magnetic sensor X T It can be expressed as:

[0099] ;

[0100] in, B x It is a matrix composed of magnetic field strength on the X-axis; B y It is a matrix composed of magnetic field strength on the Y axis; B z is a matrix composed of magnetic field strength on the Z axis. Therefore, the magnetic field strength data X T The size of is (3, 4, 4), which is equivalent to the pixel information of an RGB (red, green, blue) image with three channels.

[0101] S3, obtaining the permanent magnet pin information on the calibration plate, obtaining the pixel data corresponding to the permanent magnet pin information on the calibration plate through steps S1-S2, combining the position and posture of the permanent magnet on the calibration plate to form training data, using the training data to train the deep learning neural network to obtain a trained deep learning neural network.

[0102] In a specific embodiment, the deep learning neural network includes a convolutional neural network (CNN), a BP network (BPNN), a long short-term memory artificial neural network (LSTM), and a residual neural network (ResNet).

[0103] In a specific embodiment, the deep learning neural network includes an input layer, a hidden layer and an output layer. The pixel data of the three channels is input into the input layer. n Layer Network L n No. i Neurons n i The input result of z i (n) for:

[0104] ;

[0105] in, z i (n) It is n Layer Network L n No. i Neurons n i Input result of; [ w 1i (n) , w 2i (n) , w 3i(n) ] yes To n Layer Network L n No. i Neurons n i The weight of b 1i (n) is the deviation value b 1 to n Layer Network L n No. i Neurons n i The weight of

[0106] No. n Layer Network L n No. i Neurons n i The output of a i (n) for:

[0107] ;

[0108] in, z i (n) It is n Layer Network L n No. i Neurons n i The input result of f ( z i (n) ) is the activation function of the neural network;

[0109] Output of a deep learning neural network z i (n) for:

[0110] ;

[0111] in,[ a 1i (n-1) , a 2i (n-1) , a 3i (n-1) ] is the n -1 layer of network neurons n1, n 2 and n Output result of 3; w 1i (n) , w 2i (n) , w 3i (n) ] yes[ a 1i (n-1) , a 2i (n-1) , a 3i (n-1) ] to n Layer Network L n The weight of the output result; b ni (n) is the deviation value b n The weight of the output result.

[0112] In one embodiment, reference Figure 4 , the hidden layer is the first layer of the network L 1, contains 3 neurons, the output layer is the second layer network L 2, contains 1 neuron. The first layer of the network L The first neuron of 1 n Input result of 1 z 1 (1) for:

[0113] ;

[0114] in, [ w 11 (1) , w 21 (1) , w 31 (1) ] yes To Layer 1 Network L The first neuron of 1 n A weight of 1; b 11 (1) is the deviation value b 1 to Layer 1 Network L The first neuron of 1 n A weight of 1;

[0115] Layer 1 Network L1 The second neuron n Input result of 2 z 2 (1) for:

[0116] ;

[0117] in, z 2 (1) It is a layer 1 network L The second neuron of 1 n 2 input result; [ w 12 (1) , w 22 (1) , w 32 (1) ] yes To Layer 1 Network L The second neuron of 1 n A weight of 2; b 12 (1) is the deviation value b 1 to Layer 1 Network L The second neuron of 1 n A weight of 2;

[0118] Layer 1 Network L 1Third neuron n Input result of 3 z 3 (1) for:

[0119] ;

[0120] in, z 3 (1) It is a layer 1 network L The third neuron of 1 n 3 input result; [ w 13 (1) , w 23 (1) , w 33 (1) ] yes To Layer 1 Network L The third neuron of 1 n A weight of 3; b 13 (1) is the deviation value b 1 to Layer 1 Network L The third neuron of 1 nA weight of 3;

[0121] z 1 (1) , z 2 (1) and z 3 (1) Converted into matrix form, layer 1 network L 1 Input Matrix Z 1 is:

[0122] ;

[0123] Output of a deep learning neural network z 1 (2) for:

[0124] ;

[0125] in, [ a 11 (1) , a 21 (1) , a 31 (1) ] is a layer 1 network L 1 of 3 neurons n 1, n 2 and n Output result of 3; w 11 (2) , w 21 (2) , w 31 (2) ] yes[ a 11 (1) , a 21 (1) , a 31 (1) ] to the Layer 2 network L The weight of the neuron 2; b 21 (2) is the deviation value b 2 to Layer 2 Network L The weight of the neuron 2;

[0126] Output of a deep learning neural network Y T This is the output result of the last layer of neural network:

[0127] ;

[0128] Among them, n The output of the layer network is the output of the neural network.

[0129] In a specific embodiment, the loss function used in the deep learning neural network training process is E for:

[0130] ;

[0131] in, n is the number of layers of the neural network; z i (n) It is n Layer Network L n No. i Neurons n i The input result.

[0132] Specifically, the training steps of the deep learning neural network are:

[0133] 1. Forward calculation: input the three-channel pixel data corresponding to the three-axis magnetic field intensity data into the input layer, and calculate the output result according to the network weight of each layer.

[0134] 2. Calculate the loss function; calculate the loss function for each deep learning neural network training. By re-inputting the deep learning neural network, the loss function can be reduced to obtain more accurate position and posture results.

[0135] 3. Output the position and posture results calculated by the deep learning neural network.

[0136] Specifically, a permanent magnet is fixed on a calibration plate so that it has a certain position and posture. The permanent magnet pin information of the positioning points of the known position and posture on the calibration plate is collected by a magnetic sensor array and converted into corresponding three-channel pixel data. The known position and posture are used as label data and combined with the corresponding three-channel pixel data into training data. The training data is divided into a training set and a test set. The training set is used to train a deep learning neural network, and the test set is used to test the performance of the trained deep learning neural network. The trained deep learning neural network with better performance is determined, and the trained deep learning neural network is deployed in actual scenarios for application.

[0137] S4, obtaining the permanent magnet pin information of the point to be positioned, obtaining the pixel data corresponding to the permanent magnet pin information of the point to be positioned through steps S1-S2, inputting the pixel data corresponding to the permanent magnet pin information of the point to be positioned into the trained deep learning neural network, and obtaining the position and posture of the permanent magnet of the point to be positioned.

[0138] Specifically, in actual scenarios, the permanent magnet magnetic nail information is collected through a magnetic sensor array, and the three-axis magnetic field strength data of the permanent magnet magnetic nail information is converted into three-channel pixel data. The pixel data is input into a trained deep learning neural network, and the position and posture of the permanent magnet of the point to be located are output, thereby achieving accurate positioning.

[0139] The deep learning positioning method based on pixelated magnetic nail information proposed in the embodiment of the present application converts the magnetic field strength data of the permanent magnet magnetic nail information into RGB pixel data, so that the magnetic nail timing signal is input into the deep learning neural network as an image RGB pixel signal. The traditional magnetic nail positioning method faces the problem of complex calculation. By converting the optimization problem of the traditional magnetic dipole model into a gradient regression problem of deep learning, the magnetic field strength data collected based on the magnetic sensor array can be solved by the algorithm using a three-color image processing method, which reduces the computational cost of magnetic nail positioning and improves the operation rate. Compared with the traditional magnetic nail positioning method, this method can improve the position accuracy and attitude accuracy of magnetic nail information positioning.

[0140] Further references Figure 5 As an implementation of the methods shown in the above figures, the present application provides an embodiment of a deep learning positioning device based on pixelated magnetic nail information. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0141] The embodiment of the present application provides a deep learning positioning device based on pixelated magnetic nail information, which is characterized by comprising:

[0142] The data acquisition module 1 is configured to acquire permanent magnet nail information, wherein the permanent magnet nail information includes three-axis magnetic field strength data collected by the magnetic sensor;

[0143] The data conversion module 2 is configured to perform pixel conversion on the permanent magnet nail information to obtain three-channel pixel data corresponding to the three-axis magnetic field intensity data;

[0144] The training module 3 is configured to obtain the permanent magnet nail information on the calibration plate, obtain the pixel data corresponding to the permanent magnet nail information on the calibration plate by executing the data acquisition module and the data conversion module, and form training data in combination with the position and posture of the permanent magnet on the calibration plate. The deep learning neural network is trained using the training data to obtain a trained deep learning neural network;

[0145] The positioning module 4 is configured to obtain the permanent magnet nail information of the point to be positioned, and execute the data acquisition module and the data conversion module to obtain the pixel data corresponding to the permanent magnet nail information of the point to be positioned, and input the pixel data corresponding to the permanent magnet nail information of the point to be positioned into the trained deep learning neural network to obtain the position and posture of the permanent magnet of the point to be positioned.

[0146] Reference below Figure 6 , which shows an electronic device (eg Figure 1 A schematic diagram of the structure of a computer device 600 (a server or terminal device as shown). Figure 6 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0147] like Figure 6 As shown, the computer device 600 includes a central processing unit (CPU) 601 and a graphics processing unit (GPU) 602, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 603 or a program loaded from a storage part 609 to a random access memory (RAM) 604. In the RAM 604, various programs and data required for the operation of the device 600 are also stored. The CPU 601, GPU 602, ROM 603 and RAM 604 are connected to each other through a bus 605. An input / output (I / O) interface 606 is also connected to the bus 605.

[0148] The following components are connected to the I / O interface 606: an input section 607 including a keyboard, a mouse, etc.; an output section 608 including a display such as a liquid crystal display (LCD), etc. and a speaker, etc.; a storage section 609 including a hard disk, etc.; and a communication section 610 including a network interface card such as a LAN card, a modem, etc. The communication section 610 performs communication processing via a network such as the Internet. A drive 611 can also be connected to the I / O interface 606 as needed. A removable medium 612, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 611 as needed so that a computer program read therefrom is installed into the storage section 609 as needed.

[0149] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication part 610, and / or installed from a removable medium 612. When the computer program is executed by the central processing unit (CPU) 601 and the graphics processing unit (GPU) 602, the above-mentioned functions defined in the method of the present application are executed.

[0150] It should be noted that the computer-readable medium described in the present application may be a computer-readable signal medium or a computer-readable medium or any combination of the above two. The computer-readable medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared or semiconductor device, device or component, or any combination of the above. More specific examples of computer-readable media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution device, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable medium other than a computer-readable medium that can send, propagate or transmit a program for use by or in conjunction with an instruction execution device, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0151] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0152] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the devices, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based device that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0153] The modules involved in the embodiments described in this application may be implemented by software or hardware, and the modules described may also be set in a processor.

[0154] As another aspect, the present application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or it may exist independently and not be assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by the electronic device, the electronic device: obtains permanent magnet magnetic nail information, the permanent magnet magnetic nail information includes three-axis magnetic field strength data collected by the magnetic sensor; performs pixel conversion on the permanent magnet magnetic nail information to obtain three-channel pixel data corresponding to the three-axis magnetic field strength data; obtains permanent magnet magnetic nail information on the calibration plate, obtains pixel data corresponding to the permanent magnet magnetic nail information on the calibration plate through the above steps, combines the position and posture of the permanent magnet on the calibration plate to form training data, and uses the training data to train the deep learning neural network to obtain a trained deep learning neural network; obtains permanent magnet magnetic nail information of the point to be located, obtains pixel data corresponding to the permanent magnet magnetic nail information of the point to be located through the above steps, and inputs the pixel data corresponding to the permanent magnet magnetic nail information of the point to be located into the trained deep learning neural network to obtain the position and posture of the permanent magnet of the point to be located.

[0155] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above invention concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.

Claims

1. A deep learning positioning method based on pixelated magnetic nail information, characterized in that: The following steps are involved: S1, obtaining permanent magnet nail information, wherein the permanent magnet nail information includes three-axis magnetic field strength data collected by a magnetic sensor; S2, performing pixel conversion on the permanent magnet pin information to obtain three-channel pixel data corresponding to the three-axis magnetic field intensity data, specifically including: The magnetic sensor array is formed by N magnetic sensors, and the three-axis magnetic field strength data collected by the magnetic sensor array is: X T =[B 1x ,B 1y ,B 1z ,…,B lx ,B ly ,B lz ,...,B Nx ,B Ny ,B Nz ] T ; Where N is the number of magnetic sensors, and the three-axis magnetic field intensity vector B measured by the lth magnetic sensor is l for: Among them, the attitude vector of the permanent magnet is H0 = (m,n,p), m 2 +n 2 +p 2 =1; The three-axis magnetic field intensity data collected by the lth magnetic sensor is the magnetic field intensity vector B l Magnetic field strength data in XYZ axis (B lx ,B ly ,B lz );B T is a constant related to the material of the permanent magnet pin; The center position of the permanent magnet is (a, b, c); the position of the lth magnetic sensor is (x l ,y l ,z l );P l is the distance from the center of the permanent magnet (a, b, c) to the position of the lth magnetic sensor (x l ,y l ,z l ) vector; R l P l The Euclidean distance length, The three-axis magnetic field intensity data is converted into a matrix form with 3 i rows and j columns: The pixel data of the three channels are obtained by calculating the matrix form of the three i rows and j columns: S3, obtaining the permanent magnet pin information on the calibration plate, obtaining the pixel data corresponding to the permanent magnet pin information on the calibration plate through steps S1-S2, combining the position and posture of the permanent magnet on the calibration plate to form training data, using the training data to train the deep learning neural network to obtain a trained deep learning neural network; S4, obtaining the permanent magnet nail information of the point to be located, obtaining the pixel data corresponding to the permanent magnet nail information of the point to be located through steps S1-S2, inputting the pixel data corresponding to the permanent magnet nail information of the point to be located into the trained deep learning neural network, and obtaining the position and posture of the permanent magnet of the point to be located, the deep learning neural network includes an input layer, a hidden layer and an output layer, the pixel data of the three channels are input into the input layer, and the nth layer network L in the hidden layer n The i-th neuron n i The input result z i (n) for: Among them, z i (n) is the nth layer network L n The i-th neuron n i The input result of 1i (n) ,w 2i (n) ,w 3i (n) ] is [P1 P2 P3] to the nth layer network L n The i-th neuron n i The weight of 1i (n) is the deviation value b1 to the nth layer network L n The i-th neuron n i The weight of The nth layer network L n The i-th neuron n i The output result is a i (n) for: d i (n) =f(z i (n) ); Among them, z i (n) is the nth layer network L n The i-th neuron n i The input result of f(z i (n) ) is the activation function of the neural network; The output of the deep learning neural network z i (n) for: Among them, [a 1i (n-1) ,a 2i (n-1) ,a 3i (n-1) ] is the output result of the n-1th layer network neurons n1, n2 and n3; [w 1i (n) ,w 2i (n) ,w 3i (n) ] is [a 1i (n-1) ,a 2i (n-1) ,a 3i (n-1) ] to the nth layer network L n The weight of the output result; b ni (n) is the deviation value b n The hidden layer is the first layer network L1, which includes 3 neurons, and the output layer is the second layer network L2, which includes 1 neuron; the input result z1 of the first neuron n1 of the first layer network L1 (1) for: Among them, [w 11 (1) ,w 21 (1) ,w 31 (1) ] is the weight of [P1 P2 P3] to the first neuron n1 of the first layer network L1; b 11 (1) is the weight of the first neuron n1 of the first layer network L1 from the bias value b1; The input result z2 of the second neuron n2 of the first layer network L1 (1) for: Among them, z2 (1) is the input result of the second neuron n2 of the first layer network L1; [w 12 (1) ,w 22 (1) ,w 32 (1) ] is the weight of [P1 P2P3] to the second neuron n2 of the first layer network L1; b 12 (1) is the weight of the bias value b1 to the second neuron n2 of the first layer network L1; The input result z3 of the third neuron n3 of the first layer network L1 (1) for: Among them, z3 (1) is the input result of the third neuron n3 of the first layer network L1; [w 13 (1) ,w 23 (1) ,w 33 (1) ] is the weight of [P1 P2P3] to the third neuron n3 of the first layer network L1; b 13 (1) is the weight of the third neuron n3 from the bias value b1 to the first layer of network L1; z1 (1) , z2 (1) and z3 (1) Converted into matrix form, the input matrix Z1 of the first layer network L1 is: The output of the second layer of deep learning neural network z1 (2) for: Among them, [a 11 (1) ,a 21 (1) ,a 31 (1) ] is the output result of the three neurons n1, n2 and n3 in the first layer network L1; [w 11 (2) ,w 21 (2) ,w 31 (2) ] is [a 11 (1) ,a 21 (1) ,a 31 (1) ] to the weights of the neurons in the second layer network L2; b 21 (2) is the weight of the neuron from the bias value b2 to the second layer network L2; The output result Y of the deep learning neural network T This is the output result of the last layer of neural network: Among them, the output result of the nth layer network is the output result of the neural network, and the loss function E used in the deep learning neural network training process is: Where n is the number of layers of the neural network; z i (n) is the nth layer network L n The i-th neuron n i The input result.

2. A deep learning positioning device based on pixelated magnetic nail information, characterized in that: include: A data acquisition module is configured to acquire permanent magnet nail information, wherein the permanent magnet nail information includes three-axis magnetic field intensity data collected by a magnetic sensor; The data conversion module is configured to perform pixel conversion on the permanent magnet magnetic nail information to obtain three-channel pixel data corresponding to the three-axis magnetic field intensity data, specifically including: The magnetic sensor array is formed by N magnetic sensors, and the three-axis magnetic field strength data collected by the magnetic sensor array is: X T =[B 1x ,B 1y ,B 1z ,…,B lx ,B ly ,B lz ,B Nx ,...,B Ny ,B Nz ] T ; Where N is the number of magnetic sensors, and the three-axis magnetic field intensity vector B measured by the lth magnetic sensor is l for: Among them, the attitude vector of the permanent magnet is H0 = (m,n,p), m 2 +n 2 +p 2 =1; The three-axis magnetic field intensity data collected by the lth magnetic sensor is the magnetic field intensity vector B l Magnetic field strength data in XYZ axis (B lx ,B ly ,B lz );B T is a constant related to the material of the permanent magnet pin; The center position of the permanent magnet is (a, b, c); the position of the lth magnetic sensor is (x l ,y l ,z l );P l is the distance from the center of the permanent magnet (a, b, c) to the position of the lth magnetic sensor (x l ,y l ,z l ) vector; R l P l The Euclidean distance length, The three-axis magnetic field intensity data is converted into a matrix form with 3 i rows and j columns: The pixel data of the three channels are obtained by calculating the matrix form of the three i rows and j columns: A training module is configured to obtain information about permanent magnet pins on a calibration plate, obtain pixel data corresponding to the information about permanent magnet pins on the calibration plate by executing a data acquisition module and a data conversion module, and form training data in combination with the position and posture of the permanent magnet on the calibration plate, and use the training data to train a deep learning neural network to obtain a trained deep learning neural network; The positioning module is configured to obtain the permanent magnet nail information of the point to be positioned, and the pixel data corresponding to the permanent magnet nail information of the point to be positioned is obtained by executing the data acquisition module and the data conversion module, and the pixel data corresponding to the permanent magnet nail information of the point to be positioned is input into the trained deep learning neural network to obtain the position and posture of the permanent magnet of the point to be positioned. The deep learning neural network includes an input layer, a hidden layer and an output layer. The pixel data of the three channels are input into the input layer, and the nth layer network L in the hidden layer n The i-th neuron n i The input result z i (n) for: Among them, z i (n) is the nth layer network L n The i-th neuron n i The input result of 1i (n) ,w 2i (n) ,w 3i (n) ] is [P1 P2 P3] to the nth layer network L n The i-th neuron n i The weight of 1i (n) is the deviation value b1 to the nth layer network L n The i-th neuron n i The weight of The nth layer network L n The i-th neuron n i The output result is a i (n) for: Among them, z i (n) is the nth layer network L n The i-th neuron n i The input result of f(z i (n) ) is the activation function of the neural network; The output of the deep learning neural network z i (n) for: Among them, [a 1i (n-1) ,a 2i (n-1) ,a 3i (n-1) ] is the output result of the n-1th layer network neurons n1, n2 and n3; [w 1i (n) ,w 2i (n) ,w 3i (n) ] is [a 1i (n-1) ,a 2i (n-1) ,a 3i (n-1) ] to the nth layer network L n The weight of the output result; b ni (n) is the deviation value b n The hidden layer is the first layer network L1, which includes 3 neurons, and the output layer is the second layer network L2, which includes 1 neuron; the input result z1 of the first neuron n1 of the first layer network L1 (1) for: Among them, [w 11 (1) ,w 21 (1) ,w 31 (1) ] is the weight of [P1 P2 P3] to the first neuron n1 of the first layer network L1; b 11 (1) is the weight of the first neuron n1 of the first layer network L1 from the bias value b1; The input result z2 of the second neuron n2 of the first layer network L1 (1) for: Among them, z2 (1) is the input result of the second neuron n2 of the first layer network L1; [w 12 (1) ,w 22 (1) ,w 32 (1) ] is the weight of [P1 P2P3] to the second neuron n2 of the first layer network L1; b 12 (1) is the weight of the bias value b1 to the second neuron n2 of the first layer network L1; The input result z3 of the third neuron n3 of the first layer network L1 (1) for: Among them, z3 (1) is the input result of the third neuron n3 of the first layer network L1; [w 13 (1) ,w 23 (1) ,w 33 (1) ] is the weight of [P1 P2P3] to the third neuron n3 of the first layer network L1; b 13 (1) is the weight of the third neuron n3 from the bias value b1 to the first layer of network L1; z1 (1) , z2 (1) and z3 (1) Converted into matrix form, the input matrix Z1 of the first layer network L1 is: The output of the second layer of deep learning neural network z1 (2) for: Among them, [a 11 (1) ,a 21 (1) ,a 31 (1) ] is the output result of the three neurons n1, n2 and n3 in the first layer network L1; [w 11 (2) ,w 21 (2) ,w 31 (2) ] is [a 11 (1) ,a 21 (1) ,a 31 (1) ] to the weights of the neurons in the second layer network L2; b 21 (2) is the weight of the neuron from the bias value b2 to the second layer network L2; The output result Y of the deep learning neural network T This is the output result of the last layer of neural network: Among them, the output result of the nth layer network is the output result of the neural network, and the loss function E used in the deep learning neural network training process is: Where n is the number of layers of the neural network; z i (n) is the nth layer network L n The i-th neuron n i The input result.

3. An electronic device, comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method as claimed in claim 1.

4. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to claim 1 is implemented.

Images