A lightweight real-time neural network assisted high-precision positioning method

Abstract

The application discloses a kind of lightweight real-time neural network auxiliary high-precision positioning method, belong to neural network technical field.Model training is completed, using the picture of market shooting, after neural network model output obtains the label frame class, center point coordinate (x, y), long w and width h of each billboard in picture, as the input of positioning model, finally obtain multiple angles with the north direction, each billboard corresponds to an angle, using billboard position information on two-dimensional map, draw straight line through each billboard on two-dimensional map, the slope of this straight line and the angle of each billboard output by model correspond, multiple straight lines intersect on two-dimensional map to obtain multiple intersection points, take the center position of multiple intersection points as the final position of this positioning, to complete positioning operation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of neural networks, and more particularly to a lightweight real-time neural network assisted high-precision positioning method. BACKGROUND

[0002] Since the 21st century, indoor positioning technology has developed rapidly. Indoor positioning technologies based on infrared, radio frequency identification, ultra-bandwidth, Bluetooth, and geomagnetic technology have entered people's production and life fields. These positioning technologies generally require additional deployment of positioning equipment, which has a high cost. Thanks to the development of communication technology and Internet technology in recent years, WiFi technology and smart terminals have been widely promoted and applied, which has greatly promoted the development of indoor positioning technology based on WiFi technology. WiFi is widely used in various places where people live and work, and people can easily connect to the surrounding APs with their smart terminals. This promotes the in-depth research and wide application of WiFi-based indoor positioning technology in industrial production and life.

[0003] The most commonly used non-rangefinding positioning technology is the location fingerprint-based positioning technology. Fingerprint positioning technology does not need to consider the attenuation of received signal strength (RSSI) caused by building shielding, and does not need to obtain the positions of APs in advance, so it has been widely researched and applied. Fingerprint positioning generally consists of two steps: offline fingerprint database establishment and online positioning. In the offline fingerprint database establishment stage, the positioning scene is first divided into grids to set reference points, and the RSSI of multiple APs is collected at each reference point to form an RSSI fingerprint vector. The entire fingerprint vector of the positioning scene is sorted to form a fingerprint database. In the online positioning stage, the RSSI fingerprint vector of the to-be-positioned point is collected and matched with the RSSI fingerprint vector in the fingerprint database, and the position of the to-be-positioned point is determined through machine learning.

[0004] In large shopping malls and other environments with complex human flow, RSSI is easily affected by human interference, affecting the positioning result. The traditional WiFi fingerprint positioning technology mainly uses the matching of the RSSI fingerprint vectors collected offline and online to perform positioning, but RSSI is easily affected by the human flow in the shopping mall, and the RSSI collected offline and the RSSI collected in the shopping mall environment with large human flow are difficult to match, reducing the positioning accuracy. This paper proposes a lightweight real-time neural network assisted high-precision positioning method to provide more effective services for indoor positioning in large shopping malls. SUMMARY

[0005] Technical problems to be solved

[0006] The application aims to provide a lightweight real-time neural network assisted high-precision positioning method and a fitting process to solve the problems in the background art.

[0007] Technical scheme

[0008] A lightweight real-time neural network assisted high-precision positioning method, which is composed of a two-dimensional offline map, a neural network model and a positioning model.

[0009] Offline map: a two-dimensional map is built for a large shopping mall, and the billboards of each store merchant are marked on the map, and users can view the map in an offline state.

[0010] Neural network model: pictures around the reference point are taken as model training samples during the offline collection stage, and the target information is obtained by labeling.

[0011] Positioning model: the position of the reference point is marked on the two-dimensional offline map by using the pictures used in the neural network model training stage, and the labeled points and information positioning are recorded.

[0012] Preferably, the neural network model labels the target in the picture with a rectangular frame, and the position information and length-width information of the rectangular frame are used as the labels for model training. Through the trained model, all the multiple targets of interest in the image are found, and the category information and position information of these targets are obtained.

[0013] Preferably, the position information obtained by the neural network model is the position and size.

[0014] Preferably, the positioning model uses the pictures used in the neural network model training stage to mark the position of the reference point on the two-dimensional offline map, records the slope of the line segment connecting the billboard labeled in the picture and the reference point as the label for model training, and uses the position information and length-width information of the labeled frame as the sample for model training. The trained model is used to fit multiple straight lines on the two-dimensional map, and the intersection point of the straight lines is the positioning position.

[0015] Preferably, the offline map selects a suitable reference point, collects the picture information around the reference point, labels the collected pictures, draws a labeled frame, labels the billboards in the pictures, and each labeled frame has corresponding center point coordinates (x, y) relative to the picture and length (w) and width (h) of the labeled frame. The center point coordinates (x, y) and the length (w) and width (h) of the labeled frame are used as the sample for model training, and the billboard information corresponding to the labeled frame is used as the label for model sample training.

[0016] Preferably, the model structure using the target detection model, the input picture size is 416x416x3, and the basic structure is: a DBL layer of 3x3x32, a 3x3x64 convolution layer with a step of 2; a res1 layer, a 3x3x128 convolution layer with a step of 2; a res2 layer, a 3x3x256 convolution layer with a step of 2; a res8 layer, a 3x3x512 convolution layer with a step of 2; a res8 layer, a 3x3x1024 convolution layer with a step of 2; a res4 layer; a 5-layer DBL layer; a DBL layer, and a 1x1 conv convolution layer.

[0017] Preferably, the 3x3x32 conv convolution layer, the BN regularization layer and the Leaky relu activation function layer constitute a 3x3x32 DBL layer, the resn layer is composed of a zero padding layer, a DBL layer and n res_unit residual units, and the output of the res_unit is the sum of the initial input and the output after passing through two DBL layers.

[0018] Preferably, the network structure is constructed, the pre-training weight file is downloaded to accelerate the convergence of the loss function in the training, the training samples and the corresponding labels are prepared, and the size of the training sample pictures is uniformly preprocessed into 416x416x3.

[0019] Preferably, the sample set D is prepared, each sample i and the corresponding label ai, the sample data is normalized, the weight is initialized, and the loss function of the model is:

[0020]

[0021] Preferably, after the model training is completed, the pictures shot in the market are used, the label box class, the center point coordinates (x, y), the length w and the width h of each billboard in the pictures are obtained through the neural network model output, are used as the input of the positioning model, and finally a plurality of angles are obtained to assist in completing the positioning operation.

[0022] Beneficial effects

[0023] Compared with the prior art, the application has the advantages that:

[0024] 1. The neural network model performs rectangular frame labeling on the target in the picture, the position information and the length-width information of the rectangular frame are used as the label of the model training, all the multiple targets in the image are found out through the trained model, and the category information and the position information of the targets are obtained; the position information obtained by the neural network model is the position and the size.

[0025] 2、Positioning model refers to using the picture shooting reference point position used in the neural network model training stage, marking the position on the two-dimensional offline map, recording the line segment slope of the billboard marked in the picture and the reference point as the label of model training, the position information and length-width information of the marking box as the sample of model training, training the model to fit the multiple straight lines on the two-dimensional map, and the intersection of the straight lines is the positioning position.

[0026] 3、Offline map selects appropriate reference points, collects picture information around the reference points, marks the collected pictures, draws marking boxes, and marks billboards in the pictures. Each marking box has corresponding center point coordinates (x, y) relative to the picture and length (w) and width (h) of the marking box. The center point coordinates (x, y) and the length (w) and width (h) of the marking box are used as the sample of model training, and the billboard information corresponding to the marking box is used as the label of model sample training.

[0027] 4、Directly using the shopping mall background information for positioning.

[0028] 5、No need to configure AP wireless access point.

[0029] 6、Lightweight, real-time. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 A schematic view of a lightweight real-time neural network assisted high-precision positioning method of the present application. DETAILED DESCRIPTION

[0031] In the description of the present application, it should be understood that the orientations or positional relationships indicated by the terms "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise" and the like are based on the orientations or positional relationships shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application.

[0032] In the description of the present application, the meaning of "multiple" is two or more, unless otherwise explicitly and specifically limited.

[0033] In the description of the present application, it should be noted that unless otherwise explicitly specified and limited, the terms "mounting", "provided with", "sleeved / connected", "connected" and the like should be understood broadly, for example, "connected" can be fixedly connected, or detachably connected, or integrally connected; can be mechanically connected, or electrically connected; can be directly connected, or indirectly connected through an intermediate medium, or can be the internal communication of two elements. For those skilled in the art, the specific meanings of the above terms in the present application can be understood according to the specific circumstances.

[0034] Please refer to Figure 1 The present application provides a technical solution:

[0035] Embodiment 1

[0036] A lightweight real-time neural network assisted high-precision positioning method, which is composed of a two-dimensional offline map, a neural network model and a positioning model;

[0037] Offline map: a two-dimensional map is built for a large shopping mall, and the billboards of each store merchant are marked on the map, and users can view the map in an offline state;

[0038] Neural network model: pictures around the reference point need to be taken during offline collection as model training samples, and they are labeled to obtain various information of the target;

[0039] Positioning model: the position of the reference point is marked on the two-dimensional offline map by using the pictures used in the neural network model training stage, and the labeled points and information positioning are recorded.

[0040] The neural network model labels the target in the picture with a rectangular frame, and the position information and length-width information of the rectangular frame are used as the labels of the model training. Through the trained model, all the multiple targets of interest in the image are found, and the category information and position information of these targets are obtained. The position information obtained by the neural network model is the position and size.

[0041] Embodiment 2

[0042] The positioning model refers to marking the position of the reference point on the two-dimensional offline map by using the pictures used in the neural network model training stage, recording the slope of the line segment connecting the billboard labeled in the picture and the reference point as the label of the model training, and using the position information and length-width information of the labeled frame as the sample of the model training. The trained model is used to fit a plurality of straight lines on the two-dimensional map, and the intersection of the straight lines is the positioning position.

[0043] Embodiment 3

[0044] The reference point is selected from an offline map, picture information around the reference point is collected, the collected pictures are labeled, a label frame is drawn, and a billboard in the picture is labeled. Each label frame has corresponding center point coordinates (x, y) relative to the picture and the length (w) and width (h) of the label frame. The center point coordinates (x, y) and the length (w) and width (h) of the label frame are used as samples for model training, and the billboard information corresponding to the label frame is used as a label for model sample training.

[0045] Embodiment 4

[0046] The model structure of the target detection model is adopted, the input picture size is 416x416x3, and the basic structure is: a DBL layer of 3x3x32, a convolution layer of 3x3x64 with a step of 2; a res1 layer, a convolution layer of 3x3x128 with a step of 2; a res2 layer, a convolution layer of 3x3x256 with a step of 2; a res8 layer, a convolution layer of 3x3x512 with a step of 2; a res8 layer, a convolution layer of 3x3x1024 with a step of 2; a res4 layer; a 5-layer DBL layer; a DBL layer, and a 1x1 conv convolution layer.

[0047] Embodiment 4

[0048] A 3x3x32 conv convolution layer, a BN regularization layer and a Leaky relu activation function layer constitute a DBL layer of 3x3x32, and the resn layer is composed of a zero padding layer, a DBL layer and n res_unit residual units. The output of the res_unit is the sum of the initial input and the output of the two DBL layers.

[0049] Embodiment 5

[0050] The network structure is constructed, the pre-trained weight file is downloaded to accelerate the convergence of the loss function in the training, the training samples and the corresponding labels are prepared, and the size of the training sample pictures is uniformly preprocessed to 416x416x3.

[0051] Embodiment 6

[0052] The loss function of the neural network model is

[0053]

[0054]

[0055] After the picture is input into the model, the picture is divided into S*S grids, each grid generates B candidate boxes, and finally the corresponding labeled box is obtained, which corresponds to the center point coordinates (x, y) of the labeled box, length w and width h. Wherein the parameter I represents whether the jth candidate box of the ith grid is responsible for predicting this target, if responsible for prediction, it is 1, otherwise it is 0; The parameter confidence C represents the true value, the value of C is determined by whether the labeled box of the grid is responsible for predicting a target, if responsible, C=1, otherwise 0; P represents the classification probability. In the process of training the model, the weights of various types are continuously adjusted to reduce the loss, and finally converge.

[0056] Embodiment 7

[0057] Prepare the sample set D, each sample i and the corresponding label ai, normalize the sample data, initialize the weight preprocessing, and the loss function of the model is

[0058]

[0059] In the process of training the model, the weights of various types are continuously adjusted to reduce the size of the loss function value, and the simulated annealing method is selected to reduce overfitting.

[0060] Embodiment 8

[0061] After the model training is completed, the pictures taken in the shopping mall are used, and the labeled box class, center point coordinates (x, y), length w and width h of each billboard in the picture are obtained through the neural network model output, which are used as the input of the positioning model, and finally a plurality of angles with the north direction are obtained, each billboard corresponds to an angle, the position information of the billboards on the two-dimensional map is used to draw a straight line through each billboard on the two-dimensional map. The slope of the straight line corresponds to the angle of each billboard output by the model, a plurality of straight lines intersect on the two-dimensional map to obtain a plurality of intersection points, and the center position of the plurality of intersection points is taken as the final position of this positioning, thereby completing the positioning operation.

[0062] The basic principles, main features and advantages of the present application are shown and described above. Those skilled in the art should understand that the present application is not limited by the above embodiments, and the above embodiments and descriptions in the specification are only preferred examples of the present application and are not intended to limit the present application. Without departing from the spirit and scope of the present application, various changes and improvements can be made to the present application, and these changes and improvements all fall within the scope of the claimed present application. The scope of protection of the present application is defined by the appended claims and their equivalents.

Claims

1. A lightweight real-time neural network aided high-precision positioning method, characterized in that, The method comprises the following steps: Step 1: building a two-dimensional offline map: a two-dimensional map of a large shopping mall is built, and the billboards of each store merchant are marked on the map; Step 2: building and training a neural network model: in the offline collection stage, pictures around multiple reference points are taken as training samples, and the billboards in the pictures are labeled to obtain the label class, center point coordinates (x, y), length w and width h of each billboard; the neural network model is trained using these training samples; Step 3: real-time positioning using the trained neural network model: the picture of the point to be positioned is collected and input into the neural network model, and the label class, center point coordinates (x, y), length w and width h of each billboard in the picture are output; Step 4: positioning calculation: according to the output of the neural network model, the included angle between each billboard and the north direction is calculated; using the billboard position information on the two-dimensional offline map, a straight line is drawn for each billboard, and the slope of the straight line is related to the included angle of the corresponding billboard; the intersection of multiple straight lines obtains multiple intersection points, and the center position of the intersection points is taken as the final position of the point to be positioned.

2. The lightweight real-time neural network aided high-precision positioning method according to claim 1, characterized in that: The neural network model labels the target in the picture with a rectangular frame, and the position information and length-width information of the rectangular frame are used as the label of the model training. The position information obtained by the neural network model is the position and size.

3. The lightweight real-time neural network aided high-precision positioning method according to claim 1, characterized in that: The positioning model uses the picture shooting reference point position used in the neural network model training stage, marks the position on the two-dimensional offline map, records the slope of the line segment connecting the labeled billboard in the picture and the reference point as the label of the model training, and uses the position information and length-width information of the label frame as the sample of the model training, trains the model to fit multiple straight lines on the two-dimensional map, and obtains multiple intersection points by intersecting multiple straight lines on the two-dimensional map, and takes the center position of the multiple intersection points as the positioning position.

4. The lightweight real-time neural network aided high-precision positioning method according to claim 1, characterized in that: The offline map selects appropriate reference points, collects picture information around the reference points, draws a label frame, labels the billboards in the pictures, each label frame has corresponding center point coordinates (x, y) relative to the picture and length (w) and width (h) of the label frame, and the center point coordinates (x, y) and the length (w) and width (h) of the label frame are used as the sample of the model training, and the billboard information corresponding to the label frame is used as the label of the model sample training.

5. The lightweight real-time neural network aided high-precision positioning method according to claim 1, characterized in that: The model structure of the target detection model is adopted, the input picture size is 416x416x3, and the basic structure is: a DBL layer of 3x3x32, a convolution layer of 3x3x64 with a step of 2; a res1 layer, a convolution layer of 3x3x128 with a step of 2; a res2 layer, a convolution layer of 3x3x256 with a step of 2; a res8 layer, a convolution layer of 3x3x512 with a step of 2; a res8 layer, a convolution layer of 3x3x1024 with a step of 2; a res4 layer; a 5-layer DBL layer; a DBL layer, a 1x1 conv convolution layer.

6. The lightweight real-time neural network aided high-precision positioning method according to claim 2, characterized in that: ​ 7. The lightweight real-time neural network aided high-precision positioning method according to claim 6, characterized in that: 3x3x32 conv convolutional layer, BN regularization layer and Leaky relu activation function layer constitute a layer of 3x3x32 DBL layer, resn layer is composed of zero padding layer, DBL layer and n res_unit residual unit, the output of res_unit is the sum of the initial input and the output after two DBL layers.

8. The lightweight real-time neural network aided high-precision positioning method according to claim 1, characterized in that: The network structure is constructed, the pre-trained weight file is downloaded, the loss function convergence in training is accelerated, the training samples and corresponding labels are prepared, and the size of the training sample pictures is uniformly pretreated to 416x416x3.

9. The lightweight real-time neural network aided high-precision positioning method according to claim 1, characterized in that: Prepare the sample set D, each sample i and the corresponding label ai, normalize the sample data, initialize the weight preprocessing, and the loss function of the model is 10. The lightweight real-time neural network aided high-precision positioning method according to claim 1, characterized in that: After the model training is completed, the pictures taken in the market are used, the label box class, center point coordinates (x, y), length w and width h of each billboard in the picture are obtained through the neural network model output, which are used as the input of the positioning model, and finally multiple angles are obtained to assist the positioning operation.

Images