Distance measurement method in non-line-of-sight environment based on Mistral model fine tuning

Abstract

The invention provides a distance measurement method in a non-line-of-sight environment based on Mistral model fine tuning, and relates to a method for solving a distance measurement problem based on RSSI in a wireless network by using a large language model and an efficient parameter fine tuning technology. The method comprises the following steps: firstly, introducing a pre-training stage, and pre-training a large language model by using a constructed prior text data set; secondly, performing Gaussian filtering processing on the collected data to remove abnormal values, and designing an instruction optimizer based on a decision tree to convert a non-line-of-sight measurement data set to form a non-line-of-sight text data set; wherein the data feature importance in the non-line-of-sight measurement dataset is evaluated through the optimized decision tree model, and then the text description and the text sequence of the non-line-of-sight measurement dataset are automatically adjusted in the process of converting the non-line-of-sight measurement dataset into the non-line-of-sight text dataset, so that the purpose of instruction optimization is achieved. And finally, an LoRA fine tuning algorithm, an LARS optimization algorithm and a Consine scheduling algorithm are integrated in the SFTTrainer, operation steps are simplified for model fine tuning, an automatic process is provided, and the time cost and the calculation cost of model training are reduced. And through continuous iterative optimization of the trainer, a special distance measurement model suitable for the non-line-of-sight environment is finally obtained. The method reduces the training time cost and calculation cost in the fine tuning process, improves the distance measurement precision in the non-line-of-sight environment, and is more suitable for solving the problem of low-cost distance measurement.

Description

Technical Field

[0001] This invention belongs to the fields of communication technology and artificial intelligence, specifically a method for ranging in non-line-of-sight environments using a large language model for ZigBee networks. Background Technology

[0002] In non-line-of-sight (NLOS) environments, the relationship between RSSI and the distance between nodes is not linear and is influenced by various factors. Generally, RSSI decreases with increasing distance, but due to environmental factors, obstacles, antenna orientation, and signal attenuation, this relationship is not a simple one-to-one mapping. To date, to address the complex signal attenuation problem, researchers have used machine learning models to fit the signal attenuation pattern. Common models include recurrent neural networks (RNNs), gated recurrent units (GRNs), convolutional neural networks (CNNs), and backpropagation neural networks. However, these methods are overly reliant on the type and quantity of training data and are prone to overfitting during model training. In line-of-sight environments, ranging errors are consistently above 10% of the positioning range. In complex NLOS environments, ranging errors exceed 1 meter, which is still unsatisfactory.

[0003] With the emergence and widespread application of Large Language Models (LLMs), they have demonstrated adaptability to multiple tasks and exhibited extremely high performance in text generation and knowledge application. LLMs possess powerful model representation capabilities, enabling them to learn complex data patterns and feature relationships. Because LLMs learn from large datasets during training, they exhibit excellent generalization ability and strong transfer learning capabilities. Currently, LLMs show enormous potential in knowledge generation and application. Summary of the Invention

[0004] To address the issues of high ranging errors in wireless communication environments and over-reliance on the number of samples and feature categories in the training dataset, this invention proposes a non-line-of-sight (NLS) ranging method based on multi-stage fine-tuning of the Mistral model. First, a prior text dataset is constructed. Second, a NLS measurement dataset is collected and optimized using a Bayesian optimization decision tree method to form a NLS text dataset. Third, the Mistral large language model is fine-tuned using a multi-stage fine-tuning method to generate a dedicated ranging model. Finally, the dedicated ranging model is deployed and run offline using the Ollama model framework.

[0005] The technical solution adopted by this invention to achieve the above objectives is: a distance measurement method in a non-line-of-sight environment based on Mistral model fine-tuning, comprising the following steps:

[0006] 1) Based on prior knowledge of non-line-of-sight environments and RSSI ranging, and combined with a large language model knowledge base, a prior text dataset is constructed.

[0007] 2) For different occlusion characteristics, record the RSSI information transmitted in the network under non-line-of-sight environment and construct a non-line-of-sight measurement dataset.

[0008] 3) The method of optimizing decision trees based on Bayesian methods is used to optimize the non-line-of-sight measurement dataset to form a non-line-of-sight text dataset;

[0009] 4) Pre-train the Mistral model using a prior text dataset, and train the Mistral model for the ranging task using a non-line-of-sight text dataset; use the trainer to iteratively fine-tune and generate the ranging model.

[0010] 5) For a given scene to be tested, input the occlusion features of the scene into the distance measurement model constructed above to obtain the predicted distance of the indoor non-line-of-sight environment.

[0011] Step 1) specifically involves: utilizing expert knowledge and RSSI ranging knowledge, combined with training data on RSSI ranging knowledge representation from a large language model, summarizing it into a question-answer pair dataset to form a priori text dataset.

[0012] Dataset1={Instruction',Response'}

[0013] Here, Dataset1 represents the prior text dataset, Instruction′ represents the question description of prior knowledge as a sample, and Response′ represents the answer to the question as a knowledge label.

[0014] Step 2) specifically refers to:

[0015] A wireless local area network (WLAN) that builds a wireless ad hoc network includes a coordinator, routers, and end nodes.

[0016] The terminal node, acting as a transmitter, communicates with the router via broadcast.

[0017] The router, as the receiving end, communicates with the coordinator via multicast; the average value of multiple received data points is calculated as a group and regarded as the RSSI value corresponding to the distance between the router and the terminal node.

[0018] The coordinator is used to connect to the PC to collect information;

[0019] Different degrees of occlusion are represented by Fresnel zones in the physical environment to indicate different non-line-of-sight environments. Under the combined influence of three different variables—occlusion object, occlusion distance, and occlusion degree—the RSSI values ​​received by the PC are recorded to form a non-line-of-sight measurement dataset, including the following steps:

[0020] The obstruction area between the router and the terminal node was divided into three different degrees of obstruction: a%, b%, and c%. The following experiment was conducted for each degree of obstruction:

[0021] Select multiple sampling points at different distances between the terminal node and the obstruction;

[0022] For each distance, the RSSI value is collected N times at the sampling point, and the geometric mean after Gaussian filtering is used as the experimental data for that sampling point.

[0023] Finally, for each type of occlusion, experimental data from sampling points at different distances under different degrees of occlusion are used to construct a non-line-of-sight measurement dataset. Each set of experimental data includes multiple occlusion features: occlusion material length, occlusion material width, occlusion material thickness, occlusion distance, occlusion range, and RSSI value.

[0024] Step 3) is as follows:

[0025] (1) Define the objective function of Bayesian optimization as the mean square error function of the validation set, set the range of hyperparameters for Bayesian optimization, and divide the non-line-of-sight measurement dataset into a training set and a validation set;

[0026] (2) Construct a decision tree model using hyperparameters optimized by Bayesian search;

[0027] (3) Use the decision tree model to fit and optimize the training samples and labels on the non-line-of-sight measurement training set, and perform iterative iterations according to the set number of iterations; when the set number of iterations is reached, the optimal combination of hyperparameters is obtained.

[0028] (4) Construct the optimal decision tree model using the optimal hyperparameter combination, and determine the importance score of the feature based on the number of times the feature is split and the information gain of the dataset before and after the split.

[0029] (5) Sort the features of the non-line-of-sight measurement dataset according to their scores. The higher the score, the greater the importance evaluation value of the feature in the optimal decision tree model.

[0030] (6) Sort by features and use a format converter to convert it into a non-line-of-sight text dataset with labels.

[0031] The importance score of a feature is determined by combining the number of times the feature is split and the information gain of the dataset before and after the split, as follows:

[0032]

[0033] Among them, feature_importance i Indicates the importance of the i-th feature in the non-line-of-sight measurement dataset; splits_n i `splits_T` represents the number of times feature `i` is used for splitting; `splits_T` represents the total number of times all features are used for splitting; `gain`... i This represents the total information gain when using feature i for partitioning; gain_all represents the total information gain of all partitions.

[0034] The format converter has the following specific format:

[0035] Dataset2={Instruction”,Input”,Response”}

[0036] Here, "Instruction" represents the description of the ranging problem, "Input" represents the description of the occlusion feature data of the ranging problem, and "Instruction" and "Input" serve as samples; "Response" represents the true distance predicted by Mistral through occlusion features, serving as the true distance label.

[0037] Step 4) is as follows:

[0038] First, the iterative trainer uses the prior text dataset to make predictions through the Mistral model. The model output and the prior text dataset knowledge labels are then compared using the negative log-likelihood function to calculate the loss value of the predicted output.

[0039] Secondly, the parameter update matrix W is utilized in the multi-stage fine-tuner. A and W B The gradient of the current parameters is calculated based on the loss value and parameter update matrix, and then returned to the trainer.

[0040] Then, the optimizer and scheduler deployed in the trainer are used to update the parameters, and the model is iteratively fine-tuned to form the first-stage fine-tuned model.

[0041] Based on the first-stage fine-tuning model, the same trainer, optimizer, and scheduler are used to train the model on a non-line-of-sight text dataset. The mean squared error function is set as the loss function for the second iteration of fine-tuning, and minimizing the mean squared error is set as the optimization objective. The parameter update matrix W is then used with the parameters of the multi-stage fine-tuner. A ′ and W B Implement parameter updates during the fine-tuning process to achieve the second-stage fine-tuning of the model;

[0042] Finally, the matrix W is updated using the weights after two-stage fine-tuning. A W B and W A ′、W B This is combined with the original Mistral model to form a ranging model to predict the distance between the terminal node and the obstruction.

[0043] The weight update matrix W is obtained by using two-stage fine-tuning. A W B and W A ′、W B ′, specifically as follows:

[0044] W0+ΔW=W0+W A W B +W A 'W B '

[0045] Where W0 represents the original Mistral model parameters, ΔW represents the parameter update matrix, and W A and W B W represents the weight update matrix of the prior text dataset. A ′ and W B ′ represents the weight update matrix of the non-line-of-sight text dataset; W A and W A 'Use Gaussian initialization, W B and W B Use zero initialization.

[0046] The present invention has the following beneficial effects and advantages:

[0047] 1. This invention utilizes multi-stage fine-tuning with prior datasets and non-line-of-sight datasets, enabling the fine-tuned model to better understand and address the low ranging accuracy problem in this environment. Pre-training the model with prior datasets increases the model's relevant data and improves its adaptability to specific tasks. Simultaneously, deep fine-tuning with non-line-of-sight datasets further enhances its performance for specific tasks.

[0048] 2. This invention adjusts the importance of various features of samples based on the evaluation results of non-line-of-sight measurement datasets using the Bayesian optimization decision tree method, and uses a format converter to adjust the structured data and form a non-line-of-sight text dataset. This achieves the improvement of the stability and accuracy of the ranging model with a small amount of time and computational cost, and the dedicated model still maintains the performance of the original large model when facing other instructions. Attached Figure Description

[0049] Figure 1 This is a flowchart illustrating the principle of the method of the present invention;

[0050] Figure 2 This is a flowchart illustrating the method of the present invention;

[0051] Figure 3 This is a schematic diagram of a data acquisition scenario;

[0052] Figure 4 This is a flowchart of a Bayesian optimization decision tree model;

[0053] Figure 5 This is a schematic diagram of the optimization algorithm for non-line-of-sight datasets;

[0054] Figure 6 This is a diagram illustrating multi-stage fine-tuning;

[0055] Figure 7 This is a flowchart of the trainer iteration process. Detailed Implementation

[0056] The embodiments of the technical solution of the present invention will now be described in detail with reference to the accompanying drawings. These embodiments are merely illustrative of the technical solution of the present invention and are therefore intended to limit the scope of protection of the present invention.

[0057] The overall process of the distance measurement method in a non-line-of-sight environment based on Mistral model fine-tuning proposed in this invention is as follows: Figure 1 , Figure 2 As shown, the specific steps include:

[0058] 1) Construct a prior text dataset;

[0059] 2) Collect and generate a non-line-of-sight measurement dataset;

[0060] 3) Instructions are used to optimize the non-line-of-sight measurement dataset to form a non-line-of-sight text dataset;

[0061] 4) The Mistral model is pre-trained using a prior text dataset, and the Mistral model is trained for a dedicated ranging task using a non-line-of-sight text dataset.

[0062] 5) Utilize a trainer to iteratively fine-tune and generate a proprietary ranging model;

[0063] 6) Deploy the finely tuned proprietary ranging model and run it offline.

[0064] The construction of the prior text dataset in step 1) specifically involves: sorting out RSSI ranging knowledge, utilizing the results of decision tree feature importance indicators, and combining existing large language models' understanding of RSSI ranging, resulting in a dataset in the form of "question-answer pairs," which is then converted into the "prior text dataset" as shown below:

[0065] Dataset1={Instruction',Response'}

[0066] Here, Instruction′ represents the question description of the prior knowledge. Response′ represents the answer to the relevant question, which is exactly what the model needs to learn. An example of a prior text dataset is shown below:

[0067] “[

[0068] {

[0069] "Instruction′":"What is the principle behind RSSI ranging?"

[0070] "Response′":"RSSI ranging is a method for ranging based on Received Signal Strength Indication (RSSI)..."

[0071] A mathematical formula for distance estimation. It calculates the distance between an object or device and the transmitter by measuring the received signal strength and using attenuation models and empirical parameters. Specific ranging formulas can be adjusted and optimized according to different wireless technologies and application scenarios.

[0072] }

[0073] ]”

[0074] Step 2) involves constructing a non-line-of-sight measurement dataset. Specifically, in this embodiment, a ZigBee network is formed using three devices. The first device acts as a terminal, cyclically sending hardware IDs via unicast. The second device acts as a router, receiving the data sent by the terminal to obtain the propagated RSSI data, and then packaging and sending it out via multicast. The third device acts as a coordinator, receiving the data sent by the router. Data is collected in groups of 10, and the average value of the collected data is calculated and transmitted to a computer for recording. In the data acquisition software, a serial port debugging assistant is used to receive and record the RSSI data transmitted between the terminal and the router by the coordinator.

[0075] In this embodiment, the non-line-of-sight measurement dataset was collected in a relatively open space 4m wide and 7m long. Ten sets of sampling points were set at distances of 0.6m, 1.2m, 1.8m, 2m, 2.4m, 3.0m, 3.3m, 3.6m, 3.9m, and 4.2m between the router and the terminal. Each set of sampling points consisted of four different occlusion materials, three different occlusion distances, and three different occlusion ranges.

[0076] In this embodiment, to simulate a non-line-of-sight environment, two sizes of cardboard and one type of metal plate were used to block the propagation area of ​​the wireless signal. To differentiate the degree of blockage, based on the principle of the "first Fresnel zone," the Fresnel radius at different locations was calculated as follows:

[0077]

[0078] Among them, R n d1 represents the radius of the nth Fresnel zone, d2 represents the straight-line distance from the transmitter, d1 represents the straight-line distance from the receiver, both in meters (m), n represents the Fresnel zone number, and λ represents the frequency of wireless communication in a local area network.

[0079] In this embodiment, the occlusion area was divided into three different levels of occlusion: 60%, 80%, and 100%, and experiments were conducted separately for each level. Figure 3 As shown.

[0080] In this embodiment, sampling points at different distances are selected, and sample data corresponding to RSSI values ​​are collected at each sampling point. When the data fluctuation is small, each sampling point is sampled 30 times. The geometric mean of the 20 data points in the middle of the measurement and after Gaussian filtering is selected as the experimental data of that point. A total of 280 data points are obtained to form the collected samples. Each set of data includes six features: length, width, thickness, occlusion distance, occlusion range, and RSSI data of the occlusion material.

[0081] In this embodiment, Gaussian filtering is used to eliminate signal noise caused by measurement errors and random fluctuations in signal transmission, making the acquired RSSI data closer to the actual value and improving the accuracy of the sample data.

[0082] First, we define and construct the Gaussian kernel function, also known as the radial basis function, which is used to calculate the distance from any point in space to the center point. The Gaussian kernel function is a bell-shaped curve, with the highest point at the center and gradually decreasing towards the sides; its specific shape is determined by the standard deviation. Since RSSI values ​​are one-dimensional data, we define a one-dimensional Gaussian kernel function.

[0083] The formula for the Gaussian kernel function is:

[0084]

[0085] Where x represents the independent variable, and σ represents the standard deviation, controlling the width of the function. The larger the standard deviation, the shorter and wider the curve, and the less effective the filtering; conversely, the smaller the standard deviation, the thinner and taller the curve, and the smoother the filtered signal.

[0086] Then, the constructed Gaussian kernel function is applied to the measured RSSI data, and filtering is performed through convolution. The convolution operation involves multiplying the Gaussian kernel function by the measured RSSI data point by point and then adding the results. The formula is as follows:

[0087]

[0088] I(n) represents the filtered result of each group of RSSI samples at the same distance in step 2) at position n, where rssi m represents the input signal strength at the discrete-time index (m), M controls the interval length of the filtering operation, and f(nm) represents the Gaussian kernel function at the discrete-time index (nm).

[0089] By using a one-dimensional Gaussian filter, measurement noise caused by random fluctuations in signal transmission can be eliminated, enhancing the stability of RSSI data and improving subsequent ranging accuracy. Then, the collected data is used to calculate a geometric mean, reflecting the overall characteristics of the fluctuating data and making it closer to the true RSSI data. The geometric mean calculation formula is as follows:

[0090]

[0091] Where n represents the length of the filtered RSSI sequence, and G represents the calculated data.

[0092] In step 3), the instructions optimize the non-line-of-sight measurement dataset, as follows: Figure 4 , Figure 5 As shown, specifically:

[0093] Step 3.1) Divide the non-line-of-sight measurement dataset into a training dataset, a validation dataset, and a test dataset.

[0094] Step 3.2) Set the objective function of the Bayesian optimization to the mean squared error of the validation set.

[0095] Step 3.3) Given the feature extraction function of the decision tree model, it provides a basis for format conversion of structured data. First, using the set number of Bayesian optimization iterations and the updated hyperparameter combination, the decision tree model is constructed iteratively, and the objective function value of the current model is calculated. The minimum objective value is then found, and the optimal model hyperparameter combination is recorded.

[0096] Step 3.4) Construct the optimal decision tree model using the optimized hyperparameter combination, and use the model to determine the importance score of the feature by considering the number of feature splits and the information gain of the dataset before and after the splits.

[0097]

[0098] Among them, feature_importance i This indicates that in step 2), the importance of the i-th feature in the text dataset is measured; splits_n i `splits_T` represents the number of times feature `i` is used for splitting; `splits_T` represents the total number of times all features are used for splitting. `gain` i This represents the total information gain when using feature i for partitioning; gain_all represents the total information gain of all partitions.

[0099] Step 3.5) Sort the features of the non-line-of-sight measurement dataset according to their scores, indicating that the higher the score, the more important the feature is in the model.

[0100] Step 3.6) Based on the ranking of features in the non-line-of-sight measurement dataset, use a format converter to convert the structured data in the non-line-of-sight measurement dataset into text data. The converted format is shown below:

[0101] Dataset2={Instruction”,Input”,Response”}

[0102] Here, "Instruction" describes the ranging problem, referring to the commands or instructions received by the model to guide the LLM in performing specific tasks or operations. "Input" describes the ranging metrics and features; it refers to the data or information provided to the LLM for processing, analysis, or transformation. "Response" represents the true distance predicted by the Mistral using the aforementioned features, and the result or response generated by the LLM based on the given input. Appropriate format conversion transforms the non-line-of-sight dataset into a text dataset format suitable for large language models.

[0103] The dataset example is shown below:

[0104]

[0105] In steps 4) and 5), the multi-stage fine-tuning and trainer iterations are implemented using the SFTTrainer trainer provided by the TRL library and the LoRA loaded by the peft library. The improved multi-stage fine-tuner enables supervised fine-tuning training of the Mistral model.

[0106] In step 4), the Mistral model is preventively adjusted using a prior text dataset, specifically as follows:

[0107] Step 4.1) Based on LoRA fine-tuning technology, a multi-stage fine-tuning method is proposed. This includes two parts: fine-tuning of the prior text dataset and fine-tuning of the non-line-of-sight dataset.

[0108] While retaining the original model parameters, a parameter update matrix ΔW is constructed to store the updated parameter values ​​of the model after fine-tuning the corresponding dataset. To reduce the large amount of computation in the fine-tuning process, the update matrix is ​​approximately decomposed into two types of low-dimensional matrices for model fine-tuning, including the reduced-rank matrix W. A and W A ′, the raised-rank matrix W B and W B ′, its formula is as follows:

[0109] W0+ΔW=W0+W A W B +W A 'W B '

[0110] Where W0 represents the parameters of the pre-trained model. A and W B W represents the weight update matrix of the prior text dataset. A ′ and W B ' represents the weight update matrix of the measured text dataset. Where W... A and W A 'Use Gaussian initialization, WB and W B Use zero initialization.

[0111] Step 4.2) In this embodiment, the prior text dataset formed in step 1) is used to perform the first stage of fine-tuning on the Mistral model. The model selected is "mistral-7b-bnb-4bit", the rank of the updated weight matrix is ​​set to 8, the hyperparameter α of the updated weight matrix is ​​set to 16, the number of parallel processing processes is set to 2, the batch size is 5, the gradient accumulation step is 1, the warm-up step is 5, the iteration step is 20, the learning rate is 2e-4, and the weight decay is 0.01. This stage of fine-tuning is completed using LoRA fine-tuning technology, generating the first-stage fine-tuned model "model.generate", with weights represented as "W". 0+ W A W B ".

[0112] In this embodiment, step 4.3) uses the non-line-of-sight text dataset generated in step 3) to perform a second-stage fine-tuning of the model. The model selected is "model.generate", the rank of the updated weight matrix is ​​set to 16, the hyperparameter α of the updated weight matrix is ​​set to 16, the number of parallel processing processes is set to 2, the batch size is 5, the gradient accumulation steps are 4, the warm-up steps are 8, the number of iteration steps is 300, the learning rate is 2e-4, and the weight decay is 0.01. This stage of fine-tuning is completed using LoRA fine-tuning technology, generating a proprietary ranging model. The weights are represented as "ΔW = W". 0+ W A W B+ W A 'W B ′”.

[0113] Step 4.4) integrates the trainable low-rank matrix constructed in steps 4.2) and 4.3) into the Transformer layer to achieve parameter tuning. However, since the low-rank matrix only approximates the update matrix, it is multiplied with the low-rank matrix using a hyperparameter as a scaling factor to make it closer to the update matrix. After the above matrix transformation, the propagation process becomes: h t =h+α / γ(W) A W B +W A 'W B ')x

[0114] Among them, h t The output of the fine-tuned model represents the distance prediction, h represents the original model's prediction, x represents the measurement text dataset from step 2), α represents the hyperparameters, and γ represents the rank of the weight update matrix. A schematic diagram of the multi-stage fine-tuning process is shown below. Figure 6 As shown.

[0115] Step 5) The trainer iteration process is as follows: Figure 7 As shown, it includes the following steps:

[0116] Step 5.1) Set the negative log-likelihood function as the objective function for this fine-tuning, providing the prior text dataset prediction loss value for the original Mistral model. Set the hyperparameter epochs to m, indicating m iterations for this fine-tuning. Dataset1 is used to learn the distance measurement pattern, and Response′ is the true pattern; the pattern is learned by calculating the loss value. Dataset2 is the distance measurement sample data, and Response″ records the true distance. The model predicts distance by learning the patterns of each feature, compares it with the true distance, and corrects the patterns learned by the model.

[0117] Step 5.2) Update matrix W using parameters from the multi-stage fine-tuner. A W B The gradient of the current parameters is calculated using the loss value and then returned to the trainer.

[0118] Step 5.3) deploys the LARS optimizer and Consine scheduler to the SFTTrainer trainer to update parameters and iteratively fine-tune the model. The LARS optimizer updates model parameters by combining gradient direction and magnitude to minimize the loss function. This optimizer can adaptively adjust the step size and maintain sparsity, helping to accelerate model convergence and improve training efficiency. It keeps many parameters at or near zero, effectively reducing model storage and computational costs. The Consine scheduler is a learning rate scheduling strategy that adjusts the learning rate using the properties of the cosine function to provide a smooth and effective learning rate change strategy during the training of large language models, helping to optimize the model's training effect and convergence speed. After the above iterative training process, the first-stage fine-tuned model "model.generate" is finally formed.

[0119] Step 5.4) Based on the model fine-tuned in "model.generate", a proprietary model is trained on a non-line-of-sight text dataset using the same SFTTrainer trainer, LARS optimizer, and Consine scheduler. The number of iterations is set to n, where n > 15*m, to ensure its absolute position in the proprietary ranging model. The mean squared error function is set as the target loss function for the second iteration of fine-tuning, and minimizing the mean squared error is set as the optimization objective. The parameter update matrix W is then used with the multi-stage fine-tuner. A 'W B Implement the parameter update stage during the fine-tuning process.

[0120] Step 5.5) Update matrix W using the weights after two-stage fine-tuning.A W B and W A 'W B It is combined with the original model to form a proprietary distance measurement model.

[0121] Step 5.6) After the multi-stage fine-tuning is completed, save the LoRA model weights ΔW and merge them with the original Mistral model weights W0 to quantize them into a GGUF file.

[0122] Step 6) involves deploying a proprietary ranging model using the Ollama model framework and running it offline.

[0123] $ollamarun MistralNDR

[0124] Finally, for a given test scenario, multiple occlusion features of the scenario (occlusion material length, occlusion material width, occlusion material thickness, occlusion distance, occlusion range, RSSI value) are input into the distance measurement model constructed above to obtain the predicted distance for the indoor non-line-of-sight environment.

Claims

1. A distance measurement method for non-line-of-sight environments based on fine-tuning of the Mistral model, characterized in that, Includes the following steps: 1) Based on prior knowledge of non-line-of-sight environments and RSSI ranging, and combined with a large language model knowledge base, a prior text dataset is constructed. 2) For different occlusion characteristics, record the RSSI information transmitted in the network under non-line-of-sight environment and construct a non-line-of-sight measurement dataset. 3) The method of optimizing decision trees based on Bayesian methods is used to optimize the non-line-of-sight measurement dataset to form a non-line-of-sight text dataset; 4) Pre-train the Mistral model using a prior text dataset, and train the Mistral model for the ranging task using a non-line-of-sight text dataset; use the trainer to iteratively fine-tune and generate the ranging model. 5) For a given scene to be tested, input the occlusion features of the scene into the distance measurement model constructed above to obtain the predicted distance of the indoor non-line-of-sight environment.

2. The distance measurement method in a non-line-of-sight environment based on Mistral model fine-tuning according to claim 1, characterized in that, Step 1) specifically involves: utilizing expert knowledge and RSSI ranging knowledge, combined with training data on RSSI ranging knowledge representation from a large language model, summarizing it into a question-answer pair dataset to form a priori text dataset. Dataset1={Instruction',Response'} Here, Dataset1 represents the prior text dataset, Instruction′ represents the question description of prior knowledge as a sample, and Response′ represents the answer to the question as a knowledge label.

3. The distance measurement method in a non-line-of-sight environment based on Mistral model fine-tuning according to claim 1, characterized in that, Step 2) specifically refers to: A wireless local area network (WLAN) that builds a wireless ad hoc network includes a coordinator, routers, and end nodes. The terminal node, acting as a transmitter, communicates with the router via broadcast. The router, acting as a receiver, communicates with the coordinator via multicast. The average value of multiple received data points is calculated as a group and regarded as the RSSI value corresponding to the distance between the router and the terminal node. The coordinator is used to connect to the PC to collect information; In the physical environment, different degrees of occlusion are represented by Fresnel zones to indicate different non-line-of-sight environments; Under the combined influence of three different variables—obstruction, obstruction distance, and obstruction degree—the RSSI values ​​received by the PC are recorded to form a non-line-of-sight measurement dataset, including the following steps: The obstruction area between the router and the terminal node was divided into three different degrees of obstruction: a%, b%, and c%. The following experiment was conducted for each degree of obstruction: Select multiple sampling points at different distances between the terminal node and the obstruction; For each distance, the RSSI value is collected N times at the sampling point, and the geometric mean after Gaussian filtering is used as the experimental data for that sampling point. Finally, for each type of occlusion, experimental data from sampling points at different distances under different degrees of occlusion are used to construct a non-line-of-sight measurement dataset. Each set of experimental data includes multiple occlusion features: occlusion material length, occlusion material width, occlusion material thickness, occlusion distance, occlusion range, and RSSI value.

4. The distance measurement method in a non-line-of-sight environment based on Mistral model fine-tuning according to claim 1, characterized in that, Step 3) is as follows: (1) Define the objective function of Bayesian optimization as the mean square error function of the validation set, set the range of hyperparameters for Bayesian optimization, and divide the non-line-of-sight measurement dataset into a training set and a validation set; (2) Construct a decision tree model using hyperparameters optimized by Bayesian search; (3) Use the decision tree model to fit and optimize the training samples and labels on the non-line-of-sight measurement training set, and perform iterative iterations according to the set number of iterations; when the set number of iterations is reached, the optimal combination of hyperparameters is obtained. (4) Construct the optimal decision tree model using the optimal hyperparameter combination, and determine the importance score of the feature based on the number of times the feature is split and the information gain of the dataset before and after the split. (5) Sort the features of the non-line-of-sight measurement dataset according to their scores. The higher the score, the greater the importance evaluation value of the feature in the optimal decision tree model. (6) Sort by features and use a format converter to convert it into a non-line-of-sight text dataset with labels.

5. The distance measurement method in a non-line-of-sight environment based on Mistral model fine-tuning according to claim 4, characterized in that, The importance score of a feature is determined by combining the number of times the feature is split and the information gain of the dataset before and after the split, as follows: Among them, feature_importance i Indicates the importance of the i-th feature in the non-line-of-sight measurement dataset; splits_n i `splits_T` represents the number of times feature `i` is used for splitting; `splits_T` represents the total number of times all features are used for splitting; `gain`... i This represents the total information gain when using feature i for partitioning; gain_all represents the total information gain of all partitions.

6. The distance measurement method in a non-line-of-sight environment based on Mistral model fine-tuning according to claim 4, characterized in that, The format converter has the following specific format: Dataset2={Instruction”,Input”,Response”} in, "Instruction" represents the description of the ranging problem, "Input" represents the description of the occlusion feature data of the ranging problem, and "Instruction" and "Input" serve as samples; "Response" represents the true distance predicted by Mistral through occlusion features, serving as the true distance label.

7. The distance measurement method in a non-line-of-sight environment based on Mistral model fine-tuning according to claim 1, characterized in that, Step 4) is as follows: First, the iterative trainer uses the prior text dataset to make predictions through the Mistral model. The model output and the prior text dataset knowledge labels are then compared using the negative log-likelihood function to calculate the loss value of the predicted output. Secondly, the parameter update matrix W is utilized in the multi-stage fine-tuner. A and W B The gradient of the current parameters is calculated based on the loss value and parameter update matrix, and then returned to the trainer. Then, the optimizer and scheduler deployed in the trainer are used to update the parameters, and the model is iteratively fine-tuned to form the first-stage fine-tuned model. Based on the first-stage fine-tuned model, the same trainer, optimizer and scheduler were used to train the model on a non-line-of-sight text dataset. Set the mean squared error function as the loss function for the second iteration of fine-tuning, and set minimizing the mean squared error as the optimization objective value; Parameter update matrix W using a multi-stage fine-tuner A ′ and W B Implement parameter updates during the fine-tuning process to achieve the second-stage fine-tuning of the model; Finally, the matrix W is updated using the weights after two-stage fine-tuning. A W B and W A ′、W B This is combined with the original Mistral model to form a ranging model to predict the distance between the terminal node and the obstruction.

8. The distance measurement method in a non-line-of-sight environment based on Mistral model fine-tuning according to claim 7, characterized in that, The weight update matrix W is obtained by using two-stage fine-tuning. A W B and W A ′、W B ′, specifically as follows: W0+ΔW=W0+W A IN B +W A 'IN B ' Where W0 represents the original Mistral model parameters, ΔW represents the parameter update matrix, and W A and W B W represents the weight update matrix of the prior text dataset. A ′ and W B ′ represents the weight update matrix of the non-line-of-sight text dataset; W A and W A 'Use Gaussian initialization, W B and W B Use zero initialization.

Images