Perception positioning method and electronic equipment

Abstract

The invention provides a sensing and positioning method and electronic equipment, and relates to the technical field of positioning, the sensing and positioning method is applied to a wireless access end with a beamforming function started in a target scene, and the target scene comprises a to-be-positioned target object. And collecting beam forming information fed back to the access end by a fixed terminal connected with the wireless access end in the target scene. And inputting the beam forming information into the trained positioning model, and outputting the positioning position of the target object to be positioned through the positioning model. Through the trained positioning model, the positioning position of the target object can be accurately predicted. The data parameter quantity contained in the beam forming information is relatively low, but the information granularity is relatively fine, so that the real-time performance of position prediction and the sensing positioning precision can be considered at the same time. The data storage space and the transmission bandwidth are remarkably compressed, meanwhile, the burden of data processing and positioning position prediction is reduced, the expandability of close-range sensing positioning deployment is improved, and the method is particularly suitable for being deployed in equipment with limited resources.

Description

Technical Field

[0001] This application relates to the field of positioning technology, and in particular, to a perception positioning method and an electronic device. Background Art

[0002] The perception positioning of wireless signals can be applied to multiple fields such as communication green energy saving and health monitoring. The positioning efficiency and accuracy directly affect the integration experience between the physical world and the digital space. Satellite positioning technology can provide good accuracy outdoors, but it is difficult to guarantee the accuracy indoors. With the intelligent construction of society, the demand for generalized, low-cost, and accurate indoor positioning technology is becoming more and more prominent.

[0003] However, the deep model of the existing perception method based on machine learning has a large number of parameters, and both the inference computing power and storage requirements are relatively high, which is not conducive to popularization on ordinary devices with limited resources, resulting in limited development of short-range perception positioning technology. Summary of the Invention

[0004] In view of this, the purpose of this application is to propose a perception positioning method and an electronic device to solve the problem that the development of short-range perception positioning technology is limited due to the large number of parameters of the deep model.

[0005] Based on the above purpose, the first aspect of this application provides a perception positioning method, which is applied to a wireless access end with beamforming function enabled in a target scenario. The target scenario includes a target object to be positioned. The method includes: Collect beamforming information fed back by a fixed terminal connected to the wireless access end in the target scenario; Input the beamforming information into a trained positioning model, and output the positioning position of the target object to be positioned through the positioning model; Wherein, the positioning model is trained by using beamforming information collected in different scenarios.

[0006] Optionally, collecting beamforming information fed back by a fixed terminal connected to the wireless access end in the target scenario includes: Send a downlink data probe packet to the fixed terminal, so that after receiving the data probe packet, the fixed terminal performs channel estimation to obtain channel state information, decompresses and compresses the channel state information to obtain beamforming information, and feeds back the beamforming information to the wireless access end.

[0007] Optionally, inputting the beamforming information into a trained positioning model, and outputting the positioning position of the target object to be positioned through the positioning model includes: Input the beamforming information into a trained positioning model, and perform fusion processing on the beamforming information through the feature fusion network in the positioning model to extract a low-dimensional position feature vector; Input the low-dimensional position feature vector into the position probability regression network in the positioning model, and output the position probability distribution of the target object in the predefined grid cells of the target scene through the position probability regression network; Determine the positioning position of the target object according to the position probability distribution, and output the positioning position through the positioning model.

[0008] Optionally, the training method of the positioning model includes: Initialize the lightweight network to obtain an initial model; For each scene among multiple scenes, collect the beamforming information feedback by the fixed terminal as the first data set; Construct multiple combined data sets based on the first data set of each scene; Train the initial model through multiple combined data sets, and obtain the positioning model after the training is completed.

[0009] Optionally, for each scene among multiple scenes, collecting the beamforming information feedback by the fixed terminal as the first data set includes: For each scene among multiple scenes, divide the area corresponding to each scene into multiple grid cells, place the reference object in each grid cell respectively, and collect the beamforming information feedback by the fixed terminal. Take the beamforming information collected in each scene and the position of the grid cell where the corresponding reference object is located as the first data set.

[0010] Optionally, the constructing multiple combined data sets based on the first data set of each scene includes: Split and combine different first data sets to obtain multiple combined data sets.

[0011] Optionally, training the initial model through multiple combined data sets and obtaining the positioning model after the training is completed includes: Divide each combined data set into a first training set and a first test set; For each first training set, construct a copy model based on the initial model, and the copy model has the same model structure as the initial model; Perform multiple rounds of iterative training on the initial model, and the process of each round of iterative training is as follows: For each copy model, train the copy model through the corresponding first training set and first test set, and calculate the corresponding loss value; Update the model parameters of the initial model according to the loss values of all copy models; In response to determining that the initial model does not meet the preset convergence criterion on at least one first test set, enter the next round of iterative training; Upon determining that the initial model meets the preset convergence criterion on each first test set, exit the multi-round iterative training.

[0012] Optionally, for each replica model, train the replica model with the corresponding first training set and first test set, and calculate the corresponding loss value, including: For each replica model, train the replica model with the corresponding first training set to obtain an error loss, and update the model parameters of the replica model using the backpropagation algorithm based on the error loss; Test the updated replica model with the corresponding first test set, and calculate the corresponding loss value.

[0013] Optionally, before collecting the beamforming information fed back by the fixed terminal connected to the wireless access end in the target scenario, it includes: Divide the area corresponding to the target scenario into multiple grid cells, place the reference objects in each grid cell respectively, and collect the beamforming information fed back by the fixed terminal; Take the collected beamforming information and the grid cell position where the corresponding reference object is located as a sample data set; Fine-tune the positioning model using the sample data set.

[0014] Based on the same inventive concept, a second aspect of the present application further provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable by the processor. When the processor executes the computer program, the above-described method is implemented.

[0015] As can be seen from the above, the perception positioning method and electronic device provided by the present application, where the method is applied to a wireless access end with beamforming function enabled in a target scenario, and the target scenario includes a target object to be located. Collect the beamforming information fed back by the fixed terminal connected to the wireless access end in the target scenario, and the beamforming information contains the waveform information caused by the disturbance of the target object. Input the beamforming information into the trained positioning model, and output the positioning position of the target object to be located through the positioning model. Among them, the positioning model is trained with the beamforming information between the wireless access end and the fixed terminal collected in different scenarios. The trained positioning model can accurately predict the positioning position of the target object. The beamforming information is a lightweight data, with a low number of data parameters, but a fine information granularity, which can balance the real-time performance of position prediction and the perception positioning accuracy. Compared with the complete channel state information, it can significantly compress the data storage space and transmission bandwidth, while reducing the data processing and positioning position prediction burden, improving the scalability of short-range perception positioning deployment, especially suitable for deployment on resource-constrained devices. Description of the Drawings

[0016] To more clearly illustrate the technical solutions in the present application or related technologies, the following will briefly introduce the drawings required for use in the embodiments or related technology descriptions. Obviously, the drawings in the following description are only embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0017] Figure 1 It is a schematic flowchart of the perception positioning method according to an embodiment of the present application; Figure 2 It is a schematic diagram of the initial model training process according to an embodiment of the present application; Figure 3 It is a schematic structural diagram of the perception positioning device according to an embodiment of the present application; Figure 4 It is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present application. Detailed implementation manners

[0018] To make the objectives, technical solutions and advantages of the present application clearer and more understandable, the following further elaborates on the present application in detail with reference to specific embodiments and the accompanying drawings.

[0019] It should be noted that unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should have the ordinary meaning understood by those with ordinary skills in the field to which the present application belongs. The "first", "second" and similar terms used in the embodiments of the present application do not represent any order, quantity or importance, but are only used to distinguish different components. The terms such as "including" or "comprising" mean that the elements or objects appearing before this word cover the elements or objects listed after this word and their equivalents, without excluding other elements or objects. The terms such as "connected" or "linked" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The terms such as "upper", "lower", "left", "right" are only used to represent relative positional relationships, and when the absolute position of the object being described changes, the relative positional relationship may also change accordingly.

[0020] Classic long-distance detection perception positioning technology, which uses antenna arrays to detect the direction of arrival of signals and combines rigorous mathematical algorithm forms of signal estimation, has gradually matured. However, in close-range scenarios with complex environments and multipath reflections, conventional channel models are difficult to accurately express the diversity of wireless transmission, and rigorous mathematical analysis is difficult to achieve good performance. In recent years, with the popularization of hotspot coverage communications, using conventional WiFi communication signals for close-range perception positioning has become a feasible solution. By mining the CSI (Channel State Information) in the communication process of WiFi devices, intruding objects (such as human bodies, drones, etc.) are located. The main idea is to first collect the CSI time series of the target terminal in real time through the WiFi access point (AP) in a specific environment, and then use the data to train and fit the deep neural network, and finally achieve real-time positioning.

[0021] Wi-Fi technology, with its popularity, convenience and low privacy intrusion, combined with the perception of electromagnetic channel information generated during the communication process and the rapid response capability of deep learning, is expected to become a good solution for close-range perception and positioning, and will play an important role in human perception and low-altitude flying object detection. However, the existing machine learning-based perception methods have a large number of deep model parameters, and the inference computing power and storage requirements are high, which is not conducive to the promotion of ordinary equipment with limited resources. At the same time, the relevant technology only collects specific CSI data in a certain laboratory or simulation scenario, and trains and evaluates the model performance, but does not fully consider the diversity of environmental factors such as close-range environment layout, object material, and object distribution. When the surrounding environment changes, the wireless channel characteristics will drift significantly, resulting in an increase in the perception and positioning error of the model output, and there is a lack of effective and good solutions for the adaptation of new scenarios. In order to adapt to new scenarios, a large number of data sets are often required for time-consuming retraining, which poses a huge challenge to the real-time nature of perception reasoning and the flexibility of environmental adaptation. Therefore, how to compress the model while maintaining high accuracy and quickly adapt to variable scenarios is a key issue that needs to be solved in close-range positioning technology.

[0022] Aiming at the two major problems existing in the existing indoor and other short-range positioning technologies based on deep learning: (1) the prediction positioning calculation amount and the storage hardware overhead are large, and it is difficult to be deployed in real time on ordinary devices with limited resources; (2) the data analysis method lacks the ability to quickly adapt to environmental changes, and the accuracy often drops significantly in new scenarios. In view of this, this application proposes a perception positioning method. During the short-range positioning process, CSI is replaced by beam-forming information (BFI). BFI is mainly used in the WiFi system to adjust the direction of the beam energy of the communication signal. Applying it to perception positioning, the number of characteristic parameters is lower and the information granularity is finer, which can balance the inference real-time performance and the perception positioning accuracy. For example, under the condition of 4×4 MIMO (multiple-in multiple-out) in WiFi, the number of data parameters of a single BFI is 256×12, and the number of data parameters of CSI is 256×16×2. The former is only 37.5% of the latter's parameter amount. In deep learning tasks, when the number of input data parameters decreases, the corresponding model inference amount can also decrease accordingly. Replacing CSI with BFI can significantly reduce the data processing and inference burden, and through the domain enhancement meta-learning architecture and the online fine-tuning mechanism, use a small number of labeled samples to correct the channel drift in real time, which not only ensures the positioning accuracy but also improves the scalability of perception deployment.

[0023] The embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0024] This application proposes a perception positioning method, referring to Figure 1 , which is applied to a wireless access end with beamforming function enabled in a target scenario. The target scenario includes a target object to be located. The method includes the following steps: Step 102: Collect the beamforming information fed back by a fixed terminal connected to the wireless access end in the target scenario.

[0025] Specifically, the target scenario is a scenario covered by a wireless network. Exemplarily, the target scenario can be an office scenario, an outdoor rooftop scenario, a basement, or a low-altitude scenario, etc. The wireless access end is a wireless router (WiFi device) with beamforming function enabled, and the fixed terminal is connected to this wireless router. The fixed terminal can be another or multiple fixed-position wireless routers, or a fixed-position mobile terminal (such as a mobile phone, a computer, etc.), as long as it has signal transceiver function. According to the WiFi protocol process, the wireless access end can extract the beamforming information.

[0026] Step 104: Input the beamforming information into a trained positioning model, and output the positioning position of the target object to be located through the positioning model. Among them, the positioning model is trained by the beamforming information collected in different scenarios.

[0027] Specifically, the positioning model is a trained machine learning model, which can specifically be a deep neural network model. Exemplarily, the positioning model is a meta-learning agent. The deep neural network model is trained with beamforming information collected in different scenarios, and the positioning model is obtained after the training is completed. The positioning model can accurately predict the position of the target object in the target scenario. The target object can be a human body or other objects. When the target object is in different positions in the target scenario, the collected beamforming information is different, and the target object disturbs the beamforming information. Therefore, based on the beamforming information for real-time inference and prediction, the positioning position of the target object can be determined.

[0028] In addition, since the influences of the human body and objects on CSI are different, if predicting the positioning position of the human body in the target scenario, when training the positioning model, it is necessary to collect training data by means of the position transformation of the human body, and the reference object used in the training or fine-tuning process is the human body. If predicting the positioning position of an object in the target scenario, when training the positioning model, it is necessary to collect training data by means of the position transformation of the object, and the reference object used in the training or fine-tuning process is the object. It should be noted that due to the different materials of objects, there are also differences in the influence on CSI. Therefore, the positioning model trained with the training data collected from objects of a certain type of material can only accurately locate objects of the same type of material.

[0029] After replacing CSI with BFI, only beamforming information needs to be used for prediction during positioning. There are fewer parameters, avoiding the high-precision measurement and storage of the original amplitude / phase data, and significantly reducing the data parameter amount and storage overhead. Under the condition of the same model parameters, the inference amount is greatly reduced, reducing the real-time inference calculation overhead of edge devices.

[0030] Based on the above steps 102 to 104, the perception positioning method provided in this embodiment is applied to a wireless access end with beamforming function enabled in a target scenario, and the target scenario includes a target object to be located. The beamforming information fed back by a fixed terminal connected to the wireless access end in the target scenario is collected, and the beamforming information contains waveform information caused by the disturbance of the target object. The beamforming information is input into a trained positioning model, and the positioning position of the target object to be located is output through the positioning model. Among them, the positioning model is trained by collecting beamforming information in different scenarios. The positioning position of the target object can be accurately predicted through the trained positioning model. The beamforming information is a kind of lightweight data, with a low number of data parameters, but a finer information granularity, which can balance the real-time performance of position prediction and the perception positioning accuracy. It significantly compresses the data storage space and transmission bandwidth, reduces the data processing and positioning position prediction burden at the same time, improves the scalability of short-range perception positioning deployment, and is especially suitable for deployment on resource-constrained devices.

[0031] In some embodiments, collecting the beamforming information of a fixed terminal connected to the wireless access end in the target scenario includes: Sending a downlink data detection packet to the fixed terminal, so that after receiving the data detection packet, the fixed terminal performs channel estimation to obtain channel state information, decompresses and compresses the channel state information to obtain beamforming information, and feeds back the beamforming information to the wireless access end.

[0032] Specifically, after the wireless access terminal enables the beamforming function, it can periodically perform downlink detection on the fixed terminal connected to it; in terms of the placement position, the wireless access terminal and the fixed terminal are arranged separately, and the positioning of the area between the two devices is more accurate. Specifically, a data detection packet is sent to the fixed terminal. Exemplarily, the data detection packet is an NDP detection packet. An NDP detection packet refers to an information packet used to detect and discover other devices in the network in the Neighbor Discovery Protocol (NDP). The wireless access device sends a pilot signal known to both the transmitting and receiving ends by sending an NDP detection packet. After receiving the NDP detection packet, the fixed terminal compares the received pilot signal with the original information and calculates the channel state information of the channel. Then, according to the content specified in the standard protocol IEEE Std 802.11 Part 11: Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) Specifications. Section 19.3.12.3.6, Compressed beamforming feedback matrix. Page: 2398-2400., the channel state information is subjected to singular value decomposition, and the first spatial stream columns of the Zk matrix are taken as the precoding matrix Vk, and it is compressed according to the protocol process to obtain BFI data with relatively fewer parameters and represented by angle scalars. After obtaining the BFI data, the fixed terminal packs and feeds back the beamforming information to the wireless access terminal. After the wireless access terminal reads the corresponding field in the data packet, it can obtain the compressed BFI data. Since the BFI data has a smaller data volume than the SCI data, the subsequent process of using the BFI data for machine learning will be faster. The downlink detection and BFI feedback process runs periodically in the wireless router with beamforming enabled, and a BFI data source that can be used for model training and subsequent model inference can be obtained.

[0033] In some embodiments, inputting the beamforming information into a trained positioning model, and outputting the positioning position of the target object to be positioned by the positioning model, includes: Inputting the beamforming information into a trained positioning model, and performing fusion processing on the beamforming information through a feature fusion network in the positioning model to extract a low-dimensional position feature vector; Inputting the low-dimensional position feature vector into the position probability regression network in the positioning model, and outputting the position probability distribution of the target object in a predefined grid cell in the target scene through the position probability regression network; Determining the positioning position of the target object according to the position probability distribution, and outputting the positioning position through the positioning model.

[0034] Specifically, the positioning model includes a feature fusion network and a position probability regression network. The feature fusion network is used to extract low-dimensional position feature vectors from beamforming information. The position probability regression network can calculate the position probability of the target object in each predefined grid cell. After receiving the beamforming information, the positioning model parallelly inputs the beamforming information into the feature fusion network, and the feature fusion network performs fusion processing on multiplexed spatio-temporal-frequency information to extract low-dimensional position feature vectors. Then, the low-dimensional position feature vectors are fed into the subsequent position probability regression network, and the position probability regression network outputs the position probability distribution of the target object. The position probability distribution contains multiple position probabilities, each corresponding to a grid cell, and the position probability represents the probability of the target object appearing in the grid cell. The target scene is pre-divided into multiple grid cells. Finally, according to the position probability distribution, strategies such as maximum probability or weighted average are adopted to determine the positioning position of the target object. The maximum probability strategy refers to determining the position of the grid cell corresponding to the maximum probability in the position probability distribution as the positioning position. The weighted average strategy refers to performing weighted averaging on each position probability to obtain the predicted value of the position expectation as the positioning position. After determining the positioning position, the positioning position is output through the positioning model. The wireless access end can also send the positioning position to the upper-layer application or user terminal through an interface to realize visualization of the positioning position of the target object and subsequent service invocation.

[0035] In some embodiments, the training method of the positioning model includes: Initializing the lightweight network to obtain an initial model; For each of multiple scenarios, collecting the beamforming information fed back by the fixed terminal as the first data set; Constructing multiple combined data sets based on the first data set of each scenario; Training the initial model through multiple combined data sets, and obtaining the positioning model after the training is completed.

[0036] Exemplarily, the lightweight network includes a convolutional network, a linear layer network, etc. During the initialization process, random initialization methods are adopted for the weights and biases of each layer of the lightweight network to construct the initial model. The multiple scenarios can include an office scenario, an outdoor rooftop scenario, or a low-altitude scenario, etc.

[0037] For each scenario, collecting the beamforming information fed back by the fixed terminal as the first data set. Specifically, it includes: For each of multiple scenarios, dividing the area corresponding to each scenario into multiple grid cells, placing the reference objects in each grid cell respectively, and collecting the beamforming information fed back by the fixed terminal. The beamforming information collected in each scenario and the position of the grid cell where the corresponding reference object is located are used as the first data set.

[0038] Exemplarily, the area where each scenario is located is divided into m discrete position points, and one discrete position point corresponds to one grid cell. When collecting beamforming information, the reference object collects beamforming information once in each grid cell, and the number of finally collected beamforming information is equal to the number of grid cells. The coordinates of the discrete position points within the grid cell are the position tags corresponding to the beamforming information. In each scenario, after the beamforming information corresponding to all grid cells is collected, all the beamforming information and the corresponding position tags are used as the first data set. Among them, the reference object can be a human body or an object.

[0039] Based on the first data set of each scenario, multiple combined data sets are constructed, specifically including: splitting and combining different first data sets to obtain multiple combined data sets.

[0040] For each scenario, based on dimensions such as spatial position, time, or object position, the first data set is divided into several subsets. Then, the subsets in different scenarios are combined across scenarios to obtain multiple combined data sets. Exemplarily, two scenarios are scenario one and scenario two. The first data set of scenario one is divided into three subsets {A, B, C}, and the first data set of scenario two is divided into three subsets {a, b, c}. To obtain the common generalization features of different scenarios, the subsets of the two scenarios are combined, and several combined data sets such as {A, a}, {B, b}, {C, c} can be obtained to achieve domain-enhanced data construction. Each combined data set corresponds to a combined scenario. By training the initial model with the combined data sets, the robustness of the model to changes in channel distribution can be enhanced. During the process of the model learning based on the combined data sets, the model learns the common generalization features of different scenarios and enhances the applicability of the model.

[0041] Further, the initial model is trained with multiple combined data sets, and after the training is completed, a positioning model is obtained, including: Each combined data set is divided into a first training set and a first test set; For each first training set, a copy model is constructed based on the initial model, and the model structure of the copy model is the same as that of the initial model; The initial model is trained through multiple rounds of iterative training, and the process of each round of iterative training is as follows: For each copy model, the copy model is trained with the corresponding first training set and first test set, and the corresponding loss value is calculated; The model parameters of the initial model are updated according to the loss values of all copy models; In response to determining that the initial model does not meet the preset convergence criterion on at least one first test set, enter the next round of iterative training; Exit the multi-round iterative training in response to determining that the initial model meets the preset convergence criteria on each first test set.

[0042] Specifically, divide each combined data set into a first training set and a first test set. Exemplarily, if the number of data in the combined data set is 100, 70 of them can be used as the first training set, and the remaining 30 as the first test set. Each data in the combined data set includes the position label of a position point and the beamforming information collected by the wireless access terminal when the reference object is at this position point, that is, the position label-beamforming information pair.

[0043] For each first training set, that is, for each combined scenario, construct a copy model, and the model structure of the copy model is the same as that of the initial model. Perform multi-round iterative training on the initial model, including: for each copy model, use the first test set and the first training set to train the copy model to obtain a loss value. Update the model parameters of the initial model by integrating the loss values of all copy models. If the loss value obtained when the updated initial model is tested on at least one first test set shows a significant decrease compared with the previous time, it is determined that the preset convergence criteria are not met, and the initial model needs to be subjected to the next round of iterative training. If the loss value obtained when the updated initial model is tested on all first test sets no longer decreases or the decrease rate is extremely low, it is determined that the preset convergence criteria are met. At this time, the initial model completes the training process, and after fixing the model parameters, a positioning model is obtained. By combining data, the differences between scenario domains can be amplified, and a positioning model with high transfer ability to unknown environments can be obtained.

[0044] Figure 2 Shows a schematic diagram of the initial model training process. Multiple combined data sets include combined data set 1, combined data set 2,..., combined data set n. Input each combined data set into the corresponding copy model respectively, and the feature distributions of different combined data sets may be different. The initial model can be used as a meta-model or a meta-model agent. Output the corresponding loss value through each copy model. Specifically, copy model 1 outputs loss value 1, copy model 2 outputs loss value 2,..., copy model n outputs loss value n. Determine the comprehensive deviation loss according to all loss values. Exemplarily, the comprehensive deviation loss can be the average loss value. Input the comprehensive deviation loss into the initial model, and let the initial model update and optimize the model parameters through gradient learning. After the model parameters of the initial model are updated, copy the model parameters of the initial model to each copy model to make the model parameters of the copy model consistent with those of the initial model, and then perform the next round of iterative learning until the preset convergence condition is met. At this time, the initial model is the positioning model. Multiple combined data sets are learned in parallel in different copy models, and the learning efficiency is higher.

[0045] Further, for each replica model, the replica model is trained through the corresponding first training set and first test set, and the corresponding loss value is calculated, including: For each replica model, the replica model is trained through the corresponding first training set to obtain an error loss, and the model parameters of the replica model are updated using the backpropagation algorithm based on the error loss; The updated replica model is tested through the corresponding first test set, and the corresponding loss value is calculated.

[0046] Specifically, for each replica model, first, the replica model is trained through the first training set. The data in the first training set is input into the replica model, and the predicted position is output through the replica model. The error loss is calculated based on the error between the predicted position and the position label, and the weights and biases of the replica model are updated using the backpropagation algorithm based on the error loss. Then, the updated replica model is tested on the first test set, and the loss value during the prediction process is calculated. The data in the training set and the test set are different, which can better evaluate the learning effect of the model.

[0047] In the related art, in the face of the diversification and dynamic changes of the indoor environment, the model trained in a single scenario often has difficulty in ensuring the stability of cross-scenario performance. In this application, the positioning model is adjusted by the following method to ensure the stability of the cross-scenario performance of the model.

[0048] In some embodiments, before collecting the beamforming information fed back by the fixed terminal connected to the wireless access end in the target scenario, it includes: The area corresponding to the target scenario is divided into multiple grid cells, reference objects are placed in each grid cell respectively, and the beamforming information fed back by the fixed terminal is collected; The collected beamforming information and the grid cell position where the corresponding reference object is located are used as a sample data set; The sample data set is used to fine-tune the positioning model.

[0049] Specifically, the trained positioning model in the foregoing embodiment is downloaded and loaded to the local deployment device (such as a smart terminal, an edge computing node, etc.) from the remote end or the central server as the basic model for subsequent fine-tuning. Before applying the positioning model to the target scenario, the positioning model is fine-tuned so that the positioning model can quickly adapt to the target scenario. Specifically, it includes two parts: sample data collection and local fast adaptation fine-tuning of the positioning model.

[0050] In the target scenario to be deployed, several representative position points are selected manually or automatically to form several grid cells, and each grid cell contains a position point. The reference object is placed at each position point, and for each position point, a number of beamforming information and the precise position label of the position point are collected as a sample data set. During the collection process, the main occlusion, reflection, and multipath distribution areas in the target scenario should be covered to ensure the representativeness of the sample data. Only a small batch of sample data is required to complete the fine-tuning of the positioning model, significantly reducing the time and labor costs of large-scale data collection.

[0051] After that, the sample data set is used to perform a small-scale gradient update on the positioning model to achieve fine-tuning of the positioning model. After the fine-tuning is completed, the updated model parameters are fixed to form a localized positioning model applicable to the current target scenario, and the fine-tuned positioning model is deployed to an edge device or an embedded platform. If the target scenario changes significantly, the above process can be automatically repeated for fine-tuning, achieving continuous self-adaptation of the model. Through the fine-tuned positioning model, efficient and accurate online positioning of the target object can be achieved.

[0052] When the positioning model is deployed in a new scenario, only a very small amount of sampling and rapid fine-tuning are required to complete the smooth transition from a general meta-model to a dedicated model, significantly improving the scene robustness and "plug-and-play" deployment efficiency of the system.

[0053] It should be noted that the method of the embodiment of the present application can be executed by a single device, such as a computer or a server. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In this case of a distributed scenario, one of the multiple devices can only execute one or more steps of the method of the embodiment of the present application, and these multiple devices will interact with each other to complete the described method.

[0054] It should be noted that some embodiments of the present application have been described above. Other embodiments are within the scope of the specification. In some cases, the actions or steps recorded in the specification can be executed in a different order from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0055] Based on the same inventive concept, corresponding to the method of any of the above embodiments, the present application also provides a sensing and positioning device.

[0056] Refer to Figure 3, the perception and positioning device is applied to a wireless access end with beamforming function enabled in a target scenario. The target scenario contains a target object to be positioned. The perception and positioning device includes: An acquisition module 302, configured to acquire beamforming information fed back by a fixed terminal connected to the wireless access end in the target scenario; A positioning module 304, configured to input the beamforming information into a trained positioning model, and output the positioning position of the target object to be positioned through the positioning model; Wherein, the positioning model is obtained by training with beamforming information collected in different scenarios.

[0057] In some embodiments, the acquisition module 302 further includes: A detection unit, configured to send a downlink data detection packet to the fixed terminal, so that after receiving the data detection packet, the fixed terminal performs channel estimation to obtain channel state information, decompresses and compresses the channel state information to obtain beamforming information, and feeds back the beamforming information to the wireless access end.

[0058] In some embodiments, the positioning module 304 includes: A feature extraction unit, configured to input the beamforming information into a trained positioning model, perform fusion processing on the beamforming information through a feature fusion network in the positioning model, and extract a low-dimensional position feature vector; A probability regression unit, configured to input the low-dimensional position feature vector into a position probability regression network in the positioning model, and output the position probability distribution of the target object in a predefined grid cell in the target scenario through the position probability regression network; A position determination unit, configured to determine the positioning position of the target object according to the position probability distribution, and output the positioning position through the positioning model.

[0059] In some embodiments, it further includes a training module, configured to: Initialize the lightweight network to obtain an initial model; For each of multiple scenarios, acquire beamforming information fed back by the fixed terminal as a first data set; Construct multiple combined data sets based on the first data set of each scenario; Train the initial model through multiple combined data sets, and obtain the positioning model after training is completed.

[0060] In some embodiments, the training module includes: The first data set acquisition unit is configured to, for each of multiple scenarios, divide the area corresponding to each scenario into multiple grid cells, place reference objects in each grid cell respectively, and acquire the beamforming information fed back by a fixed terminal, and use the beamforming information acquired in each scenario and the position of the grid cell where the corresponding reference object is located as the first data set.

[0061] In some embodiments, the training module includes: The combined data set acquisition unit splits and combines different first data sets to obtain multiple combined data sets.

[0062] In some embodiments, it further includes a training module configured to: The iterative training unit is configured to divide each combined data set into a first training set and a first test set; For each first training set, a copy model is constructed based on the initial model, and the copy model has the same model structure as the initial model; Perform multiple rounds of iterative training on the initial model, and the process of each round of iterative training is as follows: For each copy model, train the copy model through the corresponding first training set and first test set, and calculate the corresponding loss value; update the model parameters of the initial model according to the loss values of all copy models; in response to determining that the initial model does not meet the preset convergence criterion on at least one first test set, enter the next round of iterative training; in response to determining that the initial model meets the preset convergence criterion on each first test set, exit the multiple rounds of iterative training.

[0063] In some embodiments, the training module includes: The loss value calculation unit is configured to, for each copy model, train the copy model through the corresponding first training set to obtain an error loss, update the model parameters of the copy model based on the error loss using the backpropagation algorithm; test the updated copy model through the corresponding first test set, and calculate the corresponding loss value.

[0064] In some embodiments, before acquiring the beamforming information fed back by the fixed terminal connected to the wireless access end in the target scenario, it further includes a fine-tuning module configured to: Divide the area corresponding to the target scenario into multiple grid cells, place reference objects in each grid cell respectively, and acquire the beamforming information fed back by the fixed terminal; use the acquired beamforming information and the position of the grid cell where the corresponding reference object is located as a sample data set; fine-tune the positioning model using the sample data set.

[0065] For the convenience of description, when describing the above device, various modules are described separately according to their functions. Of course, when implementing the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0066] The device in the above embodiment is used to implement the corresponding perception and positioning method in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be elaborated here.

[0067] Based on the same inventive concept, corresponding to the method in any of the above embodiments, the present application further provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the perception and positioning method described in any of the above embodiments.

[0068] Figure 4 FIG. shows a more specific schematic diagram of the hardware structure of the electronic device provided in this embodiment. The device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. Among them, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are communicatively connected to each other inside the device through the bus 1050.

[0069] The processor 1010 can be implemented in the form of a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, etc., and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0070] The memory 1020 can be implemented in the form of a ROM (Read Only Memory), a RAM (Random Access Memory), a static storage device, a dynamic storage device, etc. The memory 1020 can store an operating system and other application programs. When implementing the technical solutions provided in the embodiments of this specification through software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0071] The input / output interface 1030 is used to connect to the input / output module to achieve information input and output. The input / output module can be configured as a component in the device (not shown in the figure), or can be externally connected to the device to provide corresponding functions. Among them, the input devices can include keyboards, mice, touchscreens, microphones, various sensors, etc., and the output devices can include displays, speakers, vibrators, indicator lights, etc.

[0072] The communication interface 1040 is used to connect to the communication module (not shown in the figure) to achieve communication interaction between this device and other devices. Among them, the communication module can achieve communication through wired means (such as USB, network cable, etc.), or can also achieve communication through wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0073] The bus 1050 includes a passage for transmitting information between various components of the device (such as the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).

[0074] It should be noted that although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040, and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, those skilled in the art can understand that the above device may also only include the components necessary to implement the solution of the embodiments of this specification, and do not have to include all the components shown in the figure.

[0075] The electronic device of the above embodiment is used to implement the corresponding perception and positioning method in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be elaborated here.

[0076] Based on the same inventive concept, corresponding to the method of any of the above embodiments, the present application also provides a non-transitory computer-readable storage medium. The non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to make the computer execute the perception and positioning method described in any of the above embodiments.

[0077] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tapes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0078] The computer instructions stored in the storage medium of the above embodiment are used to cause the computer to execute the perception positioning method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be elaborated here.

[0079] Those of ordinary skill in the art should understand that: the discussion of any of the above embodiments is only exemplary and is not intended to imply that the scope of the present application is limited to these examples; under the idea of the present application, the technical features in the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations in different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of brevity.

[0080] In addition, for simplicity of description and discussion, and in order not to make the embodiments of the present application difficult to understand, the well-known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided drawings. In addition, the device may be shown in block diagram form to avoid making the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform on which the embodiments of the present application are to be implemented (i.e., these details should be completely within the understanding of those skilled in the art). In the case where specific details (such as circuits) are set forth to describe the exemplary embodiments of the present application, it will be apparent to those skilled in the art that the embodiments of the present application can be implemented without these specific details or with variations of these specific details. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0081] Although the present application has been described in connection with specific embodiments thereof, many alternatives, modifications, and variations of these embodiments will be apparent to those of ordinary skill in the art in light of the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0082] Embodiments of the present application are intended to cover all such alternatives, modifications, and variations that fall within the broad scope of the specification. Accordingly, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the embodiments of the present application shall be included within the protection scope of the present application.

Claims

1. A perception and positioning method, characterized in that, Applied to a wireless access end with beamforming function enabled in a target scenario, where the target scenario includes a target object to be located, the method includes: Collect beamforming information fed back by a fixed terminal connected to the wireless access end in the target scenario; Input the beamforming information into a trained positioning model, and output the positioning location of the target object to be located through the positioning model; Among them, the positioning model is trained through beamforming information collected in different scenarios.

2. The method according to claim 1, wherein The collecting beamforming information fed back by a fixed terminal connected to the wireless access end in the target scenario includes: Send a downlink data probe packet to the fixed terminal, so that after the fixed terminal receives the data probe packet, it performs channel estimation to obtain channel state information, decompresses and compresses the channel state information to obtain beamforming information, and feeds back the beamforming information to the wireless access end.

3. The method according to claim 1, characterized in that, The inputting the beamforming information into a trained positioning model and outputting the positioning location of the target object to be located through the positioning model includes: Input the beamforming information into a trained positioning model, and perform fusion processing on the beamforming information through the feature fusion network in the positioning model to extract a low-dimensional position feature vector; Input the low-dimensional position feature vector into the position probability regression network in the positioning model, and output the position probability distribution of the target object in the predefined grid cells in the target scenario through the position probability regression network; Determine the positioning location of the target object according to the position probability distribution, and output the positioning location through the positioning model.

4. The method according to claim 1, wherein The training method of the positioning model includes: Initialize the lightweight network to obtain an initial model; For each of multiple scenarios, collect beamforming information fed back by a fixed terminal as a first data set; Construct multiple combined data sets based on the first data set of each scenario; Train the initial model through multiple combined data sets, and obtain the positioning model after training is completed.

5. The method according to claim 4, wherein The for each of multiple scenarios, collect beamforming information fed back by a fixed terminal as a first data set includes: For each of multiple scenarios, divide the area corresponding to each scenario into multiple grid cells, place reference objects in each grid cell respectively, and collect beamforming information fed back by a fixed terminal. Take the beamforming information collected in each scenario and the position of the corresponding grid cell where the reference object is located as the first data set.

6. The method according to claim 4, characterized in that The constructing multiple combined data sets based on the first data set of each scenario includes: Split and combine different first data sets to obtain multiple combined data sets.

7. The method according to claim 4, wherein The training the initial model through multiple combined data sets and obtaining the positioning model after training is completed includes: Divide each combined data set into a first training set and a first test set; For each first training set, construct a copy model based on the initial model, and the copy model has the same model structure as the initial model; Perform multiple rounds of iterative training on the initial model, and the process of each round of iterative training is as follows: For each copy model, train the copy model through the corresponding first training set and first test set, and calculate the corresponding loss value; Update the model parameters of the initial model according to the loss values of all replica models; In response to determining that the initial model does not meet the preset convergence criterion on at least one first test set, enter the next round of iterative training; In response to determining that the initial model meets the preset convergence criterion on each first test set, exit the multi-round iterative training.

8. The method according to claim 7, characterized in that For each replica model, train the replica model through the corresponding first training set and first test set, and calculate the corresponding loss value, including: For each replica model, train the replica model through the corresponding first training set to obtain an error loss, and update the model parameters of the replica model using the backpropagation algorithm based on the error loss; Test the updated replica model through the corresponding first test set, and calculate the corresponding loss value.

9. The method according to claim 1, wherein Before collecting the beamforming information fed back by the fixed terminal connected to the wireless access end in the target scenario, including: Divide the area corresponding to the target scenario into multiple grid cells, place the reference objects in each grid cell respectively, and collect the beamforming information fed back by the fixed terminal; Use the collected beamforming information and the grid cell position where the corresponding reference object is located as a sample data set; Fine-tune the positioning model using the sample data set.

10. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method according to any one of claims 1 to 9.

Images