On-demand labeling for channel classification learning

Abstract

An exemplary embodiment of the present disclosure relates to on-demand labeling for channel classification learning. A first device uses a classification model to determine a classification result for a communication channel based at least in part on channel measurement information for the communication channel, and determines importance evaluation information indicating an importance of the channel measurement information in updating the classification model based at least in part on a type of the classification model. The first device transmits the importance evaluation information to a second device. In response to determining that the importance of the channel measurement information exceeds an importance threshold, the second device causes a third device to perform classification labeling for at least the communication channel for a location associated with the first device.

Description

[Technical Field]

[0001] Embodiments of this disclosure generally relate to the field of telecommunications, and more particularly to methods, apparatus, devices, and computer-readable storage media for on-demand labeling for channel classification learning. [Background technology]

[0002] Location recognition is a fundamental aspect of wireless communication networks, enabling countless location-based services in various applications. The integration and utilization of location information in everyday applications will grow significantly as the accuracy of the technology advances.

[0003] Many positioning techniques that rely on technologies such as time of arrival (TOA), time difference of arrival (TDOA), and angle of arrival (AOA) require line-of-sight (LOS) propagation between a reference point (such as network equipment) and the mobile device being positioned. However, in the case of non-line-of-sight (NLOS) propagation both indoors and outdoors, positioning accuracy deteriorates significantly because multiple reflections of radio frequency (RF) propagation from diverse angles of arrival and various delay spreads cannot be identified. On the other hand, artificial intelligence (AI) algorithms are inherently superior in terms of accuracy and efficiency in fingerprint-styled position estimation, regardless of LOS / NLOS. Therefore, classifying channel propagation is important because it at least influences the choice of positioning approach. [Overview of the Initiative]

[0004] The scope of protection required for the various embodiments of the present invention is defined by the independent claims. Embodiments / examples and features described herein that do not fall within the scope of the independent claims are to be interpreted as exemplary embodiments useful for understanding the various embodiments of the present invention. It should be noted that the terms “embodiment” or “example” should be appropriately adapted to the terminology used in this application; that is, where the term “embodiment” is used, “embodiment” is described herein, and where the term “example” is used, “example” is described herein.

[0005] Where any embodiments are not covered by the claims, they should be interpreted as illustrative examples useful for understanding the various embodiments of this disclosure.

[0006] In a first embodiment, a first device is provided. The first device includes at least one processor and at least one memory that stores instructions that, when executed by the at least one processor, cause the first device to perform at least: determine a classification result of a communication channel using a classification model, at least based on channel measurement information relating to the communication channel; determine importance assessment information indicating the importance of the channel measurement information when updating the classification model, at least based on the type of classification model; and transmit the importance assessment information to a second device.

[0007] In a second embodiment, a second device is provided. The second device includes at least one processor and at least one memory that stores instructions for causing the second device to perform, when executed by the at least one processor, importance assessment information from the first device indicating the importance of channel measurement information when updating a classification model, the classification model being used to determine the classification result of a communication channel based on the channel measurement information, to receive, determine whether the importance of the channel measurement information exceeds an importance threshold, and, in accordance with the determination that the importance of the channel measurement information exceeds an importance threshold, to cause the third device to perform classification labeling of at least the communication channel at a location related to the first device.

[0008] In a third embodiment, a method is provided. This method includes: determining a classification result for a communication channel using a classification model in a first device, at least in part on channel measurement information relating to the communication channel; determining importance assessment information to indicate the importance of the channel measurement information when updating the classification model, at least in part on the type of classification model; and transmitting the importance assessment information to a second device.

[0009] In a fourth embodiment, a method is provided. This method includes: receiving, determining whether the importance of the channel measurement information exceeds an importance threshold, and, in accordance with the determination that the importance of the channel measurement information exceeds an importance threshold, a second device receiving importance assessment information from a first device indicating the importance of channel measurement information when updating a classification model, wherein the classification model is used to determine the classification result of a communication channel based on the channel measurement information; determining whether the importance of the channel measurement information exceeds an importance threshold; and causing a third device to perform at least classification labeling of the communication channel.

[0010] In a fifth embodiment, a first device is provided. The first device includes means for determining a classification result of a communication channel, at least in part on channel measurement information for the communication channel, using a classification model; means for determining importance evaluation information to indicate the importance of the channel measurement information when updating the classification model, at least in part on the type of classification model; and means for transmitting the importance evaluation information to a second device.

[0011] In a sixth embodiment, a second device is provided. The second device includes means for receiving importance assessment information from the first device, which indicates the importance of channel measurement information when updating a classification model, the classification model being used to determine the classification result of communication channels based on the channel measurement information; means for determining whether the importance of the channel measurement information exceeds an importance threshold; and means for causing a third device to perform classification labeling for at least communication channels at locations related to the first device, in accordance with the determination that the importance of the channel measurement information exceeds an importance threshold.

[0012] In a seventh embodiment, a computer-readable medium is provided. The computer-readable medium includes stored instructions for causing a device to perform at least the method according to the first embodiment.

[0013] In an eighth embodiment, a computer-readable medium is provided. The computer-readable medium includes stored instructions for causing the device to perform at least the method according to the second embodiment.

[0014] It should be understood that the summary is not intended to identify any significant or essential features of the embodiments of this disclosure, nor is it intended to be used to limit the scope of this disclosure. Other features of this disclosure will be readily apparent through the following description. [Brief explanation of the drawing]

[0015] Next, several exemplary embodiments will be described with reference to the attached drawings. [Figure 1] Figure 1 shows an example of a communication environment in which exemplary embodiments of the present disclosure can be implemented. [Figure 2] Figure 2 shows a signaling flow for communication according to some exemplary embodiments of the present disclosure. [Figure 3A] Figure 3A shows an example of a first type of classification model in some exemplary embodiments of the present disclosure. [Figure 3B] Figure 3B shows an example of a first type of classification model in some exemplary embodiments of the present disclosure. [Figure 4] Figure 4 shows a flowchart of the process for determining importance assessment information in some exemplary embodiments of the present disclosure. [Figure 5] Figure 5 shows an example of a second type of classification model and a reference classification model generated therefrom in some exemplary embodiments of the present disclosure. [Figure 6] Figure 6 shows a flowchart of the process for determining importance assessment information in some further exemplary embodiments of the present disclosure. [Figure 7A] Figure 7A shows the gains in model performance in some exemplary embodiments of this disclosure compared to conventional model learning approaches. [Figure 7B] Figure 7B shows the model performance gains in some exemplary embodiments of this disclosure compared to conventional model learning approaches. [Figure 8] Figure 8 shows a flowchart of a method implemented in a first device according to some exemplary embodiments of the present disclosure. [Figure 9] Figure 9 shows a flowchart of a method implemented in a second device according to some exemplary embodiments of the present disclosure. [Figure 10] Figure 10 is a simplified block diagram of an apparatus suitable for carrying out exemplary embodiments of the present disclosure. [Figure 11] FIG. 11 shows a block diagram of an exemplary computer-readable medium in some exemplary embodiments of the present disclosure. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. **DETAILED DESCRIPTION OF THE INVENTION**

[0016] Next, the principles of the present disclosure will be described with reference to some exemplary embodiments. It should be understood that these embodiments are described for purposes of illustration and are useful for those skilled in the art to understand and implement the present disclosure, and do not imply any limitation regarding the scope of the present disclosure. The embodiments described herein can be implemented in various ways other than those described below.

[0017] In the following description and claims, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs.

[0018] References to "one embodiment", "an embodiment", "exemplary embodiments", etc. in the present disclosure indicate that the described embodiments may include a particular feature, structure, or characteristic, but not all embodiments need to include the particular feature, structure, or characteristic. Further, such expressions do not necessarily refer to the same embodiment. Additionally, it should be noted that if a particular feature, structure, or characteristic is described in relation to an embodiment, it is within the knowledge of those skilled in the art to affect such feature, structure, or characteristic in relation to other embodiments, whether explicitly described or not.

[0019] In this specification, terms such as “first,” “second,” etc., may be used to describe various elements, but it should be understood that these elements should not be limited by these terms. These terms are used merely to distinguish one element from another. For example, without departing from the scope of the exemplary embodiments, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element. As used herein, the terms “and / or” include any one or more of the enumerated terms and all combinations thereof.

[0020] As used herein, “at least one of two or more lists of elements,” “at least one of two or more lists of elements,” and similar expressions in which two or more lists of elements are joined by “and” or “or” mean at least one of the elements, or at least two or more of the elements, or at least all of the elements.

[0021] The terms used herein are for the purpose of describing specific embodiments and are not intended to limit the exemplary embodiments. As used herein, the singular forms "a," "an," and "the" are intended to include the plural form unless the context clearly indicates otherwise. As used herein, the terms "equip," "equip," "have," "have," "include," and / or "include" identify the presence of the described features, elements, and / or components, etc., but do not exclude the presence or addition of one or more other features, elements, components, and / or combinations thereof.

[0022] As used in this application, the term "circuit" means (a) Hardware-only circuit implementation (such as implementation of analog and / or digital circuits only), (b) combination of hardware circuitry and software (if applicable), (i) combination of analog and / or digital hardware circuits and software / firmware, (ii) A part(s) of a hardware processor, including software (including a digital signal processor), software, and memory, that works together to perform various functions on a device such as a mobile phone or a server. (c) Hardware circuits and processors such as microprocessors and parts of microprocessors that require software (such as firmware) to operate, but may not exist when the software is not needed for operation. It may refer to one or more of these, or all of them.

[0023] This definition of circuit applies to all use of the term in this application, including in all claims. As a further example, in the use of this embodiment, the term circuit also includes not only a hardware circuit or processor (or more processors) or a part of a hardware circuit or processor and the implementation of the software and / or firmware associated with it (or them). The term circuit also includes, for example, a baseband integrated circuit or processor integrated circuit for a portable device, or a similar integrated circuit in a server, cellular network equipment, or other computing or network equipment, where applicable to the elements of a particular claim.

[0024] As used herein, the term “communication network” refers to a network conforming to any appropriate communication standard, such as New Radio (NR), Long-Term Evolution (LTE), LTE-Advanced (LTE-A), Broadband Code Division Multiple Access (WCDMA®), High-Speed ​​Packet Access (HSPA), and Narrowband Internet of Things (NB-IoT). Furthermore, communication between terminal equipment and network equipment in a communication network may be carried out in accordance with any appropriate generation of communication protocol, including but not limited to first-generation (1G), second-generation (2G), 2.5G, 2.75G, third-generation (3G), fourth-generation (4G), 4.5G, fifth-generation (5G) communication protocols, and / or other protocols currently known or to be developed in the future. Embodiments of this disclosure can be applied to a variety of communication systems. Given the rapid development of communications, there will of course be future communication technologies and systems to which this disclosure can be embodied. The scope of this disclosure should not be considered to be limited to the aforementioned systems only.

[0025] As used herein, the term “network equipment” refers to a node in a communications network from which terminal equipment accesses the network and receives services. Network equipment can vary depending on the terminology and technology applied, and may include base stations (BS) or access points (APs), such as Node B (NodeB, or NB), evolved Node B (eNodeB, or eNB), NR NB (also known as gNB), remote radio units (RRUs), radio headers (RHs), remote radio heads (RRHs), relays, integrated access backhaul (IAB) nodes, low-power nodes such as femto and pico, satellite network equipment, non-terrestrial network (NTN) or non-terrestrial network equipment such as low orbit (LEO) satellites and geostationary (GEO) satellites, aircraft networks, etc. In some exemplary embodiments, a radio access network (RAN) split architecture includes centralized units (CUs) and distributed units (DUs) in an IAB donor node. An IAB node includes a mobile terminal (IAB-MT) portion that acts like a UE toward the parent node, while the DU portion of the IAB node acts like a base station toward the next hop IAB node.

[0026] The term "terminal equipment" refers to any terminal equipment capable of wireless communication. Examples, rather than being limited, may also be called communication equipment, user equipment (UE), subscriber station (SS), mobile subscriber station, mobile station (MS), or access terminal (AT). Terminal devices include, but are not limited to, mobile phones, cellular phones, smartphones, voice over IP (VoIP) phones, wireless local loop phones, tablets, wearable devices, personal digital assistants (PDAs), portable computers, desktop computers, image capture devices such as digital cameras, game consoles, music storage devices, playback appliances, in-vehicle wireless terminal devices, wireless endpoints, mobile stations, laptop embedded devices (LEEs), laptop-mounted devices (LMEs), USB dongles, smart devices, wireless home equipment (CPEs), Internet of Things (IoT) devices, wearables such as watches, head-mounted displays (HMDs), vehicles, drones, medical devices and applications (e.g., remote surgery), industrial devices and applications (e.g., robots and / or other wireless devices operating in the context of industrial and / or automated processing chains), consumer electronics devices, and devices operating on commercial and / or industrial wireless networks. Terminal devices may also correspond to the mobile terminal (MT) portion of an IAB node (e.g., a relay node). In the following explanation, the terms “terminal equipment,” “communication equipment,” “terminal,” “user equipment,” and “UE” may be used interchangeably.

[0027] In this embodiment, the terms “resource,” “transmit resource,” “resource block,” “physical resource block” (PRB), “uplink resource,” or “downlink resource” may refer to any resource for performing communication, such as any resource for performing communication between terminal equipment and network equipment, such as time-domain resources, frequency-domain resources, spatial-domain resources, code-domain resources, or any other resources that enable communication. Hereinafter, unless expressly stated otherwise, both frequency-domain and time-domain resources will be used as examples of transmit resources for describing some exemplary embodiments of this disclosure. It should be noted that the exemplary embodiments of this disclosure are equally applicable to other resources in other domains.

[0028] As used herein, the term “model” refers to the relationship between inputs and outputs learned from training data, and therefore, after training, a corresponding output can be generated for a given input. Model generation may be based on machine learning (ML) techniques, which may also be referred to as artificial intelligence (AI) techniques. In general, machine learning models can be constructed that receive input information and make predictions based on that input information. For example, a classification model can predict the category of input information from a given number of categories. In this specification, “model” may also be referred to as “machine learning model,” “learning model,” “machine learning network,” or “learning network,” and these terms are used interchangeably herein.

[0029] Deep learning (DL) is a machine learning algorithm that uses multiple layers of processing units to process input and provide a corresponding output. A neural network (NN) model is an example of a deep learning-based model. A neural network can process input and provide a corresponding output, and typically includes an input layer, an output layer, and one or more hidden layers between the input and output layers. Neural networks used in deep learning usually include many hidden layers to increase the network's depth. Each layer of a neural network is connected sequentially, with the output of the preceding layer being provided as the input to the next layer, the input layer receiving input from the neural network, and the output of the output layer being considered the final output of the neural network. Each layer of a neural network contains one or more nodes (also called processing nodes or neurons), each node processing input from the preceding layer.

[0030] Generally, model lifecycle management typically involves three stages: the learning stage, the validation stage, and the application stage (also known as the inference stage). During the learning stage, a given machine learning model is repeatedly trained (or optimized) using a large amount of training data until it can produce consistent inferences similar to those performed by human intelligence. During training, the set of model parameter values ​​is repeatedly updated until the learning objective is reached. Through the learning process, the machine learning model can be considered to have learned the relationships between inputs and outputs (also known as input-output mappings) from the training data. In the validation stage, validation inputs are applied to the trained machine learning model to test whether the model can provide the correct output, thereby determining the model's performance. Generally, the validation stage may or may not be considered a stage in the learning process. In the inference stage, the obtained machine learning model can be used to process real-world model inputs based on the set of parameter values ​​obtained from the learning process, and the corresponding model outputs can be determined. In some cases, a retraining or update stage is included in the model lifecycle management to allow the evolved model to perform better.

[0031] [Example of an environment] Figure 1 shows an exemplary communication environment 100 in which exemplary embodiments of the present disclosure may be implemented. The communication environment 100 involves a plurality of communication devices, including one or more first devices (110-1, 110-2, 110-3), a second device 120, a third device 130, and a fourth device 140. For illustrative purposes, the first devices (110-1, 110-2, and 110-3) are referred to as first device 110, either collectively or individually.

[0032] The number of devices and their connections shown in Figure 1 are for illustrative purposes only and should not be considered limiting. The communication environment 100 may include any appropriate number of devices adapted to carry out embodiments of this disclosure. One or more additional devices, not shown, may be involved in the communication environment 100.

[0033] Communication in communication environment 100 can be carried out in accordance with any suitable communication protocol(s), including but not limited to, cellular communication protocols such as first-generation (1G), second-generation (2G), third-generation (3G), fourth-generation (4G), and fifth-generation (5G), wireless local network communication protocols such as the Institute of Electrical and Electronics Engineers (IEEE) 802.11, and / or other protocols currently known or to be developed in the future. Furthermore, communication can utilize, but not limited to, suitable wireless communication technologies, including, but not limited to, code division multiple access (CDMA), frequency division multiple access (FDMA), time division multiple access (TDMA), frequency division duplexing (FDD), time division duplexing (TDD), multiple input multiple output (MIMO), orthogonal frequency division multiple access (OFDM), discrete Fourier transform spread OFDM (DFT-s-OFDM), and / or other technologies currently known or to be developed in the future.

[0034] In the communication environment 100, the first device 110 and the fourth device 140 can communicate with each other. In the example in Figure 1, the first device 110 is shown as a terminal device, and the fourth device is shown as a network device such as a Transceiver Point (TRP). In some exemplary embodiments, when the first device is a terminal device and the fourth device 140 is a network device, the link from the fourth device 140 to the first device 110 is called a downlink (DL), and the link from the first device 110 to the fourth device 140 is called an uplink (UL).

[0035] Positioning techniques may be applied to acquire location information of the first device 110. In some exemplary embodiments, the positioning technique may be based on DL and DL+UL location measurements acquired at the first device 110 for UE-assisted positioning, or on UL and DL+UL measurements acquired at the fourth device 140 for network-assisted positioning. In some cases, different positioning techniques may be applied to ensure positioning accuracy depending on the category of the communication channel between the first device 110 and the fourth device 140. For example, different positioning methods may be applied by identifying whether the communication channel has light-of-sight (LOS) propagation or non-line-of-sight (NLOS) propagation. Thus, identifying the category of the communication channel is important because it affects at least the accuracy of the positioning estimation.

[0036] To predict the classification result of the communication channel between the first device 110 and the fourth device 140, it is proposed to introduce one or more classification models into the first device 110. The classification models can be constructed based on AI technology. The processing by the classification model is as follows:

number

number

[0037] The first device 110 can detect a reference signal propagated from the fourth device 140 through each communication channel in order to acquire channel measurement information. In some exemplary embodiments, the fourth device 140 can transmit a reference signal based on settings or signaling from the second device 120. In some exemplary embodiments, the communication channels may be classified by a classification model as either an LOS channel (with LOS propagation) or an NLOS channel (with NLOS propagation).

[0038] In some exemplary embodiments, the second device 120 can maintain and manage the classification model used by the first device 110. The second device 120 may include a location server or controller. In some exemplary embodiments, the second device 120 may include network elements in a core network (CN) configured for location management. In some exemplary embodiments, the second device 120 may include a location management function (LMF), although other terms may be used.

[0039] The accuracy of a classification model depends on the training data. A prerequisite for supervised learning models is that the training data must be pre-labeled. For a classification model configured for channel classification, the labeled training data includes sample channel measurement information as sample model input and the ground truth classification results as ground-truth model labels. Field measurement and labeling typically require the support of external equipment / devices.

[0040] For example, a third device 130 in a communication environment 100 can be configured to facilitate field measurement and classification labeling. The third device 130 can typically determine its location. In some exemplary embodiments, the third device 130 may include a positioning reference unit (PRU), although other terms may be used. This third device 130 may be required by the second device 120 to perform field measurements and determine the correct classification result of channel measurement information measured at its location. Note that although one third device is illustrated, there may be multiple third devices that can be required to perform classification labeling.

[0041] In some cases, classification models may evolve or be fine-tuned to perform better (e.g., higher accuracy) even after being deployed to a first device. Such model evolution may require the addition of labeled training data. However, there is a lack of knowledge to properly determine and request whether or not a third device should perform field measurements and labeling. If labeling is requested from one or more third devices, the collected training data may not necessarily contribute to improving model performance. Requesting third devices to perform labeling in areas where the current classification model can already provide descent estimation results, or missing blind spots where more training samples are required for model training, leads to a waste of resources.

[0042] For efficient updating of training data and retraining of the model, a minimum of appropriately labeled training data that is informative and likely to improve the model's performance is required.

[0043] Operating principle and signaling flow example According to some exemplary embodiments of this disclosure, a solution for on-demand labeling for channel classification learning is provided. In this solution, a first device determines importance assessment information representing the importance of channel measurement information when updating the classification model, based on the type of classification model. The importance assessment information is transmitted to a second device. The second device compares the importance to an importance threshold. If the importance of the channel measurement information exceeds the importance threshold, the second device causes a third device to perform classification labeling for the communication channels of the first device. In this way, on-demand labeling can be performed in a collaborative manner by evaluating the importance of channel measurement information for improving the classification model. Labeling overhead for the third device is reduced, and efficient updating of labeled training data for model improvement can be achieved.

[0044] In some exemplary embodiments, more training data can be obtained from classification labeling to update or train a classification model. In this case, efficient model training can achieve optimal learning performance with a small set of well-labeled training data.

[0045] Hereinafter, exemplary embodiments of this disclosure will be described in detail with reference to the accompanying drawings.

[0046] Herein, we refer to Figure 2, which illustrates a signaling flow 200 for communication according to some exemplary embodiments of the present disclosure. As shown in Figure 2, the signaling flow 200 involves a first device 110, a second device 120, and a third device 130. For illustrative purposes, the signaling flow 200 will be described with reference to Figure 1. Although one first device 110 and one third device 130 are illustrated in Figure 2, it should be understood that there may be multiple first devices performing similar operations, as described below with respect to the first device 110, and multiple third devices performing similar operations, as described below with respect to the third device 130.

[0047] The first device 110 determines the classification result of the communication channel based at least in part on channel measurement information about the communication channel (205). A classification model is applied to determine the classification result of the communication channel.

[0048] In some exemplary embodiments, the fourth device 140 can transmit a reference signal, and the first device 110 can obtain channel measurement information by measuring the reference signal propagating through the communication channel between the first device 110 and the fourth device 140. In some exemplary embodiments, the first device 110 may include terminal equipment, and the fourth device 140 may include network equipment.

[0049] Channel measurement information may include one or more types of information useful for characterizing a communication channel. In some exemplary embodiments, channel measurement information may include channel impulse response (CIR), channel status information (CSI), received signal strength indicator (RSSI), reference signal received power (RSRP), and / or other measurable information.

[0050] A classification model may be configured to extract representative features of channel measurement information in a high-dimensional feature space using machine learning and to classify communication channels using these features. The classification model may be configured to have multiple potential channel categories into which a communication channel can be classified. In some exemplary embodiments, the classification model can perform two categorical classifications to classify a communication channel into either a first channel category or a second channel category. In some exemplary embodiments, the multiple channel categories may include LOS channels and NLOS channels. The classification result may indicate the predicted probability that a communication channel is classified as an LOS channel or an NLOS channel. Other channel categories may be defined depending on the actual application, but this is not limited to the scope of this disclosure.

[0051] In addition to classifying communication channels, the first device 110 further decides whether and / or how to report supporting information to facilitate on-demand classification labeling. Specifically, the first device 110 determines importance assessment information indicating the importance of channel measurement information when updating the classification model, at least in part based on the type of classification model (210). The first device 110 transmits the importance information to the second device 120 (215).

[0052] From the perspective of lifecycle management of the classification model applied by the first device 110, employing on-demand classification labeling in training is beneficial. In exemplary embodiments of this disclosure, the first device 110 determines importance assessment information related to channel measurement information, thereby enabling the second device 120 to trigger the third device 130 to perform classification labeling when it is found that the channel measurement information is important when updating the classification model. As used herein, channel measurement information important when updating a classification model may involve cases where the channel measurement information is beneficial and provides new features not currently captured by the classification model. In this case, classification labeling of the corresponding communication channels can provide novel and beneficial training data that helps fine-tune the classification model, for example, to correctly classify communication channels having similar features.

[0053] Different importance assessment information may be determined for different types of classification models. In some exemplary embodiments, to measure whether channel measurement information is important in updating the classification model, the importance assessment information may be determined based on the uncertainty or confidence of the classification results determined by the model based on the channel measurement information. In some exemplary embodiments, the uncertainty of the classification results may be determined, and uncertainty may represent the degree to which the classification model is confident or doubtful about its classification results. Uncertainty may also be referred to as the degree of doubt. Alternatively, in contrast to “uncertainty,” certainty, reliability, or confidence in the classification results may be measured.

[0054] Generally, it is not possible to directly identify the root cause of a classification error between "ambiguity of the channel itself" and "immaturity of the classification model's estimation." However, because the immaturity of the classification model is not considered, the second device 120 is likely to cause inappropriate follow-up behavior. For example, if the classification ambiguity is caused by ambiguity of the channel itself, while the classification model is believed to be confident in its classification results, this may affect the choice of follow-up positioning approach between geometric typed (e.g., TDOA, AOA, AOD) schemes or fingerprint typed schemes. On the other hand, if the classification ambiguity is caused by the immaturity of the classification model before it is fully trained or fine-tuned, this may affect the subsequent enrichment of training data and model updates from a model lifecycle management perspective, i.e., the need to collect more labeled training data for model fine-tuning.

[0055] For example, in Figure 1, the first device 110-1 is x i The channel measurement information is obtained as f AI-s Using a classification model represented as ( ), a classification result indicating that the communication channel is an LOS channel,

number

number

number

[0056] From the above perspective, it is useful to evaluate the uncertainty or reliability of the classification results given by the current classification model. In some exemplary embodiments, the importance of channel measurement information may be determined based on the uncertainty of the classification results. The importance assessment information may be generated to include at least the uncertainty of the classification results. In some exemplary embodiments, higher uncertainty in the classification results may correspond to higher importance of the channel measurement information, meaning that the communication channel or channel measurement information may be important enough to require classification labeling.

[0057] In some exemplary embodiments, the uncertainty for different types of classification models can be determined in different ways by considering the classification scheme implemented by the classification model. Some types of classification models may require relatively high overhead to compute the uncertainty, while some other types of classification models may require relatively low overhead.

[0058] For some types of classification models that require high overhead to calculate uncertainty (e.g., a first type of classification model), the first device 110 may determine the classification uncertainty and generate importance assessment information that includes at least the uncertainty. In some exemplary embodiments, the uncertainty of the classification result output from a first type of model may be determined based on intermediate output information obtainable from the model, and therefore, reconstruction of the classification model is not required. In such cases, the calculation of uncertainty may not introduce high overhead and therefore may be performed by the first device 110.

[0059] For some other types of classification models that require low overhead (e.g., a second type of classification model), the first device 110 may request other devices, such as a second device 120, to assist in determining certainty. In some exemplary embodiments, the certainty of the classification result output from a second type of classification model may be determined by reconstructing the classification model, which may result in high overhead and resource consumption. In those embodiments, the first device 110 may provide the second device 120 with at least channel measurement information. Thus, critical evaluation information may include at least channel measurement information. The first device 110 may send an assistance request to the second device 120 containing the critical evaluation information. Upon receiving the assistance request, the second device 120 can determine uncertainty based on the channel measurement information.

[0060] Several exemplary embodiments for calculating uncertainty for different types of classification models are described in detail below with reference to Figures 3 to 6.

[0061] In some exemplary embodiments, in addition to or alternative to factors of uncertainty in the classification results, the importance of channel measurement information can be determined based on other factors that can indicate whether the current communication channel of the first device 110 is beneficial or important for improving the classification model.

[0062] As described above from the perspective of the first device 110, importance assessment information related to channel measurement information for a given communication channel is reported to the second device. In some exemplary embodiments, the fourth device 140 can acquire channel measurement information, for example, by receiving channel measurement information from the first device 110 or by measuring a reference signal transmitted from the first device 110. In such cases, if a classification model is deployed in the fourth device 140, the fourth device 140 can perform operations similar to those described herein with respect to the first device 110. In other words, network equipment can also transmit importance assessment information to facilitate classification labeling for updating the classification model.

[0063] On the side of the second device 120, in order to receive importance evaluation information, the importance of the channel measurement information obtained by the first device 110 can be determined (220). The second device 120 determines whether the importance of the channel measurement information exceeds the importance threshold (225). The importance threshold may be any predetermined threshold level.

[0064] In some exemplary embodiments, if the importance assessment information from the first device 110 includes uncertainty in the classification result, the second device 120 can determine the importance of the corresponding channel measurement information based on the uncertainty and compare the importance to an importance threshold. For example, higher uncertainty may correspond to a higher importance of the corresponding channel measurement information. In some examples, uncertainty may be considered as the importance of the corresponding channel measurement information. In some examples, the importance of the corresponding channel measurement information may be determined based on one or more other factors other than uncertainty. In some exemplary embodiments, if the importance assessment information from the first device 110 includes the channel measurement information itself, as described above, the second device 120 can first determine the uncertainty based on the channel measurement information. The determination of uncertainty will be described in detail below.

[0065] If the importance of channel measurement information exceeds the importance threshold, the second device 120 causes the third device 130 to perform classification labeling for communication channels at locations at least related to the first device 110 (230). Based on the importance assessment information reported by the first device 110, the second device 120 can assess the importance of specific channel measurement information in improving the classification model. The third device 130 may be requested by the second device 120 to perform classification labeling in the region where important channel measurement information has been found.

[0066] In some cases, if the importance of channel measurement information is determined to be below the importance threshold, the second device 120 can discard the importance assessment information. In this way, classification labeling is not triggered for channel measurement information that is not important during model updates. Both labeling efficiency and model update efficiency are improved.

[0067] The third device 130 performs classification labeling in response to a request from the second device 120 (235). The location where the third device 130 performs classification labeling may be any location within the region where the first device 110 is located. In some exemplary embodiments, the second device 120 may select a suitable third device 130 located near the first device 110 to perform classification labeling. In some exemplary embodiments, the third device 130 may be movable and may be requested by the second device 120 to move to the region where the first device 110 is located. The third device 130 can determine the correct classification result of the communication channel with the fourth device 140 in the region. The correct classification result may label the communication channel as either a first channel category (e.g., LOS channel) or a second channel category (e.g., NLOS channel).

[0068] In some exemplary embodiments, the third device 130 can perform further measurements on the communication channel between the first device 110 and the fourth device 140 and label the communication channel with a ground truth classification result. For example, the third device 130 can acquire sample channel measurement information (represented as "x") for the communication channel and determine a ground truth classification result (represented as "y") for the sample channel measurement information. The third device 130 can then transmit a classification labeling result to the second device 120, which includes pairs {x,y} of sample channel measurement information and corresponding ground truth classification results (240).

[0069] In some exemplary embodiments, within the region in which the third device 130 is moved, the third device 130 can perform classification labeling for other communication channels. The third device 130 can obtain one or more additional pairs of sample channel measurement information and corresponding ground truth classification results by changing its position and / orientation of its antenna within the geographic region in which the first device 110 is located. The classification labeling results transmitted to the second device 120 may include multiple pairs of sample channel measurement information and corresponding ground truth classification results.

[0070] The second device 120 receives classification labeling results from the third device 130 (245) and updates the classification model used by at least the first device 110 based on the classification labeling results (250). In some exemplary embodiments, the second device 120 can update the training dataset using at least one pair of sample channel measurement information and corresponding ground truth classification results. The second device 120 can trigger an update of the classification model after sufficient training data has been collected from the third device 130 and other data sources. For example, the second device 120 may determine whether the size of the newly collected training data exceeds a threshold. If the size exceeds the threshold, an update of the classification model may be triggered. Since the training data is considered important and useful, the updated classification model can be improved to have higher accuracy.

[0071] In some cases, in addition to the classification model used in the first device 110 that reports importance assessment information, the second device 120 may maintain one or more other classification models. Sample channel measurement information and corresponding ground truth classification results collected by the third device 130 may be shared among the classification models. In other words, the second device 120 may update one or more other classification models based on the sample channel measurement information and corresponding ground truth classification results. These classification models maintained by the second device 120 may be of different types and / or different model configurations, but may all be configured to classify communication channels. Channel measurement information that is considered important when updating one classification model may also be important and useful when updating other classification models.

[0072] In some exemplary embodiments, if the inputs to different classification models are not the same (for example, if different channel measurement information is required), the third device 130 may be required by the second device 120 to collect different sample channel measurement information for the same communication channel, along with the correct classification result.

[0073] The second device 120 can apply any appropriate update technique to the classification model, but this is not limited to the scope of this disclosure.

[0074] In some exemplary embodiments, with one or more classification models updated, the second device 120 can transmit the update to the first device 110 (255). The first device 110 receives the update to the classification model (multiple) and can apply the updated classification model (multiple) for the next channel classification (260). In exemplary embodiments, the second device 120 can provide the first device 110 with an updated classification model previously used. In exemplary embodiments, other updated classification models may also be provided to the first device 110. The first device 110, configured with multiple (updated) classification models, can select one of the models to use, for example, depending on the environment related to the communication channel.

[0075] [First type of classification model and calculation of uncertainty] As mentioned above, the uncertainty of importance assessment information or channel measurement information can be determined depending on the type of classification model.

[0076] In some exemplary embodiments, for a first type of classification model, the uncertainty of the classification result output from the model can be determined based on intermediate output information obtainable from the model. For example, in the case of binary classification, the classification model can determine a first number of model votes (represented as "N1") for a first channel category and a second number of model votes (represented as "N2") for a second channel category, based on input channel measurement information.

[0077] The classification result may be determined based on the ratio of the first number to the second number (e.g., N1 / N2), where a higher ratio indicates a higher probability that the communication channel is classified into the first channel category.

[0078] In this type of classification model, the classification result is a "soft" indicator of the channel category to which the communication channel is classified. This type of classification model is fAI-s ( ) can be represented as. In the description of the embodiment of FIG. 1, it is shown that the first devices (110-1 and 110-2) perform channel classification using this type of classification model. Some examples of this type of classification model may include, but are not limited to, the k-nearest neighbor (KNN) model and the support vector machine (SVM) model.

[0079] FIGS. 3A and 3B show examples of a first type of classification model according to some exemplary embodiments of the present disclosure. FIGS. 3A and 3B show a feature space 300 including a plurality of features 302 related to a first channel category (represented as "Category 1") and a plurality of features 304 related to a second channel category (represented as "Category 2"). The classification scheme applied by the classification model is configured to measure the respective distances between the features extracted from the channel measurement information and the features in the feature space 300, and select a predetermined number (e.g., K) of features with small distances (e.g., the K features with the smallest distances).

[0080] Among the total K selected features, the classification model may count a first number of features related to the first channel category (i.e., the first number N1 of model votes for the first channel category) and a second number of features related to the second channel category (i.e., the second number N2 of model votes for the second channel category). The ratio of the first number to the second number can be used to determine the probability that the communication channel belongs to the first channel category.

[0081] In the example of FIG. 3A, the feature 312 of the channel measurement information x obtained by the first device 110-1 i is close to six features related to Category 1 and one feature related to Category 2, which means that the probability of the communication channel of the first device 110-1 is 6 / 7. In the embodiment of FIG. 3B, the channel measurement information x obtained by the first device 110-2 jFeature 314 is close to four features associated with Category 1 and three features associated with Category 2, which means that the probability that the communication channel of the first device 110-1 belongs to the first channel category is 4 / 7.

[0082] It should be understood that the examples in Figures 3A and 3B are provided for illustrative purposes only and do not imply a classification approach for the first type of classification model. Some classification models can operate in other ways to determine model votes for two channel categories and output classification results.

[0083] In some exemplary embodiments, the uncertainty of the classification results output by this type of classification model may be determined based on intermediate output information, for example, a first number N1 of model votes for a first channel category and a second number N2 of model votes for a second channel category.

[0084] Figure 4 shows a flowchart of a process 400 for determining importance assessment information in some exemplary embodiments of the present disclosure. Process 400 may be carried out, for example, by the first device 110.

[0085] In block 410, the first device 110 counts the first number N1 of model votes for the first channel classification and the second number N2 of model votes for the second channel classification. These two numbers can be obtained from the classification model.

[0086] In block 420, the first device 110 determines the degree of difference between the first number N1 and the second number N2, and in block 430, the first device 110 determines uncertainty based on the degree of difference.

[0087] In binary classification model voting, if the classification model is more reliable in its estimations, the number of model votes for one channel category may be larger, and correspondingly, the number of model votes for the other channel category may be smaller. Therefore, a large difference between the first and second numbers indicates that the classification model is more reliable in its classification results, and thus the uncertainty of the classification results is low.

[0088] In some exemplary embodiments, the degree of difference between a first number N1 and a second number N2 may be measured based on the larger value between N1 / N2 or N2 / N1, which may be expressed as max(N1 / N2,N2 / N1). In some exemplary embodiments, the sum of N1 and N2 is determined as K, and the degree of difference between the first number N1 and the second number N2 may be measured based on the larger value between the ratio of N1 to K and the ratio of N2 to K, which may be expressed as max(N1,N2) / K. In these cases, if max(N1 / N2,N2 / N1) or max(N1,N2) / K is determined to have a higher value, the uncertainty of the classification result (expressed as "γ") may be determined to be at a higher level. In some embodiments, the uncertainty γ may be determined as γ = max(N1 / N2,N2 / N1) or max(N1,N2) / K. In other embodiments, uncertainty γ can be determined by other means based on the degree of difference between N1 and N2.

[0089] In some exemplary embodiments, once the uncertainty of the classification result is determined, the first device 110 can generate importance assessment information that includes at least the determined uncertainty.

[0090] In some exemplary embodiments, the first device 110 can transmit importance assessment information when relatively high uncertainty is found. As shown in Figure 4, in block 440, the first device 110 may determine whether uncertainty γ exceeds an uncertainty threshold represented as γ0. If uncertainty γ exceeds the uncertainty threshold γ0, in block 450, the first device 110 decides to transmit important assessment information, including the uncertainty, to the second device 120. If uncertainty γ does not exceed the uncertainty threshold γ0, in block 460, the first device 110 decides that it is not necessary to transmit importance assessment information.

[0091] As mentioned above, high uncertainty can correspond to the high importance of channel measurement information in updating the classification model. By reporting uncertainty exceeding the uncertainty threshold to the second device 120, it is possible to further reduce the transmission overhead between the first device 110 and the second device 120.

[0092] In exemplary embodiments, if applicable, the first device 110 may determine the importance of channel measurement information based on uncertainty and several other possible factors in order to generate importance assessment information. The first device 110 may determine whether the importance exceeds an importance threshold and decide to transmit importance assessment information if the importance exceeds the corresponding importance threshold. The importance threshold may be one applied by the second device 120.

[0093] In some exemplary embodiments, an uncertainty threshold or importance threshold may be set by the second device 120 to control how "important" the channel measurement information is considered to be, or how "uncertain" the classification model is about its classification results. In some exemplary embodiments, the uncertainty threshold or importance threshold may be determined based on the accuracy level of the classification model.

[0094] For example, if a classification model has a low accuracy level (e.g., 55%) in its initial stages, it generally means that the model may not distinguish very well between channel categories. The uncertainty threshold or importance threshold may be set to a relatively low value so that more channel measurement information is deemed important and the third device 130 can collect more training data for model updates.

[0095] In some exemplary embodiments, the uncertainty threshold γ0 may be updated based on updates to the classification model. As the classification model is updated and becomes more mature, its accuracy level may increase, and the uncertainty threshold or importance threshold may also be set to a larger value. For example, if the accuracy level of the classification model increases to 80%, the classification model becomes more reliable in its classification results, and the uncertainty threshold or importance threshold may also increase.

[0096] It should be understood that in some other exemplary embodiments, instead of calculating the uncertainty of the first type of classification model, the first device 110 may alternatively generate importance assessment information including channel measurement information and send it to the second device 120, requesting the second device 120 to perform the calculation. In those embodiments, the operations in blocks 410, 420, and 430 of process 400 may be performed in the second device 120.

[0097] [Second classification model and calculation of uncertainty] In some exemplary embodiments, the second type of classification model is a model that has uncertainty in the classification result, which is determined by reconstructing the classification model. An example of such a classification model is a deep neural network (DNN) model, which generally provides a hard output (e.g., 0 or 1) indicating whether a communication channel is classified into a first channel category or a second channel category.

[0098] Figure 5 shows an example of a second type of classification model 510 and a reference classification model generated therefrom in some exemplary embodiments of the present disclosure. The classification model 510 may be in the form of a DNN model.

[0099] As shown in Figure 5, the classification model 510 consists of an input layer 502, one or more hidden layers 504, and an output layer 506, each layer consisting of multiple computation units (also called neurons). Computation units in one layer are connected to computation units in the next layer. In some exemplary embodiments, computation units in one layer may be connected to one or more other computation units in the same layer. Channel measurement information is input to the input layer 502 for processing, and the information propagates through the hidden layer(s) 504 according to the layer connections. The classification result of the channel measurement information is output from the output layer 506. Examples of layers included in the model include convolutional layers, batch normalization layers, activation function layers, pooling layers, fully connected layers, long-short-term memory (LSTM) layers, and other types of layers.

[0100] The number of computational units and layers in the classification model 510 are arbitrary values, independent of the exemplary embodiments of this disclosure. The structure of the model is also non-limiting, and the connections between computational units may be recursive or bidirectional. Any model applicable to channel classification can be used.

[0101] For DNN-type models and similar models, there is still no common solution to directly measure the uncertainty of prediction results based on model outputs or real-time information. It is worth noting that in classification models, even if the model outputs a probability vector, this may not be directly usable to indicate the uncertainty of the classification result. That is, a classification model may have uncertain predictions even with high output probabilities. In some exemplary embodiments, it is proposed to reconstruct the classification model by slightly modifying it to generate multiple baseline classification models and determine the uncertainty based on these multiple baseline classification models.

[0102] Figure 6 shows a flowchart of a process 600 for determining importance assessment information in some further exemplary embodiments of the present disclosure. Process 600 may be carried out by, for example, a second device 120. In those embodiments, the second device 120 receives importance assessment information consisting of channel measurement information from the first device 110. To measure the importance of the channel measurement information, the second device 120 can determine the uncertainty of the classification results output by a classification model for the channel measurement information.

[0103] In block 610, the second device 120 generates multiple reference classification models by reconstructing the classification model. In some exemplary embodiments, the second device 120 can slightly modify the classification model by applying random neural connection dropout to the classification model. Specifically, the second device 120 can randomly drop out some neural connections between computational units in the classification model to obtain a reference classification model. In some exemplary embodiments, the second device 120 may apply a Gaussian process to determine which neural connections to drop out of the classification model. The second device 120 may generate multiple different reference classification models through the dropout means. The second device 120 may apply other dropout means to generate reference classification models.

[0104] In the example in Figure 5, the second device 120 can generate P reference classification models (512-1, 512-2, ..., 512-P) (collectively or individually referred to as reference classification model 512), where P is an integer greater than 1. The reference classification model 512 is f AI-p It may be represented as (.).

[0105] In block 620, the second device 120 determines multiple criterion classification results based on channel measurement information using multiple criterion classification models. The second device 120 uses the channel measurement information provided by the first device 110 to determine each criterion classification model f having p=1, 2, ..., P AI-p It can be applied as input to (.). The second device 120 has p=1, 2, ..., P

number

[0106] In block 630, the second device 120 determines the uncertainty of the classification result based on the variance of multiple criterion classification results. If the variance of multiple criterion classification results is relatively high, this means that the criterion classification model is inconsistent in classifying channel measurement information. In this case, the uncertainty of the classification result is determined to be at a relatively high level, and it may be determined that the channel measurement information is useful and important when updating the original classification model.

[0107] This allows the classification model to be updated to a more stable version, increasing the reliability of classifying similar communication channels even if the model structure is slightly modified (for example, by excluding some connections).

[0108] In some exemplary embodiments, the uncertainty threshold or importance threshold applied to the second type of classification model may be set in a manner similar to that applied to the first type of classification model described above. In some exemplary embodiments, the uncertainty threshold or importance threshold applied to the first type of classification model and the second type of classification model may be set as the same threshold or as different thresholds.

[0109] It should be understood that in some other exemplary embodiments, instead of requesting the second device 120 to calculate uncertainty about a second type of classification model, the first device 110 may alternatively determine the uncertainty locally by performing a similar operation in process 600, and then generate importance assessment information including the uncertainty and transmit it to the second device 120.

[0110] [Comparison of performance improvements] Figures 7A and 7B illustrate the gains in model performance by some exemplary embodiments of this disclosure over conventional model learning approaches. According to conventional model learning approaches, a third device may be randomly requested by a second device to perform field measurements and classification labeling without supporting information. According to exemplary embodiments of this disclosure, through cooperation with a first device, the second device can trigger classification labeling when important and useful channel measurement information is found.

[0111] Figure 7A shows the accuracy trend curve 710 of a conventional model learning approach and the accuracy trend curve 720 of the proposed approach in some exemplary embodiments of this disclosure. The two trend curves show the increase in accuracy with respect to the amount of labeled training data for learning a first type of classification model. To approach a satisfactory accuracy of approximately 0.75, the amount of labeled training data required to make the classification model satisfactory can be reduced by approximately 60% using the proposed approach compared to the conventional approach.

[0112] Figure 7B shows the accuracy trend curve 712 of a conventional model learning approach and the accuracy trend curve 722 of the proposed approach in some exemplary embodiments of the present disclosure. The two trend curves show the increase in accuracy with respect to the amount of labeled training data for training a first type of classification model. As shown, to achieve similar performance in terms of classification accuracy, the amount of labeled training data required for the classification model can be reduced by approximately 50% using the proposed approach in some exemplary embodiments of the present disclosure compared to the conventional approach.

[0113] [Example method] Figure 8 shows a flowchart of an exemplary method 800 performed on a first device in some exemplary embodiments of the present disclosure. For illustrative purposes, method 800 is described in terms of the first device 110 in Figure 1.

[0114] In block 810, the first device 110 uses a classification model to determine the classification result of the communication channel, at least partially based on channel measurement information regarding the communication channel.

[0115] In block 820, the first device 110 determines importance assessment information indicating the importance of channel measurement information when updating the classification model, at least in part based on the type of classification model.

[0116] In block 830, the first device 110 transmits importance assessment information to the second device 120.

[0117] In some exemplary embodiments, determining importance assessment information includes determining whether the classification model is of a first type or a second type; determining the uncertainty of the classification result according to the determination that the classification model is of a first type and generating importance assessment information such that it constitutes at least the uncertainty of the classification result; and generating importance assessment information such that it constitutes at least channel measurement information according to the determination that the classification model is of a second type.

[0118] In some exemplary embodiments, the first type of classification model is a model that has uncertainty in the classification result determined without reconfiguring the classification model. In some exemplary embodiments, the second type of classification model is a model that has uncertainty in the classification result determined by reconfiguring the classification model.

[0119] In some exemplary embodiments, the classification result indicates whether a communication channel belongs to a first channel category or a second channel category, and the classification result determined using a first type of classification model is based on the ratio of a first number of model votes for the first channel category to a second number of model votes for the second channel category. In some exemplary embodiments, determining uncertainty includes determining the degree of difference between the first number and the second number, and determining uncertainty based on the degree of difference.

[0120] In some exemplary embodiments, transmitting importance assessment information to a second device includes transmitting importance assessment information to the second device in accordance with a determination that the determined uncertainty exceeds an uncertainty threshold.

[0121] In some exemplary embodiments, method 800 further includes receiving an uncertainty threshold from a second device.

[0122] In some exemplary embodiments, the classification result is determined based on a predicted probability provided by a second type of classification model to indicate whether the communication channel is classified into a first channel category or a second channel category.

[0123] In some exemplary embodiments, method 800 further includes receiving at least an update of the classification model from a second device.

[0124] In some exemplary embodiments, the classification result indicates whether the communication channel is classified as a line-of-sight channel or a non-line-of-sight channel.

[0125] In some exemplary embodiments, the first device includes terminal equipment, and the second device includes location management functions. In some exemplary embodiments, the communication channel includes a channel between the terminal equipment and network equipment.

[0126] Figure 9 shows a flowchart of an exemplary method 900 performed on a second device in some exemplary embodiments of the present disclosure. For illustrative purposes, method 900 is described in terms of the second device 120 in Figure 1.

[0127] In block 910, the second device 120 receives importance assessment information from the first device, indicating the importance of the channel measurement information when updating the classification model, and the classification model uses this information to determine the classification result of the communication channel.

[0128] In block 920, the second device 120 determines whether the importance of the channel measurement information exceeds the importance threshold.

[0129] If the importance of the channel measurement information exceeds the importance threshold, in block 930, the second device 120 causes the third device to perform classification labeling for at least the communication channels at locations associated with the first device.

[0130] In some exemplary embodiments, the method 900 further includes receiving from a third device at least one pair of sample channel measurement information relating to a communication channel and ground truth classification results labeled with respect to the sample channel measurement information, and updating at least one classification model based on the sample channel measurement information and the ground truth classification results.

[0131] In some exemplary embodiments, method 900 further includes transmitting at least an update to the classification model to the first device.

[0132] In some exemplary embodiments, receiving importance assessment information includes receiving importance assessment information that includes at least uncertainty of the classification result, according to a determination that the classification model is of a first type, and receiving importance assessment information that includes at least channel measurement information, according to a determination that the classification model is of a second type.

[0133] In some exemplary embodiments, the first type of classification model is a model that has uncertainty in the classification result determined without reconfiguring the classification model. In some exemplary embodiments, the second type of classification model is a model that has uncertainty in the classification result determined by reconfiguring the classification model.

[0134] In some exemplary embodiments, method 900 further includes determining the uncertainty of the classification result based on channel measurement information, following a determination that the importance assessment information includes at least channel measurement information.

[0135] In some exemplary embodiments, the classification model is of a second type. In some exemplary embodiments, determining the uncertainty of the classification result includes generating multiple baseline classification models by reconstructing the classification model; determining multiple baseline classification results based on channel measurement information using the multiple baseline classification models; and determining the uncertainty of the classification result based on the variance of the multiple baseline classification results.

[0136] In some exemplary embodiments, multiple reference classification models are generated by applying random neural connection dropout to the classification model.

[0137] In some exemplary embodiments, uncertainty in the classification result that exceeds an uncertainty threshold is received from the first device.

[0138] In some exemplary embodiments, method 900 further includes transmitting an uncertainty threshold to a first device.

[0139] In some exemplary embodiments, the uncertainty threshold is determined based on the accuracy level of the classification model. In some exemplary embodiments, the uncertainty threshold is updated based on updates to the classification model.

[0140] In some exemplary embodiments, the classification result indicates whether the communication channel is classified as a line-of-sight channel or a non-line-of-sight channel.

[0141] In some exemplary embodiments, the first device includes terminal equipment, the second device includes location management functions, and the third device includes a positioning reference unit. In some exemplary embodiments, the communication channel includes a channel between the terminal equipment and network equipment.

[0142] [Examples of devices, equipment, and media] In some exemplary embodiments, a first device capable of performing any of the methods 800 (e.g., the first device 110 in Figure 1) may include means for performing each operation of the methods 800. The means can be implemented in any suitable form. For example, the means may be implemented in a circuit or a software module. The first device may be implemented as the first device 110 in Figure 1, or it may be included in the first device 110 in Figure 1.

[0143] In some exemplary embodiments, the first device includes means for determining a classification result of a communication channel based at least in part on channel measurement information relating to the communication channel using a classification model, means for determining importance evaluation information indicating the importance of the channel measurement information when updating the classification model based at least in part on the type of classification model, and means for transmitting the importance evaluation information to the second device.

[0144] In some exemplary embodiments, means for determining importance assessment information include means for determining whether the type of classification model is of a first type or a second type; means for determining the uncertainty of the classification result according to the determination that the type of classification model is of a first type; means for generating importance assessment information that includes at least the uncertainty of the classification result; and means for generating importance assessment information that includes at least channel measurement information according to the determination that the type of classification model is of a second type.

[0145] In some exemplary embodiments, the first type of classification model is a model that has uncertainty in the classification result determined without reconfiguring the classification model. In some exemplary embodiments, the second type of classification model is a model that has uncertainty in the classification result determined by reconfiguring the classification model.

[0146] In some exemplary embodiments, the classification result indicates whether a communication channel is classified into a first channel category or a second channel category, and the classification result determined using a first type of classification model is based on the ratio of a first number of model votes for the first channel category to a second number of model votes for the second channel category. In some exemplary embodiments, means for determining uncertainty include means for determining the degree of difference between the first number and the second number, and means for determining uncertainty based on the degree of difference.

[0147] In some exemplary embodiments, means for transmitting importance assessment information to a second device include means for transmitting importance assessment information to the second device in accordance with the determination that the determined uncertainty exceeds an uncertainty threshold.

[0148] In some exemplary embodiments, the first device further comprises means for receiving an uncertainty threshold from the second device.

[0149] In some exemplary embodiments, the classification result is determined based on a predicted probability provided by a second type of classification model to indicate whether the communication channel is classified into a first channel category or a second channel category.

[0150] In some exemplary embodiments, the first device further comprises means for receiving at least an update of the classification model from the second device.

[0151] In some exemplary embodiments, the classification result indicates whether the communication channel is classified as a line-of-sight channel or a non-line-of-sight channel.

[0152] In some exemplary embodiments, the first device includes terminal equipment, and the second device includes location management functions. In some exemplary embodiments, the communication channel includes a channel between the terminal equipment and network equipment.

[0153] In some exemplary embodiments, the first device further comprises means for performing other operations in some exemplary embodiments of Method 800 or the first device 110. In some exemplary embodiments, the means comprises at least one processor and at least one memory for storing instructions that, when executed by the at least one processor, bring out the performance of the first device.

[0154] In some exemplary embodiments, a second device capable of performing any of the methods 900 (e.g., the second device 120 in Figure 1) may include means for performing each operation of the methods 900. The means can be implemented in any preferred form. For example, the means may be implemented in a circuit or a software module. The second device may be implemented as the second device 120 in Figure 1, or it may be included in the second device 120 in Figure 1.

[0155] In some exemplary embodiments, the second device includes means for receiving importance assessment information from the first device, which indicates the importance of channel measurement information when updating a classification model, and the classification model is used to determine the classification result of a communication channel based on the channel measurement information; means for determining whether the importance of the channel measurement information exceeds an importance threshold; and means for causing the third device to perform at least classification labeling of the communication channel at a location associated with the first device, in accordance with the determination that the importance of the channel measurement information exceeds an importance threshold.

[0156] In some exemplary embodiments, the second device further comprises means for receiving from the third device at least one pair of sample channel measurement information relating to a communication channel and a ground truth classification result labeled with respect to the sample channel measurement information, and means for updating at least a classification model based on the at least one pair of sample channel measurement information and the ground truth classification result.

[0157] In some exemplary embodiments, the second device further comprises means for transmitting at least an update of the classification model to the first device.

[0158] In some exemplary embodiments, means for receiving importance assessment information include, according to a determination that the classification model is of a first type, means for receiving importance assessment information that includes at least uncertainty of the classification result, and according to a determination that the classification model is of a second type, means for receiving importance assessment information that includes at least channel measurement information.

[0159] In some exemplary embodiments, the first type of classification model is a model that has uncertainty in the classification result determined without reconfiguring the classification model. In some exemplary embodiments, the second type of classification model is a model that has uncertainty in the classification result determined by reconfiguring the classification model.

[0160] In some exemplary embodiments, the second device further comprises means for determining the uncertainty of the classification result based on channel measurement information, according to the determination that the importance assessment information includes at least channel measurement information.

[0161] In some exemplary embodiments, the classification model is of a second type. In some exemplary embodiments, the means for determining the uncertainty of the classification result comprises means for generating a plurality of reference classification models by reconstructing the classification model; means for determining a plurality of reference classification results based on channel measurement information using the plurality of reference classification models; and means for determining the uncertainty of the classification result based on the variance of the plurality of reference classification results.

[0162] In some exemplary embodiments, multiple reference classification models are generated by applying random neural connection dropout to the classification model.

[0163] In some exemplary embodiments, uncertainty in the classification result that exceeds an uncertainty threshold is received from the first device.

[0164] In some exemplary embodiments, the second device further comprises means for transmitting an uncertainty threshold to the first device.

[0165] In some exemplary embodiments, the uncertainty threshold is determined based on the accuracy level of the classification model. In some exemplary embodiments, the uncertainty threshold is updated based on updates to the classification model.

[0166] In some exemplary embodiments, the classification result indicates whether the communication channel is classified as a line-of-sight channel or a non-line-of-sight channel.

[0167] In some exemplary embodiments, the first device includes terminal equipment, the second device includes location management functions, and the third device includes a positioning reference unit. In some exemplary embodiments, the communication channel includes a channel between the terminal equipment and network equipment.

[0168] In some exemplary embodiments, the second device further comprises means for performing other operations in some exemplary embodiments of Method 900 or the second device 120. In some exemplary embodiments, the means comprises at least one processor and at least one memory for storing instructions that, when executed by the at least one processor, cause the second device to perform its operations.

[0169] Figure 10 is a simplified block diagram of a device 1000 suitable for implementing an exemplary embodiment of the present disclosure. The device 1000 may be provided for implementing a communication device, such as a first device 110 or a second device 120 as shown in Figure 1. As shown, the device 1000 includes one or more processors 1010, one or more memories 1020 coupled to the processor 1010, and one or more communication modules 1040 coupled to the processor 1010.

[0170] The communication module 1040 is for bidirectional communication. The communication module 1040 has one or more communication interfaces to facilitate communication with one or more other modules or devices. A communication interface means any interface necessary for communication with other network elements. In some exemplary embodiments, the communication module 1040 may include at least one antenna.

[0171] The processor 1010 may be of any type suitable for a local technology network and, in non-limiting examples, may include one or more of general-purpose computers, special-purpose computers, microprocessors, digital signal processors (DSPs), and processors based on multi-core processor architectures. Device 1000 may have multiple processors, such as special-purpose integrated circuit chips that are temporally slaved to a clock that synchronizes the main processor.

[0172] Memory 1020 may include one or more non-volatile memories and one or more volatile memories. Examples of non-volatile memories include, but are not limited to, read-only memory (ROM) 1024, electronically programmable read-only memory (EPROM), flash® memory, hard disks, compact discs (CDs), digital video discs (DVDs), optical discs, laser discs, and other magnetic and / or optical storage devices. Examples of volatile memories include, but are not limited to, random-access memory (RAM) 1022 and other volatile memories that do not persist during power-down time.

[0173] The computer program 1030 includes computer executable instructions that are executed by the associated processor 1010. The program 1030 may be stored in memory, for example, ROM 1024. The processor 1010 can perform any appropriate operations and processes by loading the program 1030 into RAM 1022.

[0174] Exemplary embodiments of the present disclosure may be implemented by program 1030 so that device 1000 can perform any process of the present disclosure, as described with reference to Figures 3 to 5. Exemplary embodiments of the present disclosure may also be implemented by hardware or by a combination of software and hardware.

[0175] In some exemplary embodiments, program 1030 may be tangibly contained in a computer-readable medium that may be contained in device 1000 (such as in memory 1020) or in other storage devices accessible by device 1000. Device 1000 can load program 1030 from the computer-readable medium into RAM 1022 and execute it. The computer-readable medium may include any type of tangible non-volatile storage, such as ROM, EPROM, flash® memory, hard disk, CD, or DVD. Figure 11 shows an example of computer-readable medium 1100, which may be in the form of a CD, DVD, or other optical storage disc. Program 1030 is stored in this computer-readable medium.

[0176] In general, various embodiments of this disclosure may be implemented in hardware or special-purpose circuits, software, logic, or any combination thereof. Some embodiments may be implemented in hardware, while others may be implemented in firmware or software that can be executed by a controller, microprocessor, or other processing unit. Various embodiments of this disclosure are illustrated and described using block diagrams, flowcharts, or any other graphic representation, but it should be understood that any blocks, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, special-purpose circuits or logic, general-purpose hardware or controllers or other processing units, or any combination thereof, in non-limiting examples.

[0177] This disclosure also provides at least one computer program product tangibly stored in a non-transient computer-readable storage medium. As used herein, the term “non-transient” refers to the limitations of the medium itself (i.e., being tangible and not signaling), as opposed to limitations on the persistence of data storage (e.g., ROM versus RAM). The computer program product includes computer-executable instructions, such as those contained in a program module, which are executed on a device on a target physical or virtual processor to perform one of the methods described above with reference to Figures 2 to 6. Generally, a program module includes routines, programs, libraries, objects, classes, components, data structures, etc., that perform a specific task or implement a specific abstract data type. The functionality of a program module can be combined or divided among program modules as needed in various embodiments. The machine-executable instructions of a program module can be executed in a local or distributed device. In a distributed device, the program module can reside in both local and remote storage media.

[0178] Program code for carrying out the methods of this disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the program code is executed by the processor or controller, specific functions / operations are performed on flowcharts and / or block diagrams. The program code may run entirely on the machine, partially on the machine, as a standalone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0179] In the context of this disclosure, computer program code or related data may be carried by any suitable carrier to enable a device, arithmetic unit, or processor to perform the various processes and operations described above. Examples of carriers include signals, computer-readable media, and the like.

[0180] Computer-readable media may be computer-readable signal media or computer-readable storage media. Computer-readable media include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. More specific examples of computer-readable storage media include electrical connections with one or more wires, portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash® memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0181] Furthermore, although the operations are described in a specific order, this should not be understood as requiring that such operations be performed in a specific order or sequentially, or that all illustrated operations be performed, in order to achieve the desired result. In certain circumstances, multitasking and parallel processing may be preferable. Similarly, although some specific implementation details are included in the above description, these should not be interpreted as limiting the scope of this disclosure, but rather as descriptions of features that may be specific to a particular embodiment. Certain features described in the context of a separate embodiment may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented separately or in any suitable subcombination in multiple embodiments.

[0182] While this disclosure has been described in language specific to structural features and / or methodological actions, it should be understood that the disclosure as defined in the attached claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are disclosed as exemplary forms for carrying out the claims.

Claims

1. The first device is, At least one processor, It has at least one memory, When the at least one memory is executed by the at least one processor, the first device receives at least, Using a classification model, the classification result of the communication channel is determined based at least partially on channel measurement information regarding the communication channel. Determining importance assessment information indicating the importance of the channel measurement information to facilitate classification labeling when updating the classification model, based at least in part on the type of classification model, wherein determining the importance assessment information is To determine whether the type of the classification model is the first type or the second type, In accordance with the determination that the type of the classification model is the first type, To determine the uncertainty of the aforementioned classification results, To generate the importance assessment information such that it includes at least the uncertainty of the classification result, In accordance with the determination that the type of the classification model is the second type, the importance evaluation information is generated to include at least the channel measurement information, This includes determining importance assessment information, The importance assessment information is transmitted to the second device, A first device that stores the commands to execute.

2. The first type of classification model is a type of model that has uncertainty in the classification result, which is determined without reconstructing the classification model. The second type of classification model is a type of model that has uncertainty in the classification result, which is determined by reconstructing the classification model. The first device according to claim 1.

3. The classification result indicates whether the communication channel is classified into a first channel category or a second channel category, and the classification result determined using the first type of classification model is based on the ratio of a first number of model votes for the first channel category to a second number of model votes for the second channel category. When the at least one memory is executed by the at least one processor, the first device is: To determine the degree of the difference between the first number and the second number, Based on the degree of the difference, the uncertainty is determined, Therefore, the command that determines the aforementioned uncertainty is stored. The first device according to claim 1 or 2.

4. When the at least one memory is executed by the at least one processor, the first device is: In accordance with the determination that the determined uncertainty exceeds the uncertainty threshold, transmit the importance evaluation information to the second device. The system stores a command to transmit the importance evaluation information to the second device. The first device according to claim 1 or 2.

5. When the at least one memory is executed by the at least one processor, the first device further... Receiving the uncertainty threshold from the second device, The first device according to claim 4, which stores instructions for executing the first device.

6. The first device according to claim 1 or 2, wherein the classification result is determined based on the predicted probability provided by the second type of classification model, indicating whether the communication channel is classified into a first channel category or a second channel category.

7. When the at least one memory is executed by the at least one processor, the first device further... Receiving at least the update of the classification model from the second device, The first device according to claim 1 or 2, which stores instructions for executing a command.

8. The first device according to claim 1 or 2, wherein the classification result indicates whether the communication channel is classified as a line-of-sight channel or a non-line-of-sight channel.

9. The first device includes terminal equipment, and the second device includes location management functions. The aforementioned communication channel includes the channel between the terminal device and the network device. The first device according to claim 1 or 2.

10. The second device is At least one processor, It has at least one memory, When the at least one memory is executed by the at least one processor, the second device receives at least, The first device receives importance assessment information indicating the importance of channel measurement information when updating the classification model, and the classification model is used to determine the classification result of the communication channel based on the channel measurement information. Determining whether the importance of the channel measurement information exceeds the importance threshold, In accordance with the determination that the importance of the channel measurement information exceeds the importance threshold, the third device is requested to perform classification labeling of at least the communication channels at locations related to the first device. A second device that stores the commands to execute.

11. When the at least one memory is executed by the at least one processor, it further provides the second device with: The third device receives at least one pair of sample channel measurement information relating to the communication channel and a ground truth classification result labeled with respect to the sample channel measurement information. The classification model is updated based on at least one pair of the sample channel measurement information and the correct classification result. The second device according to claim 10, which stores instructions for executing the second device.

12. When the at least one memory is executed by the at least one processor, it further provides the second device with: To transmit at least the update of the classification model to the first device, The second device according to claim 11, which stores instructions for executing the second device.

13. When the at least one memory is executed by the at least one processor, the second device, In accordance with the determination that the classification model is of the first type, the importance assessment information, which includes at least the uncertainty of the classification result, is received. In accordance with the determination that the classification model is of the second type, the importance assessment information, which includes at least the channel measurement information, is received. The second device according to any one of claims 10 to 12, which stores a command to receive the importance evaluation information.

14. The first type of classification model is a type of model that has uncertainty in the classification result, which is determined without reconstructing the classification model. The second type of classification model is a type of model that has uncertainty in the classification result, which is determined by reconstructing the classification model. The second device according to claim 13.

15. When the at least one memory is executed by the at least one processor, it further provides the second device with: In accordance with the determination that the importance assessment information includes at least the channel measurement information, the uncertainty of the classification result is determined based on the channel measurement information. The second device according to claim 13, which stores instructions for executing the second device.

16. The classification model is of the second type, wherein the at least one memory, when executed by the at least one processor, is used in the second device. By reconstructing the aforementioned classification model, multiple reference classification models are generated, Using the aforementioned multiple criterion classification models, multiple criterion classification results are determined based on the channel measurement information. Based on the variance of the aforementioned multiple criteria classification results, the uncertainty of the classification result is determined, The second device according to claim 15, which stores an instruction for determining the uncertainty of the classification result.

17. The second device according to claim 16, wherein the at least one memory stores instructions that, when executed by the at least one processor, cause the second device to generate the plurality of reference classification models by applying random neural connection dropout to the classification model.

18. The second device according to claim 14, wherein the at least one memory stores an instruction, when executed by the at least one processor, causing the second device to receive from the first device the uncertainty of the classification result that exceeds an uncertainty threshold.

19. When the at least one memory is executed by the at least one processor, it further provides the second device with: Transmitting an uncertainty threshold to the first device, A second device according to any one of claims 10 to 12, which stores a command to execute.

20. The uncertainty threshold is determined based on the accuracy level of the classification model. The uncertainty threshold is updated based on the update of the classification model. The second device according to any one of claims 10 to 12.

21. The second device according to any one of claims 10 to 12, wherein the classification result indicates whether the communication channel is classified as a line-of-sight channel or a non-line-of-sight channel.

22. The first device includes terminal equipment, the second device includes a location management function, and the third device includes a positioning reference unit. The aforementioned communication channel includes the channel between the terminal device and the network device. The second device according to any one of claims 10 to 12.

23. In the first device, a classification model is used to determine the classification result of the communication channel, at least partially based on channel measurement information regarding the communication channel. Determining importance assessment information for indicating the importance of the channel measurement information to facilitate classification labeling when updating the classification model, based at least in part on the type of classification model, wherein determining the importance assessment information is To determine whether the type of the classification model is the first type or the second type, In accordance with the determination that the type of the classification model is the first type, To determine the uncertainty of the aforementioned classification results, To generate the importance assessment information such that it includes at least the uncertainty of the classification result, In accordance with the determination that the type of the classification model is the second type, the importance evaluation information is generated to include at least the channel measurement information, This includes determining importance assessment information, The importance assessment information is transmitted to the second device, Methods that include...

24. The second device receives importance evaluation information from the first device, which indicates the importance of channel measurement information when updating the classification model, and the classification model is used to determine the classification result of the communication channel based on the channel measurement information. Determining whether the importance of the channel measurement information exceeds the importance threshold, In accordance with the determination that the importance of the channel measurement information exceeds the importance threshold, the third device is requested to perform classification labeling of at least the communication channels at locations related to the first device. Methods that include...

25. A means for determining the classification result of a communication channel based at least partially on channel measurement information relating to the communication channel, using a classification model; Means for determining importance assessment information to indicate the importance of the channel measurement information in order to facilitate classification labeling when updating the classification model, based at least in part on the type of classification model, wherein the means for determining the importance assessment information is Means for determining whether the type of the classification model is a first type or a second type, In accordance with the determination that the type of the classification model is the first type, Means for determining the uncertainty of the classification result, Means for generating the importance assessment information such as including at least the uncertainty of the classification result, A means for generating importance evaluation information that includes at least the channel measurement information, in accordance with the determination that the type of the classification model is the second type, Means for determining importance assessment information, including, Means for transmitting the aforementioned importance evaluation information to the second device, A first device comprising the following:

26. A means for receiving importance evaluation information indicating the importance of channel measurement information when updating a classification model from a first device, wherein the classification model is used to determine the classification result of a communication channel based on the channel measurement information, and the means for receiving this information is... Means for determining whether the importance of the channel measurement information exceeds an importance threshold, A means for requesting a third device to perform classification labeling for at least the communication channels at a location related to the first device, in accordance with the determination that the importance of the channel measurement information exceeds the importance threshold, A second device comprising the following:

27. A computer-readable medium storing instructions for causing a device to perform at least the method according to claim 23 or the method according to claim 24.

Images