On-Demand Labeling for Channel Classification Learning

Abstract

An exemplary embodiment of the present disclosure relates to on-demand labeling for channel classification learning. A first device uses a classification model to determine a classification result for a communication channel based at least in part on channel measurement information for the communication channel, and determines importance evaluation information indicating an importance of the channel measurement information in updating the classification model based at least in part on a type of the classification model. The first device transmits the importance evaluation information to a second device. In response to determining that the importance of the channel measurement information exceeds an importance threshold, the second device causes a third device to perform classification labeling for at least the communication channel for a location associated with the first device.

Description

[Technical Field]

[0001] FIELD Embodiments of the present disclosure relate generally to the field of telecommunications, and more particularly to methods, apparatus, devices, and computer-readable storage media for on-demand labeling for channel classification learning. [Background technology]

[0002] Location awareness is a fundamental aspect of wireless communication networks, enabling a myriad of location-enabled services in a variety of applications. The integration and utilization of location information in everyday applications will grow significantly as technology advances in sophistication.

[0003] Many positioning technologies, such as those relying on time of arrival (TOA), time difference of arrival (TDOA), and angle of arrival (AOA), require line-of-sight (LOS) propagation between a reference point (e.g., network equipment) and the mobile device being positioned. However, in indoor and outdoor non-line-of-sight (NLOS) propagation, the inability to distinguish between multiple reflections of radio frequency (RF) propagation from various angles of arrival and various delay spreads significantly degrades positioning accuracy. On the other hand, artificial intelligence (AI) algorithms inherently excel in accuracy and efficiency for fingerprint-style location estimation, regardless of LOS / NLOS. Therefore, classifying channel propagation is important, at least in part because it influences the selection of a positioning approach. Summary of the Invention

[0004] The scope of protection sought for various embodiments of the present invention is defined by the independent claims. The present embodiments / examples and features herein that do not fall within the scope of the independent claims are to be construed as exemplary embodiments useful for understanding various embodiments of the present invention. It should be noted that the terms "embodiment" or "example" should be adapted accordingly to the terms used in this application, i.e., when the term "embodiment" is used, the "embodiment" is described herein, and when the term "example" is used, the "example" is described herein.

[0005] To the extent that any embodiments fall within the scope of the claims, they should be construed as examples useful in understanding various embodiments of the present disclosure.

[0006] In a first aspect, a first device is provided comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the first device to at least: determine a classification result for a communication channel based at least in part on channel measurement information for the communication channel using a classification model; determine importance rating information indicating an importance of the channel measurement information in updating the classification model based at least in part on a type of the classification model; and transmit the importance rating information to a second device.

[0007] In a second aspect, a second device is provided, the second device comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the second device to at least receive, from the first device, importance assessment information indicating importance of channel measurement information in updating a classification model, the classification model being used to determine a classification result of a communication channel based on the channel measurement information; determine whether the importance of the channel measurement information exceeds an importance threshold; and, in accordance with a determination that the importance of the channel measurement information exceeds the importance threshold, cause a third device to perform classification labeling of at least the communication channel at a location associated with the first device.

[0008] In a third aspect, a method is provided that includes: determining, at a first device, a classification result for a communication channel based at least in part on channel measurement information for the communication channel using a classification model; determining, at least in part on a type of the classification model, importance assessment information to indicate an importance of the channel measurement information in updating the classification model; and transmitting the importance assessment information to a second device.

[0009] In a fourth aspect, a method is provided, including: receiving, at a second device, from a first device, importance assessment information indicating importance of channel measurement information when updating a classification model, the classification model being used to determine a classification result of a communication channel based on the channel measurement information; determining whether importance of the channel measurement information exceeds an importance threshold; and causing a third device to perform classification labeling of at least the communication channel according to a determination that the importance of the channel measurement information exceeds the importance threshold.

[0010] In a fifth aspect, a first device is provided, comprising: means for determining a classification result of a communication channel using a classification model based at least in part on channel measurement information about the communication channel, means for determining importance evaluation information indicating an importance of the channel measurement information when updating the classification model based at least in part on a type of the classification model, and means for transmitting the importance evaluation information to a second device.

[0011] In a sixth aspect, a second device is provided, the second device comprising: means for receiving, from the first device, importance assessment information indicating importance of channel measurement information when updating a classification model, the classification model being used to determine a classification result of a communication channel based on the channel measurement information; means for determining whether importance of the channel measurement information exceeds an importance threshold; and means for causing a third device to perform classification labeling for at least the communication channel at a location associated with the first device according to a determination that the importance of the channel measurement information exceeds the importance threshold.

[0012] In a seventh aspect, there is provided a computer readable medium comprising instructions stored thereon for causing an apparatus to perform at least the method according to the first aspect.

[0013] In an eighth aspect, there is provided a computer readable medium comprising instructions stored thereon for causing an apparatus to perform at least the method according to the second aspect.

[0014] It should be understood that the Summary is not intended to identify key features or essential features of the embodiments of the present disclosure, nor is it intended to be used to limit the scope of the present disclosure. Other features of the present disclosure will become readily apparent through the following description. [Brief explanation of the drawings]

[0015] Some exemplary embodiments will now be described with reference to the accompanying drawings. [Figure 1] FIG. 1 illustrates an example of a communications environment in which exemplary embodiments of the present disclosure may be implemented. [Figure 2] FIG. 2 illustrates a signaling flow for communication according to some exemplary embodiments of the present disclosure. [Figure 3A] FIG. 3A illustrates an example of a first type of classification model in some exemplary embodiments of the present disclosure. [Figure 3B] FIG. 3B illustrates an example of a first type of classification model in some exemplary embodiments of the present disclosure. [Figure 4] FIG. 4 illustrates a flowchart of a process for determining importance rating information in accordance with some exemplary embodiments of the present disclosure. [Figure 5] FIG. 5 is a diagram illustrating an example of a second type classification model and a reference classification model generated therefrom, in some exemplary embodiments of the present disclosure. [Figure 6] FIG. 6 shows a flowchart of a process for determining importance rating information in accordance with some further exemplary embodiments of the present disclosure. [Figure 7A] FIG. 7A illustrates model performance gains in some exemplary embodiments of the present disclosure compared to traditional model learning approaches. [Figure 7B] FIG. 7B illustrates model performance gains in some exemplary embodiments of the present disclosure compared to traditional model learning approaches. [Figure 8] FIG. 8 illustrates a flowchart of a method implemented at a first device, according to some exemplary embodiments of the present disclosure. [Figure 9] FIG. 9 illustrates a flowchart of a method implemented in a second device, according to some exemplary embodiments of the present disclosure. [Figure 10] FIG. 10 is a simplified block diagram of an apparatus suitable for practicing exemplary embodiments of the present disclosure. [Figure 11] 11 shows a block diagram of an exemplary computer-readable medium in accordance with some exemplary embodiments of the present disclosure. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. DETAILED DESCRIPTION OF THE INVENTION

[0016] The principles of the present disclosure will now be described with reference to some exemplary embodiments. It should be understood that these embodiments are provided for illustrative purposes to help those skilled in the art understand and practice the present disclosure, and are not intended to imply any limitations on the scope of the present disclosure. The embodiments described herein can be implemented in various ways other than those described below.

[0017] In the following description and claims, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs.

[0018] References in this disclosure to "one embodiment," "embodiment," "exemplary embodiment," etc. indicate that the described embodiment may include a particular feature, structure, or characteristic, but not all embodiments need include the particular feature, structure, or characteristic. Moreover, such phrases do not necessarily refer to the same embodiment. Furthermore, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is understood that it is within the knowledge of one skilled in the art to affect such feature, structure, or characteristic in connection with other embodiments, whether or not explicitly stated.

[0019] Although terms such as "first," "second," etc. may be used herein to describe various elements, it should be understood that these elements are not limited by these terms. These terms are used merely to distinguish one element from another. For example, a first element could be termed a second element, and similarly, a second element could be termed a first element, without departing from the scope of the exemplary embodiments. As used herein, the term "and / or" includes any and all combinations of one or more of the listed terms.

[0020] As used herein, "at least one of a list of two or more elements," "at least one of a list of two or more elements," and similar expressions where a list of two or more elements is joined by "and" or "or" mean at least any one of the elements, or at least any two or more of the elements, or at least all of the elements.

[0021] The terminology in the examples is for the purpose of describing particular embodiments and is not intended to limit the example embodiments. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly dictates otherwise. It should be further understood that as used herein, the terms "comprises," "comprising," "has," "having," "includes," and / or "including" specify the presence of stated features, elements, and / or components, etc., but do not exclude the presence or addition of one or more other features, elements, components, and / or combinations thereof.

[0022] As used in this application, the term "circuit" means (a) hardware-only circuit implementations (e.g., analog and / or digital-only implementations); (b) a combination of hardware circuitry and software (if applicable); (i) a combination of analog and / or digital hardware circuitry and software / firmware; (ii) software (including digital signal processors), hardware processor portions with software and memory that cooperate to cause a device, such as a mobile phone or server, to perform various functions; (c) A hardware circuit or processor, such as a microprocessor or part of a microprocessor, that requires software (e.g., firmware) to operate, but the software may be absent when not required for operation; It may refer to one or more, or all, of the following:

[0023] This definition of circuit applies to all uses of the term in this application, including any claims. As a further example, as used herein, the term circuit also covers simply a hardware circuit or processor (or processors) or part of a hardware circuit or processor and its (or their) accompanying software and / or firmware implementation. The term circuit also covers, for example, a baseband or processor integrated circuit for a mobile device, or a similar integrated circuit in a server, cellular network equipment, or other computing device or network equipment, if applicable to particular claim elements.

[0024] As used herein, the term "communication network" refers to a network conforming to any suitable communication standard, such as New Radio (NR), Long Term Evolution (LTE), LTE-Advanced (LTE-A), Wideband Code Division Multiple Access (WCDMA), High-Speed Packet Access (HSPA), or Narrowband Internet of Things (NB-IoT). Furthermore, communications between terminal devices and network devices in a communication network may be performed according to any suitable generation of communication protocols, including, but not limited to, first-generation (1G), second-generation (2G), 2.5G, 2.75G, third-generation (3G), fourth-generation (4G), 4.5G, fifth-generation (5G) communication protocols, and / or other protocols currently known or developed in the future. Embodiments of the present disclosure may be applied to various communication systems. Given the rapid development of communications, there will, of course, be future communication technologies and systems in which the present disclosure may be embodied. The scope of the present disclosure should not be considered limited to only the aforementioned systems.

[0025] As used herein, the term "network equipment" refers to a node of a communication network through which terminal equipment accesses the network and receives services therefrom. Network equipment can be a base station (BS) or access point (AP), e.g., a Node B (NodeB or NB), an evolved Node B (eNodeB or eNB), a NR NB (also referred to as a gNB), a remote radio unit (RRU), a radio header (RH), a remote radio head (RRH), a relay, an integrated access backhaul (IAB) node, a low-power node such as a femto or pico node, a non-terrestrial network (NTN) or non-terrestrial network equipment such as a satellite network equipment, a low earth orbit (LEO) satellite, a geostationary earth orbit (GEO) satellite, an airborne network, or the like, depending on the terminology and technology applied. In some exemplary embodiments, a radio access network (RAN) split architecture includes a centralized unit (CU) and a distributed unit (DU) in an IAB donor node. The IAB node includes a mobile terminal (IAB-MT) portion that acts like a UE towards a parent node, and the DU portion of the IAB node acts like a base station towards a next-hop IAB node.

[0026] The term "terminal equipment" refers to any terminal equipment capable of wireless communication. By way of example and not limitation, a terminal equipment may also be referred to as a communication device, a user equipment (UE), a subscriber station (SS), a mobile subscriber station, a mobile station (MS), or an access terminal (AT). Terminal equipment includes, but is not limited to, mobile phones, cellular phones, smartphones, voice over IP (VoIP) phones, wireless local loop phones, tablets, wearable terminal equipment, personal digital assistants (PDAs), portable computers, desktop computers, image capture terminal equipment such as digital cameras, gaming terminal equipment, music storage devices, playback appliances, in-vehicle wireless terminal equipment, wireless endpoints, mobile stations, laptop embedded devices (LEEs), laptop mounted devices (LMEs), USB dongles, smart devices, wireless customer premises equipment (CPEs), Internet of Things (IoT) devices, wearables such as watches, head-mounted displays (HMDs), vehicles, drones, medical devices and applications (e.g., remote surgery), industrial devices and applications (e.g., robots and / or other wireless devices operating in the context of industrial and / or automated processing chains), consumer electronics devices, devices operating in commercial and / or industrial wireless networks, etc. Terminal equipment may also correspond to the mobile terminal (MT) portion of an IAB node (e.g., a relay node). In the following description, the terms "terminal equipment", "communication equipment", "terminal", "user equipment" and "UE" may be used interchangeably.

[0027] In this embodiment, the terms "resource," "transmission resource," "resource block," "physical resource block" (PRB), "uplink resource," or "downlink resource" may refer to any resource for performing communication, for example, any resource for performing communication between a terminal device and a network device, such as a time domain resource, a frequency domain resource, a space domain resource, a code domain resource, or any other resource that enables communication. Hereinafter, unless explicitly stated, both frequency domain and time domain resources are used as examples of transmission resources to describe some exemplary embodiments of the present disclosure. It should be noted that the exemplary embodiments of the present disclosure are equally applicable to other resources in other domains.

[0028] As used herein, the term "model" refers to an association between input and output learned from training data, such that a corresponding output can be generated for a given input after learning. The generation of a model may be based on machine learning (ML) techniques. Machine learning techniques can also be referred to as artificial intelligence (AI) techniques. In general, a machine learning model can be constructed that receives input information and makes a prediction based on the input information. For example, a classification model can predict the category of the input information from a predetermined number of categories. As used herein, a "model" may also be referred to as a "machine learning model," a "learning model," a "machine learning network," or a "learning network," which are used interchangeably herein.

[0029] Deep learning (DL) is a machine learning algorithm that uses multiple layers of processing units to process inputs and provide corresponding outputs. A neural network (NN) model is an example of a deep learning-based model. A neural network can process inputs and provide corresponding outputs, and typically includes an input layer, an output layer, and one or more hidden layers between the input and output layers. Neural networks used in deep learning typically include multiple hidden layers to increase the depth of the network. Each layer of a neural network is connected in sequence, with the output of the previous layer providing the input for the next layer, the input layer receiving the neural network's input, and the output of the output layer being considered the neural network's final output. Each layer of a neural network includes one or more nodes (also called processing nodes or neurons), and each node processes the input from the previous layer.

[0030] In general, model lifecycle management typically includes three stages: a training stage, a validation stage, and an application stage (also referred to as an inference stage). During the training stage, a given machine learning model is repeatedly trained (or optimized) using a large amount of training data until the model can obtain consistent inferences from the training data, similar to those made by human intelligence. During training, a set of model parameter values is repeatedly updated until a learning goal is reached. Through the training process, the machine learning model can be considered to be able to learn the association between input and output (also referred to as input-output mapping) from the training data. During the validation stage, validation inputs are applied to the trained machine learning model to test whether the model can provide correct outputs and determine the model's performance. Generally, the validation stage can be considered a stage in the training process or can be omitted. During the inference stage, the resulting machine learning model can be used to process real-world model inputs based on the set of parameter values obtained from the training process and determine the corresponding model output. In some cases, a retraining or update stage is included in model lifecycle management to enable the evolved model to achieve better performance.

[0031] [Example environment] 1 illustrates an exemplary communication environment 100 in which exemplary embodiments of the present disclosure may be implemented. The communication environment 100 involves multiple communication devices, including one or more first devices (110-1, 110-2, 110-3), a second device 120, a third device 130, and a fourth device 140. For purposes of explanation, the first devices (110-1, 110-2, and 110-3) will be collectively or individually referred to as first devices 110.

[0032] 1 is for illustrative purposes only, without implying any limitation. Communication environment 100 may include any suitable number of devices adapted to implement embodiments of the present disclosure. Although not shown, it is understood that one or more additional devices may be involved in communication environment 100.

[0033] Communications in communication environment 100 may be conducted according to any suitable communications protocol(s), including, but not limited to, cellular communications protocols such as first generation (1G), second generation (2G), third generation (3G), fourth generation (4G), and fifth generation (5G), wireless local network communications protocols such as Institute of Electrical and Electronics Engineers (IEEE) 802.11, and / or other protocols now known or developed in the future. Further, communications may utilize any suitable wireless communications technology, including, but not limited to, code division multiple access (CDMA), frequency division multiple access (FDMA), time division multiple access (TDMA), frequency division duplex (FDD), time division duplex (TDD), multiple input multiple output (MIMO), orthogonal frequency division multiple access (OFDM), discrete Fourier transform spread OFDM (DFT-s-OFDM), and / or other technologies now known or developed in the future.

[0034] In communication environment 100, first device 110 and fourth device 140 can communicate with each other. In the example of Figure 1, first device 110 is illustrated as terminal equipment and fourth device 140 is illustrated as network equipment such as a transmission / reception point (TRP). In some exemplary embodiments, when the first device is terminal equipment and fourth device 140 is network equipment, the link from fourth device 140 to first device 110 is referred to as a downlink (DL) and the link from first device 110 to fourth device 140 is referred to as an uplink (UL).

[0035] Positioning techniques may be applied to obtain location information of the first device 110. In some exemplary embodiments, the positioning techniques may be based on DL and DL+UL position measurements obtained at the first device 110 for UE-assisted positioning, or UL and DL+UL measurements obtained at the fourth device 140 for network-assisted positioning. In some cases, different positioning techniques may be applied to ensure positioning accuracy depending on the category of the communication channel between the first device 110 and the fourth device 140. For example, different positioning methods may be applied by identifying whether the communication channel has line-of-sight (LOS) propagation or non-line-of-sight (NLOS) propagation. Thus, identifying the category of the communication channel is important because it affects at least the accuracy of the positioning estimate.

[0036] In order to predict the classification result of the communication channel between the first device 110 and the fourth device 140, it is proposed to introduce one or more classification models into the first device 110. The classification models can be constructed based on AI technology. The processing by the classification models is as follows:

number

number

[0037] The first device 110 may detect a reference signal propagated from the fourth device 140 over the respective communication channel to obtain channel measurement information. In some exemplary embodiments, the fourth device 140 may transmit the reference signal based on configuration or signaling from the second device 120. In some exemplary embodiments, the communication channel may be classified by a classification model as either a LOS channel (with LOS propagation) or an NLOS channel (with NLOS propagation).

[0038] In some exemplary embodiments, the second device 120 may maintain and manage the classification model used by the first device 110. The second device 120 may include a location server or a controller. In some exemplary embodiments, the second device 120 may include a network element in a core network (CN) configured for location management. In some exemplary embodiments, the second device 120 may include a location management function (LMF), although other terms may be used.

[0039] The accuracy of a classification model depends on the training data. A prerequisite for supervised learning models is that the training data must be pre-labeled. For classification models configured for channel classification, the labeled training data includes sample channel measurement information as sample model inputs and ground-truth classification results as ground-truth model labels. Typically, on-site measurement and labeling require the support of external equipment.

[0040] For example, a third device 130 in communication environment 100 may be configured to facilitate on-site measurements and classification labeling. The third device 130 is typically capable of determining its location. In some exemplary embodiments, the third device 130 may include a positioning reference unit (PRU), although other terms may be used. This third device 130 may be requested by second device 120 to perform on-site measurements and determine a correct classification result for channel measurement information measured at its location. While one third device is illustrated, it should be noted that there may be multiple third devices that may be requested to perform classification labeling.

[0041] In some cases, a classification model may evolve or be fine-tuned to achieve better performance (e.g., higher accuracy) even after being deployed to the first device. Such model evolution may require the addition of labeled training data. However, the first device lacks the knowledge to appropriately determine and request on-site measurements and labeling. Requesting one or more third devices to perform labeling does not necessarily mean that the collected training data will help improve model performance. Requesting a third device to perform labeling in areas where the current classification model can already provide descent estimation results or overlooking blind spots where more training samples are required for model training would result in wasted resources.

[0042] For efficient training data updating and model retraining, at a minimum, properly labeled training data that is thought to be informative and useful for improving model performance is required.

[0043] Example of operation principle and signaling flow According to some exemplary embodiments of the present disclosure, a solution for on-demand labeling for channel classification learning is provided. In this solution, a first device determines importance evaluation information representing the importance of channel measurement information in updating the classification model based on the type of the classification model. The importance evaluation information is transmitted to a second device. The second device compares the importance with an importance threshold. If the importance of the channel measurement information exceeds the importance threshold, the second device causes a third device to perform classification labeling for the communication channel of the first device. In this way, by evaluating the importance of the channel measurement information for improving the classification model, on-demand labeling can be performed in a collaborative manner. This reduces the labeling overhead of the third device, and enables efficient updating of labeled training data for model improvement.

[0044] In some exemplary embodiments, more training data can be obtained from the classification labeling to update or learn the classification model, in which case efficient model learning can achieve optimal learning performance with a small set of well-labeled training data.

[0045] Exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings.

[0046] Reference is now made to FIG. 2, which illustrates a signaling flow 200 for communication in accordance with some example embodiments of the present disclosure. As shown in FIG. 2, signaling flow 200 involves a first device 110, a second device 120, and a third device 130. For purposes of explanation, signaling flow 200 will be described with reference to FIG. 1. While one first device 110 and one third device 130 are illustrated in FIG. 2, it should be understood that there may be multiple first devices that perform similar operations as described with respect to first device 110 below, and multiple third devices that perform similar operations as described with respect to third device 130 below.

[0047] The first device 110 determines 205 a classification result for the communication channel based at least in part on the channel measurement information for the communication channel. The classification model is applied to determine the classification result for the communication channel.

[0048] In some exemplary embodiments, fourth device 140 may transmit a reference signal, and first device 110 may measure the reference signal propagated over a communication channel between first device 110 and fourth device 140 to obtain channel measurement information. In some exemplary embodiments, first device 110 may include terminal equipment, and fourth device 140 may include network equipment.

[0049] The channel measurement information may include one or more types of information useful for characterizing a communication channel. In some exemplary embodiments, the channel measurement information may include a channel impulse response (CIR), a channel state information (CSI), a received signal strength indicator (RSSI), a reference signal received power (RSRP), and / or other information that can be measured.

[0050] The classification model may be configured to extract representative features of channel measurement information in a high-dimensional feature space through machine learning and classify communication channels using the features. The classification model may be configured to have multiple potential channel categories into which communication channels may be classified. In some exemplary embodiments, the classification model may perform two-category classification to classify communication channels into either a first channel category or a second channel category. In some exemplary embodiments, the multiple channel categories may include LOS channels and NLOS channels. The classification result may indicate a predicted probability that a communication channel will be classified as a LOS channel or a NLOS channel. Other channel categories may also be defined depending on actual applications, but this is not limited within the scope of the present disclosure.

[0051] In addition to classifying the communication channel, the first device 110 also determines whether and / or how to report assistance information to facilitate on-demand classification labeling. Specifically, the first device 110 determines (210) importance rating information indicating the importance of the channel measurement information in updating the classification model based at least in part on the type of classification model. The first device 110 transmits (215) the importance rating information to the second device 120.

[0052] From the perspective of at least lifecycle management of the classification model applied by the first device 110, employing on-demand classification labeling in training is beneficial. In an exemplary embodiment of the present disclosure, the first device 110 determines importance rating information associated with channel measurement information, which enables the second device 120 to trigger the third device 130 to perform classification labeling if the channel measurement information is found to be important in updating the classification model. As used herein, channel measurement information that is important in updating the classification model may involve cases where the channel measurement information is useful and provides new features not currently captured by the classification model. In this case, classification labeling of the corresponding communication channels can provide new and useful training data that can help fine-tune the classification model, for example, to correctly classify communication channels with similar characteristics.

[0053] Different importance assessment information may be determined for different types of classification models. To measure whether channel measurement information is important in updating a classification model, in some exemplary embodiments, importance assessment information may be determined based on the uncertainty or reliability of a classification result determined by a model based on channel measurement information. In some exemplary embodiments, the uncertainty of the classification result may be determined, and the uncertainty may represent the degree to which the classification model is confident or doubtful about the classification result. Uncertainty may also be referred to as the degree of doubt. Alternatively, the certainty, reliability, or confidence of the classification result may be measured, as opposed to "uncertainty."

[0054] Generally, it is not possible to directly identify the root cause of a classification error between "ambiguity in the channel itself" and "immaturity of the classification model estimation." However, since the immaturity of the classification model is not taken into account, the second device 120 is likely to cause inappropriate follow-up operations. For example, if the classification model is believed to be confident in its classification results but the classification ambiguity is caused by the ambiguity of the channel itself, this may affect the selection of the follow-up positioning approach between geometric-type (e.g., TDOA, AOA, AOD) schemes and fingerprint-type schemes. On the other hand, if the classification ambiguity is caused by the immaturity of the classification model before it is fully trained or fine-tuned, this may affect the subsequent training data enhancement and model update from the perspective of model lifecycle management, i.e., more labeled training data needs to be collected for model fine-tuning.

[0055] For example, in FIG. 1, the first device 110-1 has x i Obtain the channel measurement information expressed as f AI-s The classification result showing that the communication channel is a LOS channel using the classification model represented as ( ),

number

number

number

[0056] In view of the above, it is beneficial to evaluate the uncertainty or reliability of the classification result provided by the current classification model. In some exemplary embodiments, the importance of the channel measurement information may be determined based on the uncertainty of the classification result. The importance evaluation information may be generated to include at least the uncertainty of the classification result. In some exemplary embodiments, a higher uncertainty of the classification result may correspond to a higher importance of the channel measurement information, which means that the communication channel or the channel measurement information may be so important that classification labeling is required.

[0057] In some example embodiments, the uncertainty for different types of classification models can be determined with different approaches by considering the classification scheme implemented by the classification model. Some types of classification models may require relatively high overhead to calculate the uncertainty, while some other types of classification models may require relatively low overhead.

[0058] For some types of classification models (e.g., a first type classification model) that require high overhead to calculate uncertainty, the first device 110 may determine the classification uncertainty and generate importance assessment information that includes at least the uncertainty. In some exemplary embodiments, the uncertainty of the classification result output from the first type model may be determined based on intermediate output information obtainable from the model, and thus, no reconstruction of the classification model is required. In such cases, the calculation of the uncertainty may not introduce high overhead and thus may be performed at the first device 110.

[0059] For some other types of classification models (e.g., a second type classification model) that require low overhead, the first device 110 can request another device, such as the second device 120, to assist in determining the certainty. In some exemplary embodiments, the certainty of the classification result output from the second type classification model may be determined by reconstructing the classification model, which may cause high overhead and resource consumption. In those embodiments, the first device 110 can provide at least channel measurement information to the second device 120. In this manner, the important evaluation information may include at least the channel measurement information. The first device 110 can send an assistance request including the important evaluation information to the second device 120. Upon receiving the assistance request, the second device 120 can determine the uncertainty based on the channel measurement information.

[0060] Some exemplary embodiments for calculating uncertainty for different types of classification models are described in detail below with reference to FIGS.

[0061] In some exemplary embodiments, in addition to or as an alternative to factors contributing to the uncertainty of the classification result, the importance of the channel measurement information can be determined based on other factors that can indicate whether the current communication channel of the first device 110 is useful or important for improving the classification model.

[0062] It has been described above from the perspective of first device 110 that importance assessment information associated with channel measurement information for a communication channel is reported to the second device. In some demonstrative embodiments, fourth device 140 may obtain the channel measurement information, for example, by receiving channel measurement information from first device 110 or by measuring a reference signal transmitted from first device 110. In such a case, if a classification model is deployed at fourth device 140, fourth device 140 may perform operations similar to those described herein with respect to first device 110. In other words, network equipment may also transmit importance assessment information to facilitate classification labeling for updating the classification model.

[0063] At the second device 120, the importance assessment information is received so that the importance of the channel measurement information obtained at the first device 110 can be determined (220). The second device 120 determines (225) whether the importance of the channel measurement information exceeds an importance threshold. The importance threshold may be any predetermined level threshold.

[0064] In some exemplary embodiments, if the importance assessment information from the first device 110 includes uncertainty of the classification result, the second device 120 may determine an importance of the corresponding channel measurement information based on the uncertainty and compare the importance to an importance threshold. For example, a higher uncertainty may correspond to a higher importance of the corresponding channel measurement information. In some examples, the uncertainty may be taken into account as an importance of the corresponding channel measurement information. In some examples, the importance of the corresponding channel measurement information may be determined based on one or more factors other than the uncertainty. In some exemplary embodiments, if the importance assessment information from the first device 110 includes the channel measurement information itself, the second device 120 may first determine the uncertainty based on the channel measurement information, as described above. The determination of the uncertainty will be described in more detail below.

[0065] If the importance of the channel measurement information exceeds the importance threshold, the second device 120 causes the third device 130 to perform classification labeling (230) on the communication channels at least at a location associated with the first device 110. The importance assessment information reported by the first device 110 enables the second device 120 to evaluate the importance of specific channel measurement information in improving the classification model. The third device 130 may be requested by the second device 120 to perform classification labeling within the area where important channel measurement information is found.

[0066] In some cases, if the importance of the channel measurement information is determined to be below the importance threshold, the second device 120 can discard the importance evaluation information. In this way, classification labeling is not triggered for channel measurement information that is not important in model updating. Both labeling efficiency and model update efficiency are improved.

[0067] The third device 130 performs classification labeling in response to a request from the second device 120 (235). The location where the third device 130 performs classification labeling may be anywhere within the area where the first device 110 is located. In some exemplary embodiments, the second device 120 may select an appropriate third device 130 located near the first device 110 to perform classification labeling. In some exemplary embodiments, the third device 130 may be mobile and may be requested by the second device 120 to move to the area where the first device 110 is located. The third device 130 may determine a correct classification result for a communication channel with the fourth device 140 within the area. The correct classification result may label the communication channel as either a first channel category (e.g., a LOS channel) or a second channel category (e.g., a NLOS channel).

[0068] In some demonstrative embodiments, the third device 130 may perform further measurements on the communication channel between the first device 110 and the fourth device 140 and label the communication channel with a correct classification result. For example, the third device 130 may obtain sample channel measurement information (denoted as “x”) for the communication channel and determine a correct classification result (denoted as “y”) for the sample channel measurement information. The third device 130 may transmit (240) to the second device 120 a classification labeling result including the sample channel measurement information and the corresponding correct classification result pair {x, y}.

[0069] In some exemplary embodiments, within the region in which the third device 130 is moved, the third device 130 may perform classification labeling for other communication channels. The third device 130 may obtain one or more additional pairs of sample channel measurement information and corresponding correct classification results by changing its location and / or the orientation of its antenna within the geographic region in which the first device 110 is located. The classification labeling results transmitted to the second device 120 may include multiple pairs of sample channel measurement information and corresponding correct classification results.

[0070] The second device 120 receives (245) the classification labeling results from the third device 130 and updates (250) the classification model used by at least the first device 110 based on the classification labeling results. In some exemplary embodiments, the second device 120 can update the training dataset using at least one pair of sample channel measurement information and corresponding correct classification results. The second device 120 can trigger an update of the classification model after sufficient training data is collected from the third device 130 and other data sources. For example, the second device 120 can determine whether the size of the newly collected training data exceeds a threshold. If the size exceeds the threshold, an update of the classification model can be triggered. Because the training data is evaluated as important and useful, the updated classification model can be improved to have higher accuracy.

[0071] In some cases, in addition to the classification model used in the first device 110 reporting the importance rating information, the second device 120 may maintain one or more other classification models. Sample channel measurement information and corresponding correct classification results collected by the third device 130 may be shared between the classification models. In other words, the second device 120 may update one or more other classification models based on the sample channel measurement information and corresponding correct classification result(s). The classification models maintained by the second device 120 may be of different types and / or different model configurations, but may all be configured to classify communication channels. Channel measurement information considered important in updating one classification model may also be important and useful in updating other classification models.

[0072] In some exemplary embodiments, if the inputs to the different classification models are not the same (e.g., if different channel measurement information inputs are required), the third device 130 may be requested by the second device 120 to collect different sample channel measurement information for the same communication channel along with the correct classification result.

[0073] The second device 120 may apply any suitable update technique to the classification model, which is not a limitation within the scope of this disclosure.

[0074] In some exemplary embodiments, with one or more classification models updated, the second device 120 may send updates to the classification model(s) to the first device 110 (255). The first device 110 may receive the updates to the classification model(s) and apply the updated classification model(s) for next channel classification (260). In exemplary embodiments, the second device 120 may provide updated classification models previously used by the first device 110. In exemplary embodiments, other updated classification models may also be provided to the first device 110. The first device 110, configured with multiple (updated) classification models, may select one of the models to use depending, for example, on the environment associated with the communication channel.

[0075] [First type of classification model and uncertainty calculation] As mentioned above, the uncertainty of the importance assessment information or the channel measurement information may be determined depending on the type of classification model.

[0076] In some exemplary embodiments, for a first type of classification model, the uncertainty of the classification result output from the model can be determined based on intermediate output information obtainable from the model. As an example, for binary classification, the classification model can determine a first number of model votes (denoted as "N1") for a first channel category and a second number of model votes (denoted as "N2") for a second channel category based on input channel measurement information.

[0077] The classification result may be determined based on the ratio of the first number to the second number (e.g., N1 / N2), where a higher ratio may indicate a higher probability that the communication channel is classified into the first channel category.

[0078] In this type of classification model, the classification result is a "soft" indication of the channel category into which the communication channel falls.AI-s 1, the first devices (110-1 and 110-2) are shown using this type of classification model to perform channel classification. Some examples of this type of classification model may include, but are not limited to, a k-nearest neighbor (KNN) model and a support vector machine (SVM) model.

[0079] 3A and 3B illustrate an example of a first type of classification model according to some exemplary embodiments of the present disclosure. Figures 3A and 3B show a feature space 300 including a plurality of features 302 associated with a first channel category (denoted as "Category 1") and a plurality of features 304 associated with a second channel category (denoted as "Category 2"). The classification scheme applied by the classification model is configured to measure the respective distances between features extracted from the channel measurement information and features in the feature space 300, and select a predetermined number (e.g., K) of features with smallest distances (e.g., the K features with the smallest distances).

[0080] Of the total K selected features, the classification model may count a first number of features associated with a first channel category (i.e., a first number N of model votes for the first channel category) and a second number of features associated with a second channel category (i.e., a second number N of model votes for the second channel category). A ratio of the first number to the second number can be used to determine a probability that the communication channel belongs to the first channel category.

[0081] In the example of FIG. 3A, the channel measurement information x acquired by the first device 110-1 is i The feature 312 of category 1 is close to six features associated with category 1 and one feature associated with category 2, meaning that the probability of the communication channel of the first device 110-1 is 6 / 7. In the example of FIG. 3B, the channel measurement information x jfeature 314 is close to four features associated with category 1 and three features associated with category 2, which means that the probability that the communication channel of the first device 110-1 belongs to the first channel category is 4 / 7.

[0082] 3A and 3B are provided for illustrative purposes without implying the classification approach of the first type of classification model. Some classification models may operate in other ways to determine model votes for two channel categories and output classification results.

[0083] In some example embodiments, the uncertainty of the classification result output by this type of classification model may be determined based on intermediate output information, such as a first number N1 of model votes for a first channel category and a second number N2 of model votes for a second channel category.

[0084] 4 shows a flowchart of a process 400 for determining importance rating information in accordance with some exemplary embodiments of the present disclosure. The process 400 may be performed by the first device 110, for example.

[0085] In block 410, the first device 110 counts a first number N1 of model votes for the first channel classification and a second number N2 of model votes for the second channel classification, both of which may be obtained from the classification model.

[0086] In block 420, the first device 110 determines a degree of difference between the first number N1 and the second number N2, and in block 430, the first device 110 determines an uncertainty based on the degree of difference.

[0087] In the case of model voting for binary classification, if the classification model is more reliable in its estimation, the number of model votes for one channel category may be larger and the number of model votes for the other channel category may be correspondingly smaller. Thus, if the difference between the first number and the second number is large, the classification model is more reliable in its classification result, and therefore the uncertainty of the classification result is low.

[0088] In some exemplary embodiments, the degree of difference between the first number N1 and the second number N2 may be measured based on the larger value between N1 / N2 or N2 / N1, which may be expressed as max(N1 / N2, N2 / N1). In some exemplary embodiments, the sum of N1 and N2 is determined as K, and the degree of difference between the first number N1 and the second number N2 may be measured based on the larger value between the ratio of N1 to K and the ratio of N2 to K, which may be expressed as max(N1, N2) / K. In these cases, if max(N1 / N2, N2 / N1) or max(N1, N2) / K is determined to have a higher value, the uncertainty (expressed as "γ") of the classification result may be determined to be at a higher level. The uncertainty γ in some examples may be determined as γ=max(N1 / N2, N2 / N1) or max(N1, N2) / K. In other embodiments, the uncertainty γ can be determined in other ways based on the degree of difference between N1 and N2.

[0089] In some exemplary embodiments, once the uncertainty of the classification result is determined, the first device 110 may generate importance rating information to include at least the determined uncertainty.

[0090] In some exemplary embodiments, the first device 110 may transmit importance assessment information if a relatively high uncertainty is found. As shown in FIG. 4, in block 440, the first device 110 may determine whether the uncertainty γ exceeds an uncertainty threshold represented as γ. If the uncertainty γ exceeds the uncertainty threshold γ, in block 450, the first device 110 determines to transmit importance assessment information including the uncertainty to the second device 120. If the uncertainty γ does not exceed the uncertainty threshold γ, in block 460, the first device 110 determines that transmission of importance assessment information is not necessary.

[0091] As mentioned above, a high uncertainty may correspond to a high importance of the channel measurement information in updating the classification model. By reporting uncertainties that exceed an uncertainty threshold to the second device 120, the transmission overhead between the first device 110 and the second device 120 can be further reduced.

[0092] In an exemplary embodiment, if applicable, first device 110 may determine the importance of the channel measurement information based on the uncertainty and possibly some other factors to generate importance assessment information. First device 110 may determine whether the importance exceeds an importance threshold and decide to transmit the importance assessment information if the importance exceeds a corresponding importance threshold. The importance threshold may be applied by second device 120.

[0093] In some exemplary embodiments, the uncertainty threshold or importance threshold may be set by the second device 120 to control whether the channel measurement information is rated as “important” or how “uncertain” the classification model is about its classification results. In some exemplary embodiments, the uncertainty threshold or importance threshold may be determined based on the accuracy level of the classification model.

[0094] For example, if a classification model has a low accuracy level (e.g., 55%) in the initial stage, it generally means that the model may not distinguish channel categories well. The uncertainty threshold or importance threshold may be set to a relatively low value so that more channel measurement information is evaluated as important and the third device 130 can collect more training data for model updating.

[0095] In some exemplary embodiments, the uncertainty threshold γ may be updated based on updates to the classification model. As the classification model is updated and becomes more mature, its accuracy level may increase, and the uncertainty threshold or importance threshold may also be set to a larger value. For example, if the accuracy level of the classification model increases to 80%, the classification model may become more confident in its classification results, and the uncertainty threshold or importance threshold may also be increased.

[0096] It should be appreciated that in some other exemplary embodiments, instead of calculating the uncertainty of the first type classification model, the first device 110 may alternatively generate and transmit importance assessment information including channel measurement information to the second device 120, requesting the second device 120 to perform the calculation. In those embodiments, the operations at blocks 410, 420, and 430 of process 400 may be performed at the second device 120.

[0097] [Second classification model and uncertainty calculation] In some exemplary embodiments, the second type of classification model is a type of model having uncertainty in the classification result determined by reconstructing the classification model. One example of such a classification model is a deep neural network (DNN) model, which generally provides a hard output (e.g., 0 or 1) indicating whether a communication channel is classified into the first channel category or the second channel category.

[0098] 5 illustrates an example of a second type classification model 510 and a reference classification model generated therefrom, in accordance with some exemplary embodiments of the present disclosure. The classification model 510 may be in the form of a DNN model.

[0099] As shown in FIG. 5, the classification model 510 is composed of an input layer 502, one or more hidden layers 504, and an output layer 506, each of which is composed of multiple computational units (also called neurons). The computational units of a layer are connected to the computational units of the next layer. In some exemplary embodiments, the computational units of a layer may be connected to one or more other computational units of the same layer. Channel measurement information is input to the input layer 502 for processing, and the information is propagated through the hidden layer(s) 504 according to the layer connections. The classification results of the channel measurement information are output from the output layer 506. Examples of layers included in the model include convolutional layers, batch normalization layers, activation functions, pooling layers, fully connected layers, long-short-term memory (LSTM) layers, and other types of layers.

[0100] Note that the number of arithmetic units and layers in the classification model 510 is not related to the exemplary embodiment of the present disclosure and may be any value. The structure of the model is also not limited, and the connections between the arithmetic units may be recursive or bidirectional. Any model applicable to channel classification may be used.

[0101] For DNN-type models and similar models, there is no general solution yet for directly measuring the uncertainty of prediction results based on model output or real-time information. It is worth noting that in classification models, even if the model outputs a probability vector, it may not be directly usable to indicate the uncertainty of the classification result. That is, a classification model may be uncertain in its predictions even if it has a high output probability. In some exemplary embodiments, it is proposed to reconstruct the classification model by slightly modifying the model, generate multiple reference classification models, and determine the uncertainty based on the multiple reference classification models.

[0102] 6 shows a flowchart of a process 600 for determining importance assessment information in some further exemplary embodiments of the present disclosure. Process 600 may be performed, for example, by second device 120. In these embodiments, second device 120 receives importance assessment information consisting of channel measurement information from first device 110. To measure the importance of the channel measurement information, second device 120 may determine the uncertainty of the classification result output by the classification model for the channel measurement information.

[0103] In block 610, the second device 120 generates multiple reference classification models by reconfiguring the classification model. In some exemplary embodiments, the second device 120 may slightly modify the classification model by applying random neural connection dropout to the classification model. Specifically, the second device 120 may randomly drop out some neural connections between computing units in the classification model to obtain the reference classification model. In some exemplary embodiments, the second device 120 may apply a Gaussian process to determine which neural connections to drop out from the classification model. The second device 120 may generate multiple different reference classification models through the dropout method. The second device 120 may also apply other dropout methods to generate the reference classification models.

[0104] In the example of FIG. 5, the second device 120 may generate P reference classification models (512-1, 512-2, . . . , 512-P) (collectively or individually referred to as reference classification models 512), where P is an integer greater than 1. The reference classification models 512 may be f AI-p It may be represented as (.).

[0105] In block 620, the second device 120 determines a plurality of reference classification results based on the channel measurement information using a plurality of reference classification models. The second device 120 uses the channel measurement information provided by the first device 110 to classify the channel measurement information into respective reference classification models f with p=1, 2, . . . , P. AI-p (.) The second device 120 has p=1, 2, . . . , P

number

[0106] In block 630, the second device 120 determines the uncertainty of the classification result based on the variance of the multiple reference classification results. If the variance of the multiple reference classification results is relatively high, this means that the reference classification model is inconsistent in classifying the channel measurement information. In this case, the uncertainty of the classification result is determined to be at a relatively high level, and the channel measurement information may be determined to be useful and important in updating the original classification model.

[0107] This allows us to update the classification model to be stable and more reliable in classifying similar communication channels even if the model structure is slightly changed (e.g., by excluding some connections).

[0108] In some exemplary embodiments, the uncertainty threshold or importance threshold applied to the second type classification model may be set in a manner similar to that applied to the first type classification model described above. In some exemplary embodiments, the uncertainty threshold or importance threshold applied to the first type classification model and the second type classification model may be set as the same threshold or different thresholds.

[0109] It should be understood that in some other exemplary embodiments, instead of requesting the second device 120 to calculate the uncertainty for the second type of classification model, the first device 110 may alternatively determine the uncertainty locally by performing similar operations in process 600, and then generate and send importance assessment information including the uncertainty to the second device 120.

[0110] [Comparison of performance improvements] 7A and 7B illustrate gains in model performance according to some exemplary embodiments of the present disclosure over conventional model training approaches. According to conventional model training approaches, a third device may be randomly requested by a second device to perform field measurements and classification labeling without assistance. According to exemplary embodiments of the present disclosure, through cooperation with a first device, the second device can trigger classification labeling if significant and useful channel measurement information is found.

[0111] 7A shows an accuracy trend curve 710 for a conventional model learning approach and an accuracy trend curve 720 for the proposed approach in some exemplary embodiments of the present disclosure. The two trend curves show the increase in accuracy versus the amount of labeled training data for learning a first type of classification model. To approach a satisfactory accuracy of approximately 0.75, the amount of labeled training data required to make the classification model satisfactory can be reduced by roughly 60% using the proposed approach compared to the conventional approach.

[0112] 7B shows an accuracy trend curve 712 for a conventional model learning approach and an accuracy trend curve 722 for the proposed approach in some exemplary embodiments of the present disclosure. The two trend curves show the increase in accuracy versus the amount of labeled training data for learning a first type of classification model. As shown, to achieve similar performance in terms of classification accuracy, the amount of labeled training data required for the classification model can be reduced by approximately 50% using the proposed approach in some exemplary embodiments of the present disclosure compared to the conventional approach.

[0113] [Example of method] 8 shows a flowchart of an example method 800 implemented at the first device in accordance with some exemplary embodiments of the present disclosure. For purposes of explanation, the method 800 will be described from the perspective of the first device 110 of FIG.

[0114] At block 810, the first device 110 uses the classification model to determine a classification result for the communication channel based at least in part on the channel measurement information for the communication channel.

[0115] At block 820, the first device 110 determines importance rating information indicating the importance of the channel measurement information in updating the classification model based at least in part on the type of the classification model.

[0116] In block 830 , the first device 110 transmits the importance rating information to the second device 120 .

[0117] In some exemplary embodiments, determining the importance assessment information includes determining whether the type of the classification model is a first type or a second type; determining an uncertainty of the classification result according to determining that the type of the classification model is the first type and generating the importance assessment information to configure at least the uncertainty of the classification result; and generating the importance assessment information to configure at least the channel measurement information according to determining that the type of the classification model is the second type.

[0118] In some exemplary embodiments, the first type of classification model is a type of model having classification result uncertainty determined without reconstructing the classification model, and in some exemplary embodiments, the second type of classification model is a type of model having classification result uncertainty determined by reconstructing the classification model.

[0119] In some exemplary embodiments, the classification result indicates whether the communication channel is classified into a first channel category or a second channel category, and the classification result determined using the first type of classification model is based on a ratio of a first number of model votes for the first channel category to a second number of model votes for the second channel category. In some exemplary embodiments, determining the uncertainty includes determining a degree of difference between the first number and the second number and determining the uncertainty based on the degree of difference.

[0120] In some exemplary embodiments, transmitting the importance assessment information to the second device includes transmitting the importance assessment information to the second device in accordance with a determination that the determined uncertainty exceeds an uncertainty threshold.

[0121] In some exemplary embodiments, the method 800 further includes receiving an uncertainty threshold from the second device.

[0122] In some exemplary embodiments, the classification result is determined based on a predicted probability provided by a second type of classification model to indicate whether the communication channel is classified into the first channel category or the second channel category.

[0123] In some exemplary embodiments, the method 800 further includes receiving at least an update of the classification model from the second device.

[0124] In some exemplary embodiments, the classification result indicates whether the communication channel is classified as a line-of-sight channel or a non-line-of-sight channel.

[0125] In some exemplary embodiments, the first device includes a terminal device and the second device includes a location management function. In some exemplary embodiments, the communication channel includes a channel between the terminal device and the network device.

[0126] 9 shows a flowchart of an example method 900 implemented in a second device in some example embodiments of the present disclosure. For purposes of explanation, the method 900 is described from the perspective of the second device 120 of FIG.

[0127] In block 910, the second device 120 receives importance evaluation information from the first device indicating the importance of the channel measurement information when updating the classification model, and the classification model is used to determine a classification result of the communication channel based on the channel measurement information.

[0128] At block 920, the second device 120 determines whether the importance of the channel measurement information exceeds an importance threshold.

[0129] If the importance of the channel measurement information exceeds the importance threshold, then in block 930, the second device 120 causes a third device to perform classification labeling for at least the communication channel of the location associated with the first device.

[0130] In some demonstrative embodiments, the method 900 further includes receiving, from the third device, at least one pair of sample channel measurement information regarding the communication channel and a ground truth classification result labeled for the sample channel measurement information, and updating at least the classification model based on the at least one pair of the sample channel measurement information and the ground truth classification result.

[0131] In some exemplary embodiments, the method 900 further includes transmitting at least an update of the classification model to the first device.

[0132] In some exemplary embodiments, receiving the importance assessment information includes, in accordance with a determination that the classification model is of a first type, receiving importance assessment information including at least uncertainty of the classification result, and in accordance with a determination that the classification model is of a second type, receiving importance assessment information including at least channel measurement information.

[0133] In some exemplary embodiments, the first type of classification model is a type of model having classification result uncertainty determined without reconstructing the classification model, and in some exemplary embodiments, the second type of classification model is a type of model having classification result uncertainty determined by reconstructing the classification model.

[0134] In some demonstrative embodiments, the method 900 further includes, in accordance with determining that the importance assessment information includes at least the channel measurement information, determining an uncertainty of the classification result based on the channel measurement information.

[0135] In some exemplary embodiments, the classification model is of a second type. In some exemplary embodiments, determining the uncertainty of the classification result includes generating a plurality of reference classification models by reconstructing the classification model, determining a plurality of reference classification results based on the channel measurement information using the plurality of reference classification models, and determining the uncertainty of the classification result based on the variance of the plurality of reference classification results.

[0136] In some exemplary embodiments, the multiple reference classification models are generated by applying random neural connection dropout to the classification model.

[0137] In some exemplary embodiments, the uncertainty of the classification result that exceeds the uncertainty threshold is received from the first device.

[0138] In some exemplary embodiments, the method 900 further includes transmitting the uncertainty threshold to the first device.

[0139] In some exemplary embodiments, the uncertainty threshold is determined based on the accuracy level of the classification model. In some exemplary embodiments, the uncertainty threshold is updated based on updates to the classification model.

[0140] In some exemplary embodiments, the classification result indicates whether the communication channel is classified as a line-of-sight channel or a non-line-of-sight channel.

[0141] In some exemplary embodiments, the first device includes a terminal device, the second device includes a location management function, and the third device includes a positioning reference unit. In some exemplary embodiments, the communication channel includes a channel between the terminal device and the network device.

[0142] [Examples of equipment, devices, and media] In some demonstrative embodiments, a first device capable of performing any of the methods 800 (e.g., first device 110 in FIG. 1 ) may comprise means for performing each operation of method 800. The means may be implemented in any suitable form. For example, the means may be implemented in a circuit or a software module. The first device may be implemented as or included in first device 110 in FIG. 1 .

[0143] In some exemplary embodiments, the first device comprises: means for determining a classification result of a communication channel using a classification model based at least in part on channel measurement information relating to the communication channel; means for determining importance evaluation information indicating an importance of the channel measurement information when updating the classification model based at least in part on a type of the classification model; and means for transmitting the importance evaluation information to the second device.

[0144] In some exemplary embodiments, the means for determining importance assessment information comprises: means for determining whether the type of the classification model is a first type or a second type; means for determining, in accordance with a determination that the type of the classification model is the first type, uncertainty of the classification result; means for generating the importance assessment information to include at least the uncertainty of the classification result; and means for generating, in accordance with a determination that the type of the classification model is the second type, the importance assessment information to include at least channel measurement information.

[0145] In some exemplary embodiments, the first type of classification model is a type of model having classification result uncertainty determined without reconstructing the classification model, and in some exemplary embodiments, the second type of classification model is a type of model having classification result uncertainty determined by reconstructing the classification model.

[0146] In some exemplary embodiments, the classification result indicates whether the communication channel is classified into a first channel category or a second channel category, and the classification result determined using the first type of classification model is based on a ratio of a first number of model votes for the first channel category to a second number of model votes for the second channel category. In some exemplary embodiments, the means for determining the uncertainty comprises means for determining a degree of difference between the first number and the second number, and means for determining the uncertainty based on the degree of difference.

[0147] In some exemplary embodiments, the means for transmitting the importance assessment information to the second device comprises means for transmitting the importance assessment information to the second device in accordance with a determination that the determined uncertainty exceeds an uncertainty threshold.

[0148] In some exemplary embodiments, the first device further comprises means for receiving an uncertainty threshold from the second device.

[0149] In some exemplary embodiments, the classification result is determined based on a predicted probability provided by a second type of classification model to indicate whether the communication channel is classified into the first channel category or the second channel category.

[0150] In some exemplary embodiments, the first device further comprises means for receiving at least an update of the classification model from the second device.

[0151] In some exemplary embodiments, the classification result indicates whether the communication channel is classified as a line-of-sight channel or a non-line-of-sight channel.

[0152] In some exemplary embodiments, the first device includes a terminal device and the second device includes a location management function. In some exemplary embodiments, the communication channel includes a channel between the terminal device and the network device.

[0153] In some exemplary embodiments, the first device further comprises means for performing method 800 or other operations in some exemplary embodiments of first device 110. In some exemplary embodiments, the means comprises at least one processor and at least one memory that stores instructions that, when executed by the at least one processor, cause performance of the first device.

[0154] In some demonstrative embodiments, a second device capable of performing any of the methods 900 (e.g., second device 120 in FIG. 1 ) may comprise means for performing each operation of method 900. The means may be implemented in any suitable form. For example, the means may be implemented in a circuit or a software module. The second device may be implemented as or included in second device 120 in FIG. 1 .

[0155] In some exemplary embodiments, the second device comprises: means for receiving, from the first device, importance assessment information indicating importance of the channel measurement information when updating a classification model, the classification model being used to determine a classification result of the communication channel based on the channel measurement information; means for determining whether the importance of the channel measurement information exceeds an importance threshold; and means for causing a third device to perform classification labeling of at least the communication channel at a location associated with the first device according to a determination that the importance of the channel measurement information exceeds the importance threshold.

[0156] In some exemplary embodiments, the second device further comprises means for receiving, from the third device, at least one pair of sample channel measurement information related to the communication channel and a ground truth classification result labeled for the sample channel measurement information, and means for updating at least the classification model based on the at least one pair of sample channel measurement information and the ground truth classification result.

[0157] In some exemplary embodiments, the second device further comprises means for transmitting at least an update of the classification model to the first device.

[0158] In some exemplary embodiments, the means for receiving importance assessment information comprises: means for receiving, in accordance with a determination that the classification model is of a first type, importance assessment information including at least uncertainty of the classification result; and means for receiving, in accordance with a determination that the classification model is of a second type, importance assessment information including at least channel measurement information.

[0159] In some exemplary embodiments, the first type of classification model is a type of model having classification result uncertainty determined without reconstructing the classification model, and in some exemplary embodiments, the second type of classification model is a type of model having classification result uncertainty determined by reconstructing the classification model.

[0160] In some exemplary embodiments, the second device further comprises means for determining, in accordance with a determination that the importance assessment information includes at least the channel measurement information, determining an uncertainty of the classification result based on the channel measurement information.

[0161] In some exemplary embodiments, the classification model is of a second type. In some exemplary embodiments, the means for determining the uncertainty of the classification result comprises: means for generating a plurality of reference classification models by reconstructing the classification model, means for determining a plurality of reference classification results based on the channel measurement information using the plurality of reference classification models, and means for determining the uncertainty of the classification result based on the variance of the plurality of reference classification results.

[0162] In some exemplary embodiments, the multiple reference classification models are generated by applying random neural connection dropout to the classification model.

[0163] In some exemplary embodiments, the uncertainty of the classification result that exceeds the uncertainty threshold is received from the first device.

[0164] In some exemplary embodiments, the second device further comprises means for transmitting the uncertainty threshold to the first device.

[0165] In some exemplary embodiments, the uncertainty threshold is determined based on the accuracy level of the classification model. In some exemplary embodiments, the uncertainty threshold is updated based on updates to the classification model.

[0166] In some exemplary embodiments, the classification result indicates whether the communication channel is classified as a line-of-sight channel or a non-line-of-sight channel.

[0167] In some exemplary embodiments, the first device includes a terminal device, the second device includes a location management function, and the third device includes a positioning reference unit. In some exemplary embodiments, the communication channel includes a channel between the terminal device and the network device.

[0168] In some exemplary embodiments, the second device further comprises means for performing method 900 or other operations in some exemplary embodiments of second device 120. In some exemplary embodiments, the means comprises at least one processor and at least one memory that stores instructions that, when executed by the at least one processor, cause the performance of the second device to be performed.

[0169] 10 is a simplified block diagram of a device 1000 suitable for implementing an exemplary embodiment of the present disclosure. The device 1000 may be provided to implement a communications device such as the first device 110 or the second device 120 as shown in FIG. 1. As shown, the device 1000 includes one or more processors 1010, one or more memories 1020 coupled to the processors 1010, and one or more communications modules 1040 coupled to the processors 1010.

[0170] The communication module 1040 is for two-way communication. The communication module 1040 has one or more communication interfaces to facilitate communication with one or more other modules or devices. A communication interface refers to any interface necessary for communication with other network elements. In some exemplary embodiments, the communication module 1040 may include at least one antenna.

[0171] The processor 1010 may be of any type suitable for a local technology network and may include, by way of non-limiting example, one or more of a general-purpose computer, a special-purpose computer, a microprocessor, a digital signal processor (DSP), and a processor based on a multi-core processor architecture. The device 1000 may have multiple processors, such as application-specific integrated circuit chips that are time-slaved to a clock that synchronizes a main processor.

[0172] The memory 1020 may include one or more non-volatile memories and one or more volatile memories. Examples of non-volatile memory include, but are not limited to, read-only memory (ROM) 1024, electronically programmable read-only memory (EPROM), flash memory, hard disks, compact disks (CDs), digital video disks (DVDs), optical disks, laser disks, and other magnetic and / or optical storage devices. Examples of volatile memory include, but are not limited to, random access memory (RAM) 1022 and other volatile memories that do not persist through power-down periods.

[0173] The computer program 1030 includes computer-executable instructions that are executed by the associated processor 1010. The program 1030 may be stored in a memory, for example, in the ROM 1024. The processor 1010 can load the program 1030 into the RAM 1022 to perform any suitable operations and processes.

[0174] An exemplary embodiment of the present disclosure may be implemented by a program 1030 such that the device 1000 may execute any process of the present disclosure, as described with reference to Figures 3 to 5. An exemplary embodiment of the present disclosure may also be implemented by hardware or a combination of software and hardware.

[0175] In some exemplary embodiments, the program 1030 may be tangibly contained in a computer-readable medium, which may be included in the device 1000 (such as in memory 1020) or other storage accessible by the device 1000. The device 1000 may load the program 1030 from the computer-readable medium into RAM 1022 for execution. The computer-readable medium may include any type of tangible non-volatile storage, such as ROM, EPROM, flash memory, hard disk, CD, DVD, etc. FIG. 11 illustrates an example of a computer-readable medium 1100, which may be in the form of a CD, DVD, or other optical storage disc. The computer-readable medium has the program 1030 stored thereon.

[0176] In general, various embodiments of the present disclosure may be implemented in hardware or special purpose circuits, software, logic, or any combination thereof. Some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device. While various aspects of embodiments of the present disclosure are illustrated and described using block diagrams, flowcharts, or some other graphical representations, it should be understood that the blocks, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, special purpose circuits or logic, general purpose hardware or a controller or other computing device, or some combination thereof, in non-limiting examples.

[0177] The present disclosure also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The term "non-transitory" as used herein refers to the medium itself (i.e., tangible, not a signal) as opposed to the persistence of data storage (e.g., RAM versus ROM). The computer program product includes computer-executable instructions, such as those included in program modules, that execute on a target physical or virtual processor to perform any of the methods described above with reference to Figures 2 through 6. Generally, program modules include routines, programs, libraries, objects, classes, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The functionality of the program modules may be combined or split among program modules as desired in various embodiments. The machine-executable instructions of the program modules may be executed in local or distributed devices. In distributed devices, the program modules may be located in both local and remote storage media.

[0178] Program code for implementing the methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus, and when executed by the processor or controller, causes the specific functions / operations shown in the flowcharts and / or block diagrams to be performed. The program code can run entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0179] In the context of the present disclosure, computer program code or associated data may be carried by any suitable carrier to enable a device, computing device, or processor to perform the various processes and operations as described above. Examples of carriers include signals, computer-readable media, etc.

[0180] The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. Computer-readable media include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. More specific examples of computer-readable storage media include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0181] Furthermore, although operations are depicted in a particular order, this should not be understood as requiring such operations to be performed in the particular order shown, or sequentially, or to perform all of the operations shown, to achieve desirable results. In certain situations, multitasking and parallel processing may be preferred. Similarly, while several specific implementation details are included in the above description, these should not be construed as limiting the scope of the disclosure, but rather as descriptions of features that may be unique to particular embodiments. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.

[0182] Although the present disclosure has been described in language specific to structural features and / or methodological acts, it is to be understood that the present disclosure, as defined by the appended claims, is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. a first device, at least one processor; When executed by the at least one processor, the first device is configured to: determining a classification result for a communication channel based at least in part on channel measurement information for the communication channel using the classification model; determining importance assessment information indicating an importance of the channel measurement information in updating the classification model based at least in part on a type of the classification model; transmitting the importance assessment information to a second device; at least one memory storing instructions for executing the A first device comprising:

2. The at least one memory, when executed by the at least one processor, causes the first device to: determining whether the type of the classification model is a first type or a second type; in response to determining that the type of the classification model is the first type, determining the uncertainty of the classification result; generating the importance assessment information so as to include at least the uncertainty of the classification result; generating the importance assessment information to include at least the channel measurement information in accordance with a determination that the type of the classification model is the second type; The first device of claim 1 , further comprising: a memory for storing instructions for causing the importance rating information to be determined by:

3. the classification model of the first type is a type of model having uncertainty of a classification result that can be determined without reconstructing the classification model; the classification model of the second type is a type of model having uncertainty of the classification result determined by reconstructing the classification model; The first device of claim 2 .

4. the classification result indicates whether the communication channel is classified into a first channel category or a second channel category, and the classification result determined using the classification model of the first type is based on a ratio of a first number of model votes for the first channel category to a second number of model votes for the second channel category; The at least one memory, when executed by the at least one processor, causes the first device to: determining a degree of difference between the first number and the second number; determining the uncertainty based on the degree of the difference; and storing instructions for determining the uncertainty by A first device according to claim 2 or 3.

5. The at least one memory, when executed by the at least one processor, causes the first device to: transmitting the importance assessment information to the second device in accordance with a determination that the determined uncertainty exceeds an uncertainty threshold; and storing an instruction to transmit the importance evaluation information to the second device by A first device according to any one of claims 2 to 4.

6. The at least one memory, when executed by the at least one processor, causes the first device to further: receiving the uncertainty threshold from the second device; 6. The first device of claim 5, further comprising instructions for:

7. 7. The first device of claim 2, wherein the classification result is determined based on a predicted probability provided by the classification model of the second type and indicates whether the communication channel is classified into a first channel category or a second channel category.

8. The at least one memory, when executed by the at least one processor, causes the first device to further: receiving at least an update to the classification model from the second device; 8. A first device according to claim 2, further comprising instructions for:

9. The first device according to claim 1 , wherein the classification result indicates whether the communication channel is classified as a line-of-sight channel or a non-line-of-sight channel.

10. the first device includes a terminal device, and the second device includes a location management function; the communication channel includes a channel between the terminal device and a network device; A first device according to any one of claims 1 to 9.

11. a second device, at least one processor; When executed by the at least one processor, the second device receives at least: receiving, from a first device, importance assessment information indicating importance of channel measurement information when updating a classification model, the classification model being used to determine a classification result of a communication channel based on the channel measurement information; determining whether the importance of the channel measurement information exceeds an importance threshold; and causing a third device to perform classification labeling of at least the communication channel at a location associated with the first device in accordance with a determination that the importance of the channel measurement information exceeds the importance threshold. at least one memory storing instructions for executing the A second device comprising:

12. The at least one memory, when executed by the at least one processor, causes the second device to further: receiving, from the third device, at least one pair of sample channel measurement information regarding the communication channel and a ground truth classification result labeled for the sample channel measurement information; updating at least the classification model based on the at least one pair of the sample channel measurement information and the ground truth classification result; 12. The second device of claim 11, further comprising:

13. The at least one memory, when executed by the at least one processor, causes the second device to further: sending at least an update of the classification model to the first device; 13. The second device of claim 12, further comprising:

14. The at least one memory, when executed by the at least one processor, causes the second device to: receiving the importance assessment information including at least an uncertainty of the classification result in accordance with determining that the classification model is of a first type; receiving the importance assessment information including at least the channel measurement information in accordance with determining that the classification model is of a second type; The second device according to claim 11 , further comprising: a memory for storing instructions for receiving the importance evaluation information by:

15. the classification model of the first type is a type of model having uncertainty of a classification result that can be determined without reconstructing the classification model; the classification model of the second type is a type of model having uncertainty of the classification result determined by reconstructing the classification model; The second device of claim 14.

16. The at least one memory, when executed by the at least one processor, causes the second device to further: determining an uncertainty of the classification result based on the channel measurement information in accordance with determining that the importance assessment information includes at least the channel measurement information; 15. The second device of claim 14, further comprising instructions for:

17. the classification model is of a second type, and the at least one memory, when executed by the at least one processor, causes the second device to generating a plurality of reference classification models by reconstructing the classification model; determining a plurality of reference classification results based on the channel measurement information using the plurality of reference classification models; determining the uncertainty of the classification result based on the variance of the plurality of reference classification results; 17. The second device of claim 16, further storing instructions for causing the second device to determine the uncertainty of the classification result by:

18. 18. The second device of claim 17, wherein the at least one memory stores instructions that, when executed by the at least one processor, cause the second device to generate the plurality of reference classification models by applying random neural connection dropout to the classification models.

19. 16. The second device of claim 15, wherein the at least one memory stores instructions that, when executed by the at least one processor, cause the second device to receive from the first device the uncertainty of the classification result that exceeds an uncertainty threshold.

20. The at least one memory, when executed by the at least one processor, causes the second device to further: transmitting the uncertainty threshold to the first device; 20. A second device according to any one of claims 11 to 19, storing instructions to cause the second device to perform the following:

21. the uncertainty threshold is determined based on an accuracy level of the classification model; the uncertainty threshold is updated based on updates to the classification model. A second device according to any one of claims 11 to 20.

22. The second device of claim 11 , wherein the classification result indicates whether the communication channel is classified as a line-of-sight channel or a non-line-of-sight channel.

23. the first device includes a terminal device, the second device includes a location management function, and the third device includes a positioning reference unit; the communication channel includes a channel between the terminal device and a network device; A second device according to any one of claims 11 to 22.

24. determining, at the first device, a classification result for a communication channel based at least in part on channel measurement information for the communication channel using the classification model; determining importance assessment information to indicate an importance of the channel measurement information in updating the classification model based at least in part on a type of the classification model; transmitting the importance assessment information to a second device; A method comprising:

25. receiving, at a second device, from the first device, importance evaluation information indicating importance of channel measurement information when updating a classification model, the classification model being used to determine a classification result of a communication channel based on the channel measurement information; determining whether the importance of the channel measurement information exceeds an importance threshold; and causing a third device to perform classification labeling of at least the communication channel at a location associated with the first device in accordance with a determination that the importance of the channel measurement information exceeds the importance threshold. A method comprising:

26. means for determining a classification result for a communication channel based at least in part on channel measurement information for the communication channel using the classification model; means for determining importance assessment information indicating the importance of the channel measurement information in updating the classification model based at least in part on the type of the classification model; means for transmitting the importance assessment information to a second device; A first device comprising:

27. means for receiving, from the first device, importance assessment information indicating importance of channel measurement information when updating a classification model, the classification model being used to determine a classification result of a communication channel based on the channel measurement information; and means for determining whether the importance of the channel measurement information exceeds an importance threshold; means for causing a third device to perform classification labeling for at least the communication channel at a location associated with the first device in response to a determination that the importance of the channel measurement information exceeds the importance threshold; A second device comprising:

28. A computer readable medium having stored thereon instructions for causing an apparatus to perform at least the method of claim 24 or the method of claim 25.

Images