Wireless System Measurement Data Collection for Artificial Intelligence Use Cases

Abstract

The UE may send a UE capability report to the network node. The UE capability report may indicate that the UE supports the collection of logged data for artificial intelligence (AI) training. The network node may send a data collection request to the UE. The UE may verify data collector credentials for the data collection server corresponding to the network node. The UE may send the logged data to the network node in batches, along with other additional data samples collected at different times than the measurements and related information.

Description

Technical Field

[0001] This application generally relates to a wireless communication system that includes generating logged measurements for training an artificial intelligence model.

Background Art

[0002] Wireless mobile communication technologies use various standards and protocols to transmit data between a base station and a wireless communication device. Wireless communication system standards and protocols can include, for example, the 3rd Generation Partnership Project (3GPP) Long Term Evolution (LTE) (e.g., 4G), 3GPP New Radio (NR) (e.g., 5G), and the Institute of Electrical and Electronics Engineers (IEEE) 802.11 standard for Wireless Local Area Network (WLAN) (commonly known as Wi-Fi (registered trademark) to industry groups).

[0003] As contemplated by 3GPP, different wireless communication system standards and protocols can use various radio access networks (RANs) to communicate between a base station of a radio access network (RAN) (commonly referred to as a RAN node, network node, or simply a node) and a wireless communication device known as a user equipment (UE). 3GPP RANs can include, for example, the Global System for Mobile Communications (GSM) for mobile communications, the Enhanced Data Rate for GSM Evolution (EDGE) RAN (GERAN), the Universal Terrestrial Radio Access Network (UTRAN), the Evolved Universal Terrestrial Radio Access Network (E-UTRAN), and / or the Next Generation Radio Access Network (NG-RAN).

[0004] Each RAN can use one or more radio access technologies (RATs) to perform communication between base stations and UEs. For example, GERAN implements GSM and / or EDGE RATs, UTRAN implements Universal Mobile Telecommunication System (UMTS) RATs or other 3GPP RATs, E-UTRAN implements LTE RATs (sometimes simply referred to as LTE), and NG-RAN implements NR RATs (sometimes referred to herein as 5G RATs, 5G NR RATs, or simply NR). In certain deployments, E-UTRAN may also implement NR RATs. In certain deployments, NG-RAN may also implement LTE RATs.

[0005] Base stations used by a RAN can be compatible with that RAN. An example of an E-UTRAN base station is an Advanced Universal Terrestrial Radio Access Network (E-UTRAN) Node B (commonly also called Advanced Node B, Extended Node B, eNode B, or eNB). An example of an NG-RAN base station is a Next Generation Node B (sometimes referred to as gNode B or gNB).

[0006] A RAN provides communication services with external entities via a connection to the core network (CN). For example, E-UTRAN can utilize the Evolved Packet Core (EPC), and NG-RAN can utilize the 5G Core Network (5GC).

[0007] The 5G NR frequency band can be divided into two or more different frequency ranges. For example, frequency range 1 (FR1) may include frequency bands operating at sub-6 GHz frequencies, some of which can be used by previous standards and can potentially be extended to cover new spectra providing 410 MHz to 7125 MHz. Frequency range 2 (FR2) may include frequency bands from 24.25 GHz to 52.6 GHz. Note that in some systems, FR2 may also include frequency bands from 52.6 GHz to 71 GHz (or above). The millimeter wave (mmWave) range of FR2 may have a smaller range than the FR1 range, but the available bandwidth is potentially wider. It will be understood by those skilled in the art that these frequency ranges provided as examples may vary from time to time or region to region.

[0008] To facilitate the identification of any particular element or action, the most significant digit(s) of the reference number refers to the number of the figure in which that element was first introduced. [Brief explanation of the drawing]

[0009] [Figure 1] A table comparing real-time AI data collection frameworks and logged AI data collection frameworks in several embodiments is shown.

[0010] [Figure 2] The following shows signal flow diagrams for logging data collection frameworks in several embodiments.

[0011] [Figure 3] The first set of combined reports and the second set of combined reports 304 are shown according to several embodiments.

[0012] [Figure 4A] This shows the codebook configuration information elements.

[0013] [Figure 4B] The structure of a CSI report is shown.

[0014] [Figure 5] A flowchart of a method for UE according to embodiments of this specification is shown.

[0015] [Figure 6] A flowchart of a method for a network node according to embodiments of this specification is shown.

[0016] [Figure 7] This specification shows an exemplary architecture of a wireless communication system according to embodiments disclosed herein.

[0017] [Figure 8] This specification describes a system for performing signaling between a wireless device and a network device, according to embodiments disclosed herein. [Modes for carrying out the invention]

[0018] Various embodiments are described with respect to user equipment (UE). However, references to UE are provided for illustrative purposes only. Exemplary embodiments may be used with any electronic component, which consists of hardware, software, and / or firmware capable of establishing connectivity to a network and exchanging information and data with the network. Thus, the UE described herein is used to represent any suitable electronic component.

[0019] The goals of a wireless communication system include providing reliable, efficient, and secure communication between a UE and a network node. Artificial intelligence (AI) can be used to assist in achieving these goals. For example, AI technology can be used in several ways within a wireless system, including network optimization. For example, AI can be used to optimize the performance of a network by analyzing data, predicting future traffic patterns, and identifying areas of congestion. This can help improve network efficiency and reduce downtime. Additionally, AI can be used to automate network operations such as provisioning, configuration, and optimization. This can potentially help reduce costs and improve operational efficiency.

[0020] Three exemplary use cases in which AI can be used to improve network operations include Channel State Information (CSI) feedback, beam management (BM), and positioning. For example, in the case of BM, an AI model can be used to select a beam based on a subset of beam measurements. In the case of CSI feedback, the AI model can reduce overhead. For positioning, the AI model may be able to estimate the position more accurately.

[0021] To generate an AI model, the system can collect data, and the collected data can be used to train the AI model. The model can then be used to improve the performance of the system. Data collection can be used for model training, feedback from UEs for the model, and monitoring of the model.

[0022] The AI model improves its performance using training data. Such data may already have been collected by the UE. For example, in the case of beam management (e.g., time domain prediction), there can be several scenarios where the UE can report beam measurement values to the network. Similarly, when network-side training is used during data collection for AI model training, the UE also reports measurement values to the network. Data collection may be required for AI-CSI and AI positioning.

[0023] As an example where measurement data is collected, there is radio measurement value collection for Minimization of Drive Test (MDT). As MDT data, two modes of data logging, namely, immediate MDT and logging MDT, are incorporated. In logging MDT, the network can set specific conditions such as UE failures for the UE to log data. When such conditions occur, the UE can record the data and then upload the data to the network later. The network can use that information for optimization.

[0024] The MDT framework can be leveraged for AI model training. However, there are several issues that need to be addressed, particularly with regard to AI. The first issue is how the network can ensure that data collection for set A and set B occurs simultaneously, in beam management use cases, when set B is not a subset of set A. For example, how can the network establish correlation or dependency between the collections of input and output data for an AI model? The second issue is how data collection can be controlled. While UEs can provide large amounts of data, it may be advantageous to reduce the amount of data for efficient operation of the wireless network and / or for power saving of the UE. For example, the network can limit the data to measurements occurring at the Reference Signal Received Power (RSRP). The third issue is that much of the information collected about UEs in MDT may pose a threat to UE privacy. For example, information about UEs may include the geographical range of measurement logging. It is possible to define geographical areas from which a defined set of measurements can be collected. The information may also include location information. Measurements may be linked to available location information and / or other information or measurements that can be used to derive location information. The information may include time information. Measurements in the measurement log may be linked to timestamps. The information may include sensor information. Measurements may be linked to available sensor information that can be used to derive the orientation of the UE in the global coordinate system, uncompensated atmospheric pressure, and the velocity of the UE.

[0025] Therefore, an AI data acquisition framework is needed. This AI data acquisition framework could involve either immediate or logging-based data acquisition. For example, similar to immediate MDT, data acquisition can be performed in a segmented manner. For instance, a network could leverage CSI reports, beam management reports, and positioning reports, and the UE could be instructed to perform measurements and reports even if none of these reports directly benefit the UE's operation. Such practices could be designated as an immediate AI data acquisition framework.

[0026] Similar to logged MDT, data collection for AI model training can be performed in batches. For example, a UE (User Engineer) may be instructed to collect many data samples and upload them to the network at a suitable time. This approach can be designated as a logged AI data collection framework. This approach can support the fair use of data collection. Since training data is key to AI, the collection of training data depends on cooperation between the UE vendor and the end user, and the success of the AI ​​should be shared by both the UE vendor and the end user. Data collection conditions, UE capabilities for data collection, and data upload may also be considered.

[0027] Figure 1 shows Table 100, which compares an immediate AI data acquisition framework 102 with a logged AI data acquisition framework 104 in several embodiments. The immediate AI data acquisition framework 102 may have little impact on specifications because it can leverage existing CSI / BM / positioning measurement / reporting frameworks. However, even if data acquisition is not for real-time network operation, the UE processing load for data acquisition can be high. Furthermore, since data acquisition in this case is disguised as normal wireless network operation, the UE simply follows measurement instructions from the network without recognizing the true intent behind the instructions, and there may be no monetization opportunity for either the UE vendor or the end user.

[0028] In contrast to current New Radio (NR) designs where CSI / BM / positioning measurements and reporting are used for data collection purposes (immediate AI data collection framework), the logged AI data collection framework 104 can allow for reduced processing and reporting times and provides opportunities for monetizing data collection. Furthermore, if data collection is explicitly indicated from the network to the UE, the UE can check with a server (e.g., UE server) to confirm whether the network has permission to collect data from the UE. The UE vendor or end user (subscriber) can monetize the data collection.

[0029] As recent AI successes demonstrate, training data is key to AI. Since data collection is difficult to distinguish from normal wireless network operation, the immediate AI data collection framework 102 allows the network to collect data through UEs without any tangible benefit to the UE vendor or UE owner.

[0030] In the logged AI data collection framework 104, consent for data collection may need to be properly obtained for data collection. Establishing consent may be based on one or more agreements between the UE vendor / UE owner and the network vendor and operator. For this purpose, a new network signaling protocol may be introduced for the data collector to provide its credentials to the UE. After credential verification, the UE may consent to the data collection request, and the end user (UE owner), UE vendor, etc., can participate in the successful use of any AI.

[0031] Figure 2 shows a signal flow diagram 200 for a logged data collection framework in several embodiments. In step 0, the UE vendor or end user can sign a data collection agreement 210 with the infrastructure vendor, operator, or a third-party data collector (e.g., providing data collection services to the infrastructure vendor or operator), and the agreement may include monetization terms. For example, the UE 202 may provide up to 10,000 data samples per month, and may be paid 1 cent each time data sample collection is triggered.

[0032] In some embodiments, UE202 can directly verify the data collector credentials. For example, several tokens can be provisioned to UE202 periodically or aperiodically (e.g., every hour) through an application installed on a phone, and as a result, multiple such token provisioning applications or instances may be used (e.g., one for each vendor). To avoid direct interaction with the data collector (e.g., infrastructure vendor / operator / third-party data collector), verification of data collector credentials can be done via a central entity (e.g., data collector credential verifier 208), thereby enabling secure querying and response regarding data collector credentials.

[0033] In the illustrated embodiment, in step 1 (i.e., UE capability signaling 212), the UE reports to network node 204 its capability to support the logged AI data acquisition framework. The capability report sent to network node 204 may indicate support for different sub-functions of the logged AI data acquisition framework. The logged AI data acquisition framework may include multiple sub-functions for CSI, BM, and positioning, etc. In some embodiments, the UE capability signaling 212 may have finer granularity to indicate what the UE 202 supports. For example, the UE may indicate that it supports a specific number of ports for a given period or number of samples (e.g., 32Tx ports for 1k samples, 16Tx ports over four time instances for 512 samples, etc.).

[0034] In step 2, network node 204 can send a data collection request 214 to UE 202. In step 2A, network node 204 can send its data collector credentials 216 to UE 202. In steps 2 and 2A, the signaling flow is not necessarily chronological. For example, in some embodiments, steps 2A and even step 3 may be performed before step 2. In the illustrated embodiment, step 2 is signaling from the network for data collection, and the source can be a network node or another network entity.

[0035] In step 2A, the data collector credentials 216 are provided to the UE 202. The source of the data collector credentials 216 may be a wireless network (e.g., network node 204) or provisioned through higher signaling from a core network entity (e.g., data collection server 206). In some embodiments, network node 204 may receive the data collector credentials 216 from another entity (e.g., data collection server 206) and then transmit the data collector credentials 216 to the UE 202. Since the signaling itself can be encrypted, the wireless network cannot modify or inspect it.

[0036] The data collector credentials 216 can include data collector identification information, verification (e.g., verification in the form of a public key or hash), and a timestamp. This signaling protocol can have standardization support, allowing phones from different vendors to contribute to data collection.

[0037] In step 3, UE202 may verify 218 the provided data collector credentials 216. In some embodiments, UE202 may verify 218 the data collector credentials 216 using data collector credentials stored in UE202, or UE may send the provided data collector credentials for verification to the data collector credential verifier 208. In some embodiments, the verification process may be performed locally in UE202 or by communication with the data collector credential verifier 208. In some embodiments, the data collector credential verifier 208 may not be standardized.

[0038] In step 4, UE202 responds to the data collection request. The response 220 sent to network node 204 may accept or reject the data collection request 214. The response 220, which accepts or rejects the data collection request 214, may benefit scheduling. If UE202 does not indicate whether the data collection request 214 is rejected or accepted, network node 204 may assume that data collection is in progress, which may affect network scheduling or configuration decisions and thus affect UE202 or end users without any substantive benefit. On the other hand, if network node 204 is aware that UE202 has rejected or accepted the data collection request 214, scheduling or configuration decisions may take this into appropriate consideration.

[0039] In step 5, the network node 204 can send a data acquisition configuration 222 to the UE to configure data acquisition conditions, measurement signals, reporting format, etc. Note that in some embodiments, steps 5 and 2 are combined. In step 6, the network node 204 can send a measurement signal 224 to the UE 202, and the UE 202 can perform a measurement on the transmission from the network node 204. In step 7, the UE 202 can perform processing on the measurement to generate the data requested by the network node 204. The UE can store the logged data in the UE 202 226 or upload the logged data to a server.

[0040] In step 8, if the logged data is stored on the device, UE202 can send the logged data directly to network node 204. In step 7, if the logged data has already been uploaded to the server, UE can send the uniform resource locator (URL) to network node 204. Both control plane and user plane solutions can be considered here. The logged data may be sent later as a batch, possibly along with other logged data from other data collection requests.

[0041] Figure 3 shows a first set of combined reports 302 and a second set of combined reports 304 according to several embodiments. In a logged AI data collection framework, it may be important that there is correlation or dependency between two collections of logged data. Therefore, network nodes may be configured to correlate or have dependency between two collections of logged data.

[0042] For example, when set B306 is a subset of set A308, it may be sufficient to construct the measurement signal for the higher-level set that covers the set A308 beam, or for the set A308 beam where the higher-level set is set A itself. From the measurements reported by the UE, the network can extract the measurements for set B306. However, there may usually be a time lag between the beam measurements in set B306 and the beam measurements in set A308. When the time lag is not negligible, it may not be possible to extract the measurements for set B from the reports of the higher-level set.

[0043] When set B306 is not a subset of set A308, it may not be possible to extract beam measurements in set B306 from beam measurements in the higher-level set. For example, if set B306 consists of Synchronization Signal Block (SSB) beams, but set A308 consists of beams associated with CSI reference signal (RS) resources, it may not be possible to extract beam measurements in set B306 from beam measurements in the higher-level set.

[0044] The network can configure two combined CSI or BM reports for both quasi-co-location (QCL) assumptions or TCI states (one or more) that are identical in terms of the receive (Rx) spatial filter but not defined by the network. For example, there may be no source signal in the QCL chain, which can be used for connected mode UEs and idle mode UEs when the measurement resources are configured in a non-UE-specific manner. In some embodiments, for both reports, the QCL assumptions or TCI states (one or more) may be identical with respect to the Rx spatial filter, which can be linked to the current TCI state of a Physical Downlink Control Channel (PDCCH) or Physical Downlink Shared Channel (PDSCH), which may be more preferable for connected mode UEs.

[0045] For each report, the measurement resources can be an SSB resource / SSB resource set, or a CSI resource / resource set(s) / resource configuration. Reports can define periodic, semi-persistent, or event-driven conditions.

[0046] Figure 4A shows the codebook configuration information element 400a. The codebook configuration information element 400a includes the cell's antenna configuration 402. Figure 4B shows the CSI reporting configuration 400b. The antenna configuration 402 may not be known to the cell from which the UE reports the logged data. The UE may provide this information along with the logged data.

[0047] With logged data, data collection may not be visible to the network. This can hinder the network from fully understanding the data for AI model training. For example, because logged data is measured and reported later, the network may not know information about the cell from which the data was collected if the UE does not report it. For instance, the UE might collect data in cell 1 several hours earlier than when the logged data is reported. Then, when the UE uploads the logged data, the UE may be roaming in cell 2. The configuration of cell 2 (e.g., antenna configuration) may differ from that of cell 1. Therefore, if the UE does not provide cell configuration information, it may be difficult for the network to properly classify the data for training / testing.

[0048] In some embodiments, for logged data, the base station antenna configuration can be provided as part of a CSI-RS resource or a set of CSI-RS resources. In some embodiments, the base station antenna configuration can be provided as a CSI report, as shown in Figure 4B. Conventionally, for beam management, an antenna configuration such as "eight-two" indicates the arrangement of antenna elements / antenna modules on the base station side, including the number of rows / columns in the antenna array configuration. There may be an arrangement order CSI-RS according to the antenna array configuration, for example, when a DFT beam is used for analog beamforming of a CSI-RS resource, the analog beam angle of a given CSI-RS resource is determined by the CSI-RS index. Then, to scan a 2D matrix consisting of many rows and many columns, the CSI-RS index / analog beam angle is arranged in a row-first manner, where the CSI-RS index increases from left to right in the rows, from the top row to the bottom row (or from the bottom row to the top row). Alternatively, the CSI-RS index is increased in a column-first manner with possible variations.

[0049] Logged data records reported by the UE to network nodes may include the antenna configuration of the cell where the measurement was taken. For example, the logged data may include "eight-two" to indicate the antenna configuration of the cell from which the logged data was collected. Note that in older systems, antenna configuration 402 is part of the CSI reporting configuration 400b, but the UE does not report the antenna configuration to the network. However, for data collection purposes, the UE may need to report antenna configuration 402 to the network along with the collected data. Therefore, in some embodiments, the antenna configuration may be provided to the UE by the network node, firstly in a CSI-RS resource / CSI-RS resource set or reporting configuration, for beam management purposes. The logged data then reports or attaches the antenna configuration, thereby not restricting the cell or network receiving the logged data to being the same cell or network from which the data is logged.

[0050] Furthermore, the available data can be extensive, and not all of it may be useful to the AI ​​model trainer. For example, the model trainer may focus on a small subset of the data for training. For logged data from an AI framework, the network can set collection conditions for data collection. For example, the network may focus on high or low signal-to-noise ratios (SNR) (e.g., SNR ≥ -3dB or SNR ≥ -6dB, or 5 ≥ SNR ≥ 0dB). Collection conditions can be set to ensure that data collection is effective (does not exceed the measurement capabilities of the UE) and can also provide sufficient statistical points in the data collection. For logged data from an AI framework, collection can be set to start at a specific time (e.g., rush hour (8am) or 7pm). For logged data from an AI framework, collection can be set to start in a specific geographic area. To ensure the quality of the collected data, a Cell Global Identity (CGI) may be logged by the UE, and each data point can be associated with a timestamp. In some embodiments, combinations of collection conditions may be used.

[0051] In some embodiments, the system may age the data to enhance the privacy of user data. Information collected in logged data for an AI framework that may be used to train AI models may include privacy-sensitive information. For example, the data may be configured for geometric logging. The data may include location information, time information, and sensor information.

[0052] The threat to user privacy can be greater when logged data is uploaded in real time. Therefore, to address privacy concerns, in some embodiments, the system can "age" the data for a certain period (e.g., 8 hours). This can be done so that the collected information cannot be used to track the UE in real time. For example, in step 8 of Figure 2, where UE202 sends logged data to the network 228, aging may occur before the logged data is sent. Logged data can be stored and aged in the UE or on a server. If logged data is uploaded to a server, UE202 may not need to store the logged data in its modem storage. Rather, the application processor's storage can be used temporarily for the upload, and the aging effect can be effectively carried out using the server's storage. For example, UE202 may upload the data to the server 20 minutes after logging it, but the URL may be intentionally provided much later (e.g., 1 hour later).

[0053] AI model training can be improved by using data quality indicators. Therefore, in some embodiments, the UE can provide network nodes with data quality indicators for logged data. The AI ​​model trainer can take the data quality indicators into consideration when using or deciding to use data. For example, if data has a low data quality indicator, the data may be discarded, or the trainer may assign a lower weight to the data. The UE can collect data under several circumstances. For example, the UE can collect data on high geometry (cell center) versus low geometry (cell edge). The UE can collect data on high mobility versus low mobility. For location information, the location of data collection can be logged using a global navigation satellite system (GNSS). This can be used in any of the following situations: outdoor GNSS with a line of sight (LoS), GNSS in urban canyons, or GNSS in an indoor environment (e.g., GNSS signals routed indoors). The reliability / accuracy of location information can vary, which can substantially affect how the data is used by the model trainer (e.g., for AI-BM or AI positioning). Therefore, the variance of measurement errors, SINR, and confidence levels in RSRP may be reported by the UE.

[0054] Data collection for AI training may be performed in parallel with current measurements or may not be tied to ongoing operations. In some embodiments, the UE capability for data collection and the UE operation for data collection are determined. If data collection is triggered and derived from ongoing operations of a UE that are not dedicated to data collection (e.g., when the RSRP exceeds -90 dBm, or when the L1-SNR exceeds 5 dB for the first time (or on the first N occasions after the trigger condition)), there may be no additional computational complexity in collecting data (e.g., RSRP measurements). The UE may include sufficient storage / memory to maintain data entries. In some embodiments, a circular buffer may be used for writing. A circular buffer may be used to store data with timestamps.

[0055] However, if data acquisition is dedicated or not tied to the ongoing operation of the UE, it may inevitably be necessary to consider computational complexity in some way. For example, even if a connected mode UE has two activated CCs in FR1 and no activated CCs in FR2, data acquisition for several CCs in FR2 may be configured. In some embodiments, different methods may be employed to consider additional computational complexity in the UE to ensure that the requested measurements and reports do not exceed the UE's capabilities. In some embodiments, rules similar to CPU occupancy rules and memory count rules are defined specifically for data acquisition. Network nodes may need to ensure that the configured measurements and reports do not exceed the UE's capabilities. For example, if the UE reports that it can measure 1024 samples of RSRP and keep them for up to 8 hours, requests from network nodes should not exceed that capability. In some embodiments, the system may leverage existing CPU occupancy rules and memory count rules and modify existing designs (e.g., CSI / BM for inactive cells).

[0056] Some embodiments of this specification consider data acquisition frameworks for wireless AI. The network may provide data collector credentials for the UE to verify, and data acquisition may be conditional on the verification of the data collector credentials. In AI-BM, the combined report can be used to establish links between measurements on set B and set A. In some embodiments, privacy concerns for logged data can be addressed by aging the data. The quality of data acquisition may be indicated along with the logged data.

[0057] Figure 5 shows a flowchart of Method 500 for a UE according to an embodiment of this specification. Method 500 is a UE capability report, which includes sending a UE capability report to a network node 502 indicating that the UE supports the collection of logged data for AI training. Method 500 further includes receiving a data collection request from the network node 504. Method 500 further includes receiving data collector credentials for a data collection server corresponding to the network node 506. Method 500 further includes verifying the data collector credentials 508. Method 500 further includes performing measurements 510 and collecting relevant information in order to generate logged data. Method 500 further includes sending the logged data to the network node in batches 512 along with other additional data samples collected at different times than the measurements and relevant information.

[0058] In some embodiments of Method 500, in order to verify the data collector credentials, the Method further includes sending the data collector credentials to a data collector credential verifier.

[0059] In some embodiments of Method 500, the data collector credentials include data collector identification information, a verification key, and a timestamp.

[0060] In some embodiments of Method 500, the UE capability report indicates one or more sub-functions supported by the UE, the sub-functions include at least one of the following: data collection for CSI, data collection for BM, data collection for positioning, number of ports, or sample size.

[0061] In some embodiments, method 500 further includes sending a response to the data collection request to the network node indicating whether the UE accepts or rejects the data collection request.

[0062] In some embodiments of Method 500, the logged data includes a combined report that establishes links between sets of measurements.

[0063] In some embodiments, Method 500 further includes receiving a data acquisition configuration from a network node, the data acquisition configuration including data acquisition conditions indicating that the UE will perform measurements and collect relevant information based on at least one of signal-to-noise ratio, time, or geographical area.

[0064] In some embodiments, Method 500 further includes sending logged data to a server, and sending logged data to a network node includes sending a URL corresponding to the location of the logged data, the URL being sent after a predetermined time has elapsed since the logged data was sent to the server.

[0065] In some embodiments, method 500 further includes sending a data quality indicator for the logged data.

[0066] Embodiments contemplated herein include an apparatus comprising means for performing one or more elements of Method 500. This apparatus may be, for example, an apparatus of a UE (such as a wireless device 802, which is a UE as described herein).

[0067] Embodiments contemplated herein include one or more non-temporary computer-readable media containing instructions, which, when an instruction is executed by one or more processors of the electronic device, cause the electronic device to execute one or more elements of Method 500. This non-temporary computer-readable media may be, for example, the memory of a UE (such as the memory 806 of a wireless device 802, which is a UE, as described herein).

[0068] Embodiments contemplated herein include devices comprising logic, modules, or circuits that perform one or more elements of Method 500. These devices may be, for example, devices of a UE (such as the wireless device 802, which is a UE as described herein).

[0069] Embodiments contemplated herein include an apparatus comprising one or more processors and one or more computer-readable media containing instructions that, when executed by the one or more processors, cause the one or more processors to execute one or more elements of Method 500. This apparatus may be, for example, an apparatus of a UE (such as a wireless device 802, which is a UE as described herein).

[0070] Embodiments contemplated herein include signals described in or related to one or more elements of Method 500.

[0071] Embodiments contemplated herein include a computer program or computer program product that includes instructions, and the execution of the program by a processor causes the processor to execute one or more elements of Method 500. The processor may be the processor of the UE (such as the processor(s) 804 of the wireless device 802, which is the UE, as described herein). These instructions may be, for example, within the processor and / or on the memory of the UE (such as the memory(s) 806 of the wireless device 802, which is the UE, as described herein).

[0072] Figure 6 shows a flowchart of Method 600 for a network node according to embodiments of this specification. Method 600 includes receiving a UE capability report from the UE, the UE capability report indicating that the UE supports the collection of logged data for AI training 602. Method 600 further includes sending a data collection request to the UE 604. Method 600 further includes sending data collector credentials for a data collection server corresponding to the network node 606. Method 600 further includes requesting measurements and related information for AI training 608. Method 600 further includes receiving the logged data in batches, along with additional data samples collected at different times than the measurements and related information 610.

[0073] In some embodiments of Method 600, the data collector credentials include data collector identification information, a verification key, and a timestamp.

[0074] In some embodiments of Method 600, the UE capability report indicates one or more sub-functions supported by the UE, the sub-functions include at least one of the following: data collection for CSI, data collection for BM, data collection for positioning, number of ports, or sample size.

[0075] In some embodiments, method 600 further includes receiving a response from the UE to the data collection request, indicating whether the UE accepts or rejects the data collection request.

[0076] In some embodiments of Method 600, the logged data includes a combined report that establishes links between sets of measurements.

[0077] In some embodiments, Method 600 further includes transmitting a data acquisition configuration, which includes data acquisition conditions indicating that the UE will perform a measurement and collect relevant information based on at least one of the signal-to-noise ratio, time, or geographical area.

[0078] In some embodiments of Method 600, the logged data is received from the server after a predetermined time has elapsed since the logged data was sent to the server.

[0079] In some embodiments, method 600 further includes receiving a data quality indicator for the logged data.

[0080] Embodiments contemplated herein include an apparatus comprising means for performing one or more elements of Method 600. This apparatus may be, for example, a base station apparatus (such as network device 818, which is a base station as described herein).

[0081] Embodiments contemplated herein include one or more non-temporary computer-readable media containing instructions, which, when an instruction is executed by one or more processors of the electronic device, cause the electronic device to execute one or more elements of Method 600. This non-temporary computer-readable media may be, for example, the memory of a base station (such as the memory 822 of a network device 818 which is a base station, as described herein).

[0082] Embodiments contemplated herein include devices comprising logic, modules, or circuits that perform one or more elements of Method 600. Such devices may be, for example, base station devices (such as network device 818, which is a base station as described herein).

[0083] Embodiments contemplated herein include an apparatus comprising one or more processors and one or more computer-readable media containing instructions that, when executed by the one or more processors, cause the one or more processors to execute one or more elements of Method 600. This apparatus may be, for example, a base station apparatus (such as network device 818, which is a base station as described herein).

[0084] Embodiments contemplated herein include signals described in or related to one or more elements of Method 600.

[0085] Embodiments contemplated herein include a computer program or computer program product that includes instructions, and the execution of the program by a processing element causes the processing element to execute one or more elements of method 600. The processor may be a processor of a base station (such as a processor(s) 820 of a network device 818 which is a base station, as described herein). These instructions may be, for example, in the processor and / or in the memory of the base station (such as memory 822 of a network device 818 which is a base station, as described herein).

[0086] Training data structure

[0087] In beam management using spatial domain prediction, the training dataset may contain multiple samples. Each sample includes an input and an output. For example, the input might contain eight RSRP measurements or eight CSI resources from an SSB, and the output might contain four beam IDs, each associated with a CSI-RS resource. In another example, the input might contain eight RSRP measurements or eight CSI resources from an SSB, and the output might contain four RSRPs, each associated with a CSI-RS resource. The output values ​​can be formulated as relative values ​​(e.g., the strongest RSRP is 0 dB, the weakest RSRP is -40 dB, etc.).

[0088] In beam management using time-domain prediction, the training dataset may contain multiple samples. Each sample may include an input sequence and an output. In the first example, we may provide a sequence of four inputs, each input being for a time instance. One input may contain eight RSRP measurements or eight CSI resources from an SSB, and the output may contain four beam IDs, each associated with a CSI-RS resource for a future time instance.

[0089] In another example, the input section could include eight RSRP measurements from an SSB or eight CSI resources. For example, Table 1 shows four time instances (Instance 1, Instance 2, Instance 3, and Instance 4), each time instance containing training data for eight inputs (Input 1, Input 2, ..., Input 8). The output section could include four RSRPs, each associated with a CSI-RS resource for a future time instance. The output values ​​can be formulated as relative values ​​(e.g., the strongest RSRP is 0 dB, the weakest RSRP is -40 dB, etc.). [Table 1]

[0090] Reducing overhead of training data

[0091] In certain embodiments, the overhead of the training data can be expressed as (N1 × M1 × B1) + (N2 × M2 × B2) bits, where N1 is the number of time instances, M1 is the number of inputs per time instance, B1 is the number of bits to represent the RSRP for each input, N2 is the number of predicted time instances (for example, N2 = 1 for beam management using spatial domain prediction, and N2 can be 1 or greater for beam management using time domain prediction), M2 is the number of output beam IDs / RSRPs, and B2 is the number of bits to represent the RSRP for each output (if the output is not in the form of a beam ID).

[0092] Regarding the output, if beam IDs are used, a bitmap or combined index can be used to indicate the beam ID, which can be favorably compared to "M2 × N2" bits. However, if N1 is 100-300 time instances, the size of the training dataset cannot be considered small.

[0093] Several methods can be used to compress the training dataset. If transported on a data plane, a source coding algorithm may be applied to the training data. However, if the training dataset is provided to the network via RRC signaling or MAC CE, overhead reduction can be provided according to the embodiments disclosed herein. Even if beam reporting (i.e., from UE to network) and provisioning of training datasets for performance monitoring (i.e., from network to UE) are different, overhead reduction schemes for them can be leveraged against each other.

[0094] Examples of overhead reduction schemes, including strong beam selection and indication, reference beam (strongest beam) selection and indication, quantization scheme, and quantizer design, can be found, for example, in U.S. Patent Provisional Application No. 63 / 501,875, which is incorporated herein by reference. A first set of downlink reference signals can be shared for AI / ML inference and / or data acquisition and / or performance monitoring.

[0095] A second set of downlink reference signals can be shared for AI / ML inference and / or data acquisition and / or performance monitoring. AI / ML inference and / or data acquisition and / or performance monitoring may each have a separate first set of downlink reference signals. AI / ML inference and / or data acquisition and / or performance monitoring may each have a separate second set of downlink reference signals.

[0096] Figure 7 shows an exemplary architecture of the wireless communication system 700 according to embodiments disclosed herein. The following description is provided for an exemplary wireless communication system 700 that operates in conjunction with LTE system standards and / or 5G or NR system standards, as provided by 3GPP technical specifications.

[0097] As shown in Figure 7, the wireless communication system 700 includes UE702 and UE704 (however, any number of UEs can be used). In this example, UE702 and UE704 are shown as smartphones (e.g., handheld touchscreen mobile computing devices that can connect to one or more cellular networks), but it may also include any mobile or non-mobile computing devices configured for wireless communication.

[0098] UE702 and UE704 may be configured to communicate with RAN706. In embodiments, RAN706 may be NG-RAN, E-UTRAN, etc. UE702 and UE704 utilize connections (or channels) with RAN706 (referred to as connection 708 and connection 710, respectively), each of which has a physical communication interface. RAN706 may include one or more base stations (such as base stations 712 and 714) that enable connections 708 and connection 710.

[0099] In this example, connections 708 and 710 are air interfaces to enable such communication coupling and may correspond to RAT(s) used by RAN706, such as LTE and / or NR.

[0100] In some embodiments, UE702 and UE704 can also directly exchange communication data via the sidelink interface 716. UE704 is configured to access an access point (shown as AP718) via connection 720, as illustrated. For example, connection 720 may include a local radio connection, such as a connection compliant with any IEEE 802.11 protocol, and AP718 may include a Wi-Fi® router. In this example, AP718 may be connected to another network (e.g., the Internet) without going through CN724.

[0101] In this embodiment, UE702 and UE704 can be configured to communicate with each other or with base stations 712 and / or 714 using orthogonal frequency division multiplexing (OFDM) communication signals via a multi-carrier communication channel according to various communication technologies, such as orthogonal frequency division multiple access (OFDMA) communication technology (for downlink communication) or single-carrier frequency division multiple access (SC-FDMA) communication technology (for uplink and ProSe or sidelink communication), but not limited to these, and the scope of the embodiment is not limited in this respect. The OFDM signal may include multiple orthogonal subcarriers.

[0102] In some embodiments, all or part of base stations 712 or 714 may be implemented as one or more software entities running on a server computer as part of a virtual network. In addition, or in other embodiments, base stations 712 or 714 may be configured to communicate with each other via interface 722. In embodiments where the wireless communication system 700 is an LTE system (e.g., when CN724 is an EPC), interface 722 may be an X2 interface. The X2 interface may be defined between two or more base stations (e.g., two or more eNBs) connected to the EPC, and / or between two eNBs connected to the EPC. In embodiments where the wireless communication system 700 is an NR system (e.g., when CN724 is a 5GC), interface 722 may be an Xn interface. The Xn interface may be defined between two or more base stations (e.g., two or more gNBs) connected to the 5GC, between base station 712 (e.g., a gNB) and an eNB connected to the 5GC, and / or between two eNBs connected to the 5GC (e.g., CN724).

[0103] RAN706 is shown to be communicatively coupled to CN724. CN724 may include one or more network elements 726 configured to provide various data and telecommunications services to customers / subscribers (e.g., users of UE702 and UE704) connected to CN724 via RAN706. Components of CN724 may be implemented in one or separate physical devices, including components for reading and executing instructions from machine-readable or computer-readable media (e.g., non-temporary machine-readable storage media).

[0104] In this embodiment, CN724 may be an EPC, and RAN706 may be connected to CN724 via S1 interface 728. In this embodiment, S1 interface 728 may be divided into two parts: an S1 user plane (S1-U) interface that carries traffic data between base station 712 or base station 714 and a serving gateway (S-GW), and an S1-MME interface which is a signaling interface between base station 712 or base station 714 and mobility management entities (MMEs).

[0105] In the embodiment, CN724 may be a 5GC, and RAN706 may be connected to CN724 via NG interface 728. In the embodiment, NG interface 728 may be divided into two parts: an NG user plane (NG-U) interface that carries traffic data between base station 712 or base station 714 and user plane functions (UPF), and an S1 control plane (NG-C) interface that is a signaling interface between base station 712 or base station 714 and access and mobility management functions (AMFs).

[0106] Generally, the application server 730 may be an element that provides applications using Internet Protocol (IP) bearer resources (e.g., packet-switched data services) with the CN724. The application server 730 may also be configured to support one or more communication services (e.g., VoIP sessions, group communication sessions, etc.) for the UE702 and UE704 via the CN724. The application server 730 may communicate with the CN724 via the IP communication interface 732.

[0107] Figure 8 shows a system 800 for performing signaling 834 between a wireless device 802 and a network device 818 according to an embodiment disclosed herein. System 800 may be part of a wireless communication system as described herein. The wireless device 802 may be, for example, a UE of the wireless communication system. The network device 818 may be, for example, a base station (e.g., eNB or gNB) of the wireless communication system.

[0108] The wireless device 802 may include one or more processors 804. The processors 804 can execute instructions to perform various operations of the wireless device 802, as described herein. The processors 804 may include, for example, one or more baseband processors implemented using a central processing unit (CPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a controller, a field programmable gate array (FPGA) device, another hardware device, a firmware device, or any combination thereof configured to perform the operations described herein.

[0109] The wireless device 802 may include a memory 806. The memory 806 may be a non-temporary computer-readable storage medium that stores instructions 808 (for example, instructions being executed by processor(s) 804). Instructions 808 may also be referred to as program code or computer programs. The memory 806 may also store data used by processor(s) 804 and results calculated by processor(s) 804.

[0110] The wireless device 802 may include one or more transceivers 810 that include radio frequency (RF) transmitter and / or receiver circuits that use the antenna(s) 812 of the wireless device 802 to facilitate signaling to and from the wireless device 802 with other devices (e.g., network device 818) according to the corresponding RAT (e.g., signaling 834).

[0111] The wireless device 802 may include one or more antennas 812 (e.g., one, two, four, or more). In embodiments having multiple antennas 812, the wireless device 802 can leverage the spatial diversity of such multiple antennas 812 to transmit and / or receive multiple different data streams on the same time and frequency resources. This behavior is sometimes referred to as multiple input multiple output (MIMO) behavior (referring to the multiple antennas used in each of the transmitting and receiving devices that enable this embodiment). MIMO transmission by the wireless device 802 can be achieved by precoding (or digital beamforming) applied in the wireless device 802 to multiplex the data streams across the antennas 812 according to known or assumed channel characteristics, so that each data stream is received at a desired location in the spatial domain (e.g., the location of the receiver associated with that data stream) with appropriate signal strength relative to the other streams. Certain embodiments may employ a single-user MIMO (SU-MIMO) method (where all data streams are directed to a single receiver) and / or a multi-user MIMO (MU-MIMO) method (where individual data streams may be directed to individual (different) receivers at different locations within the spatial domain).

[0112] In certain embodiments having multiple antennas, the wireless device 802 may implement analog beamforming technology so that the phase of the signals transmitted by antenna(s) 812 is relatively adjusted so that the (joint) transmission of antenna(s) 812 can be directed (this is sometimes referred to as beam steering).

[0113] The wireless device 802 may include one or more interfaces 814. Interfaces 814 can be used to provide inputs to or outputs from the wireless device 802. For example, a wireless device 802 that is a UE may include interfaces 814 such as a microphone, speaker, touchscreen, or buttons to enable inputs and / or outputs to the UE by a user of the UE. Other interfaces of such a UE may consist of transmitters, receivers, and other circuits (other than, for example, the transceiver(s) 810 / antenna(s) 812 already described) that enable communication between the UE and other devices and may operate according to known protocols (e.g., Wi-Fi®, Bluetooth®, etc.).

[0114] The wireless device 802 may include a logged data module 816. The logged data module 816 can be implemented via hardware, software, or a combination thereof. For example, the logged data module 816 can be implemented as an instruction 808 stored in a processor, circuitry, and / or memory 806 and executed by a processor(s) 804. In some examples, the logged data module 816 can be integrated within a processor(s) 804 and / or a transceiver(s) 810. For example, the logged data module 816 can be implemented by a combination of software components (e.g., executed by a DSP or general-purpose processor) and hardware components (e.g., logic gates and circuits) within a processor(s) 804 or a transceiver(s) 810.

[0115] The logged data module 816 can be used for various embodiments of this disclosure, for example, the embodiments shown in Figures 1 to 7. The logged data module 816 is configured to generate UE capability reports, verify data collector credentials, and generate and transmit logged data.

[0116] The network device 818 may include one or more processors 820. The processors 820 can execute instructions to perform various operations of the network device 818, as described herein. The processors 820 may include, for example, one or more baseband processors implemented using a CPU, DSP, ASIC, controller, FPGA device, another hardware device, firmware device, or any combination thereof configured to perform the operations described herein.

[0117] The network device 818 may include memory 822. Memory 822 may be a non-temporary computer-readable storage medium that stores instructions 824 (for example, instructions being executed by processor(s) 820). Instructions 824 may also be referred to as program code or computer programs. Memory 822 may also store data used by processor(s) 820 and results calculated by processor(s) 820.

[0118] The network device 818 may include one or more transceivers 826 that can include RF transmitter and / or receiver circuits that use the antenna(s) 828 of the network device 818 to facilitate signaling to and from the network device 818 (e.g., signaling 834) with other devices (e.g., wireless device 802) according to the corresponding RAT.

[0119] The network device 818 may include one or more antennas 828 (e.g., one, two, four, or more). In embodiments having multiple antennas 828, the network device 818 can perform MIMO, digital beamforming, analog beamforming, beam steering, and the like, as described.

[0120] The network device 818 may include one or more interfaces 830. Interfaces 830 can be used to provide inputs to or outputs from the network device 818. For example, a network device 818 that is a base station may include interfaces 830 consisting of transmitters, receivers, and other circuits (other than transceivers 826 / antennas 828 already described) that enable the base station to communicate with other equipment in the core network and / or enable the base station to communicate with external networks, computers, databases, etc., for the purpose of operating, managing, and maintaining the base station or other equipment operably connected thereto.

[0121] The network device 818 may include an AI data module 832. The AI ​​data module 832 can be implemented via hardware, software, or a combination thereof. For example, the AI ​​data module 832 can be implemented as instructions 824 stored in a processor, circuitry, and / or memory 822 and executed by a processor(s) 820. In some examples, the AI ​​data module 832 can be integrated within a processor(s) 820 and / or a transceiver(s) 826. For example, the AI ​​data module 832 can be implemented by a combination of software components (e.g., executed by a DSP or general-purpose processor) and hardware components (e.g., logic gates and circuits) within a processor(s) 820 or a transceiver(s) 826.

[0122] The AI ​​data module 832 can be used for various aspects of this disclosure, for example, the aspects shown in Figures 1 to 7. The AI ​​data module 832 is configured to provide the UE with data collector credentials, data acquisition conditions, measurement signals, reporting format, and the like.

[0123] In one or more embodiments, at least one of the components shown in one or more of the aforementioned figures may be configured to perform one or more operations, techniques, processes and / or methods as described herein. For example, a baseband processor described herein in relation to one or more of the aforementioned figures may be configured to operate according to one or more of the examples described herein. In another embodiment, a circuit associated with a UE, base station, network element, etc., as described above in relation to one or more of the aforementioned figures may be configured to operate according to one or more of the examples described herein.

[0124] Any of the embodiments described above can be combined with any other embodiment (or combination of embodiments) unless otherwise specified. The above descriptions of one or more implementation forms are illustrative and illustrative, but are not intended to be exhaustive or to limit the scope of the embodiments to the exact forms disclosed. Modifications and variations are possible based on the above teachings or can be learned from the practice of various embodiments.

[0125] The embodiments and implementations of the systems and methods described herein may include a variety of operations that can be embodied by machine-executable instructions performed by a computer system. The computer system may include one or more general-purpose computers or dedicated computers (or other electronic devices). The computer system may include hardware components that include specific logic for performing operations, or it may include a combination of hardware, software, and / or firmware.

[0126] It should be recognized that the systems described herein include descriptions of specific embodiments. These embodiments may be combined into a single system, partially combined into other systems, divided into multiple systems, or otherwise divided or combined. In addition, parameters, attributes, aspects, etc. of one embodiment are intended to be used in another embodiment. Parameters, attributes, aspects are described in one or more embodiments for clarity only, and it should be recognized that parameters, attributes, aspects, etc. may be combined with or substituted for parameters, attributes, aspects, etc. of another embodiment unless specifically abandoned herein.

[0127] It should be fully understood that the use of personally identifiable information should adhere to privacy policies and practices that are generally recognized as meeting or exceeding industry or government requirements for maintaining user privacy. In particular, personally identifiable information data should be managed and handled in a manner that minimizes the risk of unintended or unauthorized access or use, and the nature of permitted use should be clearly indicated to the user.

[0128] While the foregoing has been described in some detail for clarity, it will be clear that certain changes and modifications can be made without departing from the principles. It should be noted that there are many alternative ways of implementing both the processes and apparatus described herein. Therefore, these embodiments should be considered illustrative and not limiting, and the description is not limited to the details given herein and may be modified within the appended claims and equivalents.

Claims

1. A method of wireless communication using user equipment (UE), wherein the method is: A UE capability report, wherein the UE capability report indicates that the UE supports the collection of logged data for artificial intelligence (AI) training, and the UE capability report is transmitted to a network node. Receiving a data collection request from the aforementioned network node, Receiving data collector credentials for a data collection server corresponding to the aforementioned network node, Verifying the aforementioned data collector credentials, To generate logged data, measurements are performed and relevant information is collected. A method comprising transmitting the logged data to the network node.

2. The method according to claim 1, further comprising transmitting the data collector credentials to a data collector credential verifier in order to verify the data collector credentials.

3. The method according to claim 1, wherein the data collector credentials include the identification information of the data collector, a verification key, and a timestamp.

4. The method according to claim 1, wherein the UE capability report indicates one or more sub-functions supported by the UE, the sub-functions include at least one of the following: data acquisition for channel status information (CSI), data acquisition for beam management (BM), data acquisition for positioning, number of ports, or sample size.

5. The method according to claim 1, further comprising transmitting to the network node a response to the data collection request indicating whether the UE accepts or rejects the data collection request.

6. The method according to claim 1, wherein the logged data includes a combined report that establishes a link between sets of measurement values.

7. The method according to claim 1, further comprising receiving a data acquisition configuration from the network node, wherein the data acquisition configuration includes data acquisition conditions indicating that the UE performs the measurement and collects the relevant information, based on at least one of signal-to-noise ratio, time, or geographical area.

8. The method according to claim 1, further comprising transmitting the logged data to a server, wherein transmitting the logged data to the network node includes transmitting a uniform resource locator (URL) corresponding to the location of the logged data, the URL being transmitted after a predetermined time has elapsed since the logged data was transmitted to the server.

9. The method according to claim 1, further comprising transmitting a data quality indicator for the logged data.

10. The method according to claim 1, wherein the logged data is transmitted in batches to the network node along with other additional data samples collected at different times from the measured values ​​and the related information.

11. A method for wireless communication by network nodes, wherein the method is The system receives a UE capability report from a user device (UE), the UE capability report indicating that the UE supports the collection of logged data for artificial intelligence (AI) training, Sending a data collection request to the aforementioned UE, To transmit data collector credentials for the data collection server corresponding to the aforementioned network node, Requesting the measurement values ​​and related information for the aforementioned AI training, A method comprising receiving logged data in batches along with additional data samples collected at different times than the measured values ​​and the associated information.

12. The method according to claim 11, wherein the data collector credentials include the identification information of the data collector, a verification key, and a timestamp.

13. The method according to claim 11, wherein the UE capability report indicates one or more sub-functions supported by the UE, the sub-functions include at least one of the following: data acquisition for channel status information (CSI), data acquisition for beam management (BM), data acquisition for positioning, number of ports, or sample size.

14. The method according to claim 11, further comprising receiving a response from the UE indicating whether the UE accepts or rejects the data collection request.

15. The method according to claim 11, wherein the logged data includes a combined report that establishes a link between sets of measurement values.

16. The method according to claim 11, further comprising transmitting a data acquisition configuration, the data acquisition configuration including data acquisition conditions indicating that the UE performs the measurement and collects the relevant information, based on at least one of signal-to-noise ratio, time, or geographical area.

17. The method according to claim 11, wherein the logged data is received from the server after a predetermined time has elapsed since the logged data was sent to the server.

18. The method according to claim 11, further comprising receiving a data quality indicator for the logged data.

19. An apparatus comprising means for carrying out the method described in any one of claims 1 to 18.

20. A computer-readable medium containing instructions, wherein when the instructions are executed by one or more processors of an electronic device, the electronic device is instructed to perform the method according to any one of claims 1 to 18.

21. An apparatus comprising a logic, module, or circuit that performs the method according to any one of claims 1 to 18.

Images