Apparatus and method for monitoring trained machine learning processes associated with location determination

Abstract

Methods, systems, and apparatus for monitoring or configuring one or more user equipments (UEs). For example, a computing device may send a report request for positioning data to a user device. Additionally, the computing device may receive report data indicating a comparison between first positioning data generated from the first process and second positioning data generated from the second process. Further, the computing device may generate an instruction based on the report data and send the instruction to the user device, the instruction causing the user device to implement the first process.

Description

Technical Field

[0001] The disclosed embodiments generally relate to monitoring a trained machine learning process. Background Art

[0002] Wireless communication systems can provide various telecommunication services, including, for example, audio, video, data, messaging, and network access, among other examples. For example, wireless communication systems can allow communication between various devices (e.g., IoT devices). These wireless communication systems can be based on various technologies, such as code division multiple access (CDMA) systems, time division multiple access (TDMA) systems, frequency division multiple access (FDMA) systems, orthogonal frequency division multiple access (OFDMA) systems, single carrier frequency division multiple access (SC-FDMA) systems, time division synchronous code division multiple access (TDSCDMA) systems, long term evolution (LTE) systems, WiMax systems, and evolved high speed packet access (HSPA+) systems. These systems and other wireless communication systems may conform to certain standards, such as third generation (3G) broadband cellular network technology, fourth generation (4G) broadband cellular network technology, and the most recent fifth generation (5G) broadband cellular network technology (also referred to as new radio (NR)).

[0003] A wireless communication system may include multiple base stations (BS) and multiple user equipments (UE). In some examples, the BS may enable multiple UEs to perform wireless communication. Additionally, the wireless communication system may also provide location services. For example, the wireless communication system may include a location management function (LMF) that may provide location services to multiple UEs. Summary of the invention

[0004] According to one aspect, an apparatus may include a non-transitory machine-readable storage medium storing instructions, and at least one processor coupled to the non-transitory machine-readable storage medium. At least one processor may be configured to send a report request for positioning data to a user device. In some examples, the report request may cause the user device to perform an operation in a first mode. In some cases, the operation may include: implementing a first process to generate first positioning data, and implementing a second process to generate second positioning data, and generating report data, the report data indicating a comparison between the first positioning data and the second positioning data. Additionally, at least one processor may be configured to receive report data from a user device. In addition, at least one processor may be configured to: generate instructions based on the report data and send instructions to the user device, wherein the instructions cause the user device to implement at least one of the first process or the second process.

[0005] According to another aspect, a non-transitory machine-readable storage medium stores instructions that, when executed by at least one processor of a location server, cause the at least one processor to perform operations including sending a report request for positioning data to a user device. In some examples, the report request may cause the user device to perform operations in a first mode. In some examples, the operations of the first mode may include: implementing a first process to generate first positioning data, and implementing a second process to generate second positioning data, and generating report data, the report data indicating a comparison between the first positioning data and the second positioning data. Additionally, the operations may include receiving the report data from the user device. Furthermore, the operations may include: generating instructions based on the report data and sending the instructions to the user device, wherein the instructions cause the user device to implement at least one of the first process and / or the second process.

[0006] According to another aspect, a computer-implemented method includes sending a report request for positioning data to a user device. In some examples, the report request may cause the user device to perform operations in a first mode. In some instances, the operation of the first mode may include: implementing a first process to generate first positioning data, and implementing a second process to generate second positioning data, and generating report data, the report data indicating a comparison between the first positioning data and the second positioning data. Additionally, the computer-implemented method may include receiving the report data from the user device. In addition, the computer-implemented method may include: based on the report data, generating instructions and sending the instructions to the user device, wherein the instructions cause the user device to implement at least one of the first process and / or the second process.

[0007] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only, and do not limit the claimed content of the present invention. In addition, the accompanying drawings, which are incorporated into this specification and constitute part of this specification, illustrate various aspects of the present invention and are used together with the description to explain the principles of the disclosed embodiments set forth in the appended claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 is a block diagram of an exemplary wireless communication system according to some exemplary embodiments;

[0009] Figure 2-Figure 6 is a block diagram illustrating portions of an exemplary wireless communication system according to some exemplary embodiments;

[0010] Figure 7 is a flow chart of an exemplary process 700 for determining performance of a process deployed by a UE 104; and

[0011] Figure 8is a flow diagram of an exemplary process 800 for generating report data in accordance with some exemplary embodiments.

[0012] Like reference numbers and designations in the various drawings indicate like elements. DETAILED DESCRIPTION

[0013] Although the features, methods, devices, and systems described herein may be embodied in various forms, some exemplary and non-limiting embodiments are shown in the drawings and described below. Some components described in this disclosure are optional, and some implementations may include additional components, different components, or fewer components than those explicitly described in this disclosure.

[0014] The embodiments described herein are directed to a wireless communication system, which includes a computing device or system and several user equipments (UEs). Such an embodiment may enable a computing device or system to monitor the performance of a specific process implemented on one or more UEs in several UEs. Additionally, when a computing device or system monitors the performance of a specific process implemented on each UE in one or more UEs, each UE in one or more UEs may be configured to implement a first or reporting mode. When the corresponding UE is in the first mode, the UE may generate first data (e.g., first measurement data) associated with a specific process and second data (e.g., second measurement data) associated with a traditional process. In addition, a computing device or system may determine the performance of a specific process based in part on the first data and the second data. In addition, based on the performance of a specific process, a computing device or system may modify or change the configuration of one or more UEs (e.g., so that one or more UEs operate in the second mode instead of the first mode). In some cases, when the corresponding UE operates in the second mode, the corresponding UE may implement another process to generate a third data (e.g., a third measurement data). In this case, the computing device or system may provide location services for the corresponding UE using the third data.

[0015] A. Exemplary Wireless Communication Systems

[0016] Figure 1A block diagram of an example wireless communication system 100 (e.g., a 5G wireless communication system) is shown, which includes one or more computing systems (e.g., location management function (LMF) computing system 102A, LMF computing system 102B, LMF computing system 102C, LMF computing system 102D, LMF computing system 102E, and LMF computing system 102F, at least one base station (BS) 103, and one or more user equipment (UE) 104 (e.g., UE 104A, UE 104B, UE 104C, UE 104D, UE 104E, and UE 104F), etc. The one or more computing systems (e.g., LMF computing system 102) and the one or more UE Each of 104 may be operatively connected to one or more communication networks and interconnected across one or more networks. Although the wireless communication system 100 may include additional components (e.g., access and mobility management function (AMF), session management function (SMF), relay stations, and any other suitable components), they are not shown for simplicity. Additionally, although the wireless communication system 100 may only show one BS 103, six LMF computing systems 102, and six UEs 104, the wireless communication system 100 may include any number of LMF computing systems 102, BSs 103, and UEs 104.

[0017] LMF computing systems 102 (e.g., LMF computing system 102A, LMF computing system 102B, LMF computing system 102C, LMF computing system 102D, LMF computing system 102E, and LMF computing system 102F) may each represent a computing system that includes one or more servers (e.g., server 202), and one or more tangible, non-transitory storage devices for storing executable code, application engines, or application modules. Each of the one or more servers may include one or more processors that may be configured to execute stored code, application engines or modules, or portions of application programs to perform operations consistent with the disclosed exemplary embodiments. For example, with reference to Figure 2 , one or more servers of the LMF computing system 102A may include a server 202 having one or more processors configured to execute stored code, application engines, or portions of modules or applications maintained in one or more tangible non-transitory memories.

[0018] In some cases, the LMF computing system 102 may correspond to a discrete computing system, while in other cases, the LMF computing system 102 may correspond to a distributed computing system having multiple computing components distributed across an appropriate computing network. In addition, the LMF computing system 102 may also include one or more communication interfaces (e.g., one or more wireless transceivers) coupled to one or more processors for accommodating communication with other computing systems and devices operating within the wireless communication system 100 ( Figure 1 ) (not shown) (e.g., additional components of the wireless communication system 100 (e.g., access and mobility management function (AMF), session management function (SMF), relay stations, and any other suitable components)) across wired or wireless Internet communications of the communication network.

[0019] In some examples, the LMF computing system 102 can be associated with a wireless communication service provider and can be operated by one or more operators of the wireless communication service provider. Additionally, the LMF computing system 102 can be configured to provide supported location services to each of the one or more UEs 104. For example, referring to Figure 1 , the LMF computing system 102A may receive measurement data for each beam transmitted by the BS 103 and detected by the UE 104A from the UE 104A. In some cases, each element of the measurement data may include measurement information of the detected beam determined by the corresponding UE 104. Additionally, the measurement information may be associated with specific data or attributes (e.g., assistance data, positioning frequency layer (PFL) ID, positioning reference signal (PRS) resource ID, TRP, PRS resource set ID, timestamp, etc.). Examples of measurement information that may be included in the measurement data include reference signal time difference (RSTD), received signal received power (RSRP), reference signal received power path (RSRPP), time difference (Rx-Tx) between receiving and transmitting measurement values, and information characterizing the position estimate of the corresponding UE 104 (e.g., UE 104A). Based on the measurement data of each UE in one or more UEs 104, the LMF computing system 102 may provide location services to the corresponding UE 104 (e.g., generate assistance data and send the assistance data to the UE 104A). For example, the assistance data may include reference time, reference position, ionospheric model, earth position parameters, time offset, differential correction, ephemeris and clock model, health, data bit assistance, acquisition assistance, almanac, UTC model, and carrier phase data. In some cases, one or more UEs 104 may request location services (e.g., assistance data) from the LMF computing system 102. In this case, the LMF computing system 102 may provide location services to each of the one or more UEs 104 that request location services.

[0020] In other examples, the LMF computing system 102 can monitor the performance of the new process implemented by one or more UEs 104 in the plurality of UEs 104. Additionally, the LMF computing system 102 can monitor the performance of the new process based on the measurement data associated with the new process and the measurement data associated with the traditional process. In such an example, each of the one or more UEs 104 can implement the new process and the traditional process to generate the measurement data associated with the new process and the traditional process. In some cases, the LMF computing system 102 determines the performance of the new process by comparing the measurement data of the new process with the measurement data of the traditional process. Additionally, the LMF computing system 102 can determine the difference between the measurement data of the new process and the measurement data of the traditional process, and whether the difference exceeds the difference threshold or is lower than the quality / standard threshold. In an example where the LMF computing system 102 determines that the difference between the measurement data of the new process and the measurement data of the traditional process is higher than the quality or difference threshold (e.g., lower than a predetermined standard), the LMF computing system 102 can modify or change the configuration of one or more UEs 104. In some cases, a new process may be associated with a trained machine learning process. In other cases, a traditional process may be associated with a trained machine learning process.

[0021] As an example and reference Figure 2 To facilitate the performance of one or more of these exemplary processes, the LMF computing system 102 (e.g., the LMF computing system 102A) may maintain a data repository (e.g., the data repository 204) in one or more tangible non-transitory memories, including but not limited to a UE data store (e.g., the UE data store 206). Figure 2 As shown, a UE data store (e.g., UE data store 206) may store a UE data set for each of one or more UEs 104 in communication with the LMF computing system 102. As described herein, the UE data set for each of the UEs 104 (e.g., UE data 206A associated with UE 104A) may include data identifying the corresponding UE 104 (e.g., a corresponding serial number or identification number), data identifying one or more processes that may be implemented by the corresponding UE 104 (e.g., a new first trained machine learning process and a second conventional trained machine learning process), data including parameters of each of the one or more processes (e.g., model parameters), and data indicating a performance status of the one or more processes. In some cases, examples of the performance status of the one or more processes include: a performance status indicating that the performance of a particular process is below a predetermined standard, and a performance status indicating that the performance of a particular trained machine learning process is below a predetermined standard.

[0022] In addition, and to facilitate the performance of any of the exemplary processes described herein, the LMF computing system 102 (e.g., the LMF computing system 102A) may include one or more servers (e.g., the server 202), which may also maintain an application repository (e.g., the application repository 208) within one or more tangible non-transitory memories. For example, the application repository 208 (or any application repository of any LMF computing system 102) may maintain a UE engine 208A, etc. The UE engine 208A may initiate monitoring of the performance of one or more processes implemented by each of the one or more UEs 104 by generating and sending (e.g., broadcasting) a report request to each of the one or more UEs 104. In some examples, the report request may include parameter data identifying one or more monitoring parameters. In some cases, based on the one or more monitoring parameters, each of the one or more UEs 104 may generate measurement data from a particular process (e.g., a new and trained machine learning process) and a traditional process, which may be comparable for the purpose of determining the performance of the particular process. For example, each measurement data generated from the specific process and the traditional process may include the same type of data or attributes and have matching or similar timestamps. In addition, each UE 104 in one or more UEs 104 may generate report data indicating the performance of the specific process based on the comparable measurement data of the specific process and the measurement data of the traditional process.

[0023] In some examples, one or more monitoring parameters of parameter data may include timing parameters. Timing parameters may indicate that the corresponding UE 104 can implement the time period of the process to be monitored and the traditional process, so that the corresponding measurement data generated may have elements with timestamps that match or are within the predetermined time threshold or margin. In such an example, the process to be monitored (e.g., new and trained machine learning process) and the traditional process (e.g., trained traditional machine learning process) may each have different measurement cycle configurations, and may monitor and determine measurements at different time intervals (e.g., RAN 4 requirements). Therefore, the timing parameters may indicate the extended measurement period of the process to be monitored and the traditional process (e.g., extended RAN 4 requirements). In this way, one or more elements of the measurement data of the process to be monitored and one or more elements of the measurement data of the traditional process may have matching timestamps or timestamps within the predetermined time threshold or margin. In other examples, one or more monitoring parameters of parameter data may include model or process parameters. Model or process parameters may identify a specific process that the LMF computing system 102 can monitor or a model that the UE 104 can implement.

[0024] In various examples, one or more monitoring parameters of the parameter data may include resource parameters. The resource parameters may identify one or more resources, attributes, or data that the corresponding UE 104 can measure from one or more detected beams sent by the BS 103. In some cases, such resources, attributes, or data may be associated with auxiliary data, PFL ID, PRS resource ID, TRP, and PRS resource set ID. In addition, based on the resource parameters, the UE 104 may implement a process to be monitored and a conventional process, each of which generates measurement data including measurements or measurement information of the same resource, attribute, or data. For example, based on the resource parameters, the UE 104 may implement a process to be monitored and a conventional process, each of which generates measurement data including data identifying the RSTD. In other cases, the resource parameters may identify a subset of resources, attributes, or data that the corresponding UE 104 can measure from one or more detected beams sent by the BS 103, such as a subset of TRP resources. In this case, based on the resource parameters, the UE 104 may implement at least two processes, including a process to be monitored and a conventional process, wherein each process generates measurement data (e.g., measurements, measurement information) of the same subset of resources, attributes, or data.

[0025] In various cases, the resource parameters may identify a first subset of resources, an attribute or data, and a second subset of resources, an attribute or data that the corresponding UE 104 may measure from one or more beams transmitted by the BS 103. Additionally, the resource parameters may indicate which process (e.g., a process to be measured or a conventional process) will generate measurement data associated with which resource subset (e.g., the first resource subset or the second resource subset). For example, the resource parameters may indicate that the first resource subset is associated with the conventional process, wherein the first resource subset includes a TRP (e.g., a reference TRP, a line-of-sight (LOS) re-TRP, a non-line-of-sight (NLOS) TRP, a service TRP, or any combination thereof). Additionally, the resource parameters may indicate that the second resource subset is associated with the process to be measured. For example, the second resource set may include resources, attributes, or data associated with auxiliary data, a PFL ID, a PRS resource ID, a TRP, a PRS resource set ID, a PRS resource, or any combination thereof. Additionally, and based on the resource parameters, the UE 104 may implement conventional procedures to generate measurement data for a first subset of resources, attributes, or data, and implement procedures to be monitored to generate measurement data for a second subset of resources, attributes, or data.

[0026] In some examples, the parameter data of the report request may be associated with a specific positioning method (e.g., UE-assisted or UE-based). In these examples, the measurement data may be associated with the positioning method associated with the parameter data. For example, the report request may include parameter data associated with the Multi-RTT positioning method. In such an example, the corresponding UE 104 may implement a first process to be monitored and a second traditional process, which generate measurement data associated with the Multi-RTT positioning method. For example, the measurement data may include data identifying the UE ERx-Tx, RSRP and / or RSRPP of each process in the first process and the second traditional process. In another example, the report request may include parameter data associated with the DL-AoD positioning method. In such an example, the corresponding UE 104 may implement a process to be monitored and a second traditional process, which generate measurement data associated with the DL-AoD positioning method. For example, the measurement data may include data identifying RSRP and / or RSRPP. In another example, the report request may include parameter data associated with a UE-based positioning method. In such an example, the corresponding UE 104 may implement a process to be monitored and a second traditional process, which generate measurement data associated with the positioning method. For example, the measurement data may include data identifying and characterizing a position estimate of the UE.

[0027] In addition, if Figure 2 As shown, the UE engine 208A can modify or change the configuration of one or more UEs 104 based on the measurement data of the process to be monitored and the measurement data of the traditional process. For example, the UE engine 208A can receive report data generated by each UE in the one or more UEs 104. The report data can indicate the performance of the process to be monitored. In some cases, the report data can indicate that the performance of the process is higher than a predetermined standard or lower than a quality threshold. In an example in which the UE engine 208A determines that the performance of the process to be monitored is lower than the quality / standard threshold based on the report data, the UE engine 208A can modify or change the configuration of the corresponding one or more UEs 104. For example, the report data may indicate that the difference between the measurement data of the new process and the measurement data of the traditional process exceeds the difference threshold. In an example in which the UE engine 208A determines that the difference between the measurement data of the new process and the measurement data of the traditional process exceeds the difference threshold based on the report data, the UE engine 208A can modify or change the configuration of one or more UEs 104.

[0028] In some cases, the report data may include raw data of first measurement data generated by a specific process to be monitored and second measurement data generated by another process, the latter being used to determine the performance of the specific process. In this case, the LMF computing system 102 (e.g., LMF computing system 102A) may process the raw data of the second measurement data to determine the accuracy of the first measurement data and the performance of the specific process to be monitored. For example, the LMF computing system 102 may obtain report data of UE 104 (e.g., UE 104A). The report data may include measurement data of a first trained machine learning process applied by UE 104 to a detected beam sent from BS 103. The measurement data may include data that characterizes and identifies a position estimate of UE 104. Additionally, the report data may include raw measurement data of a second traditional process applied by UE 104 to a detected beam sent from BS 103. Based on the raw measurement data of the second traditional process, the LMF computing system 102 may determine a position estimate of the UE derived from the raw measurement data of the second traditional process. In addition, the LMF computing system 102 may compare the location estimate of the first trained machine learning process with the location estimate determined from the second traditional process to determine the accuracy and performance of the first trained machine learning process. Based on such a determination, the LMF computing system 102 may modify or change the configuration of the UE 104. For example, the LMF computing system 102 may determine the difference between the value associated with the location estimate of the first trained machine learning process and the value associated with the location estimate determined from the second traditional process. Additionally, the LMF computing system 102 may determine whether the determined difference exceeds a difference threshold. In response to the LMF computing system 102 determining that the determined difference exceeds the difference threshold, the LMF computing system 102 may modify or change the configuration of the UE 104 (e.g., to operate in a mode that utilizes a second traditional process or another process to replace the first trained machine learning process to generate measurement data). Otherwise, in an example where the determined difference is at or below the difference threshold, the LMF computing system 102 may cause the UE 104 to continue to implement the first trained machine learning process to generate measurement data.

[0029] Reference Figure 1, BS 103, which may also be referred to as Node B, gNB, transmit receive point (TRP), access point (AP), etc., may provide communication coverage for a particular geographic area (e.g., geographic area 101). For example, geographic area 101 may correspond to a macro cell, a pico cell, a femto cell, or any other type of cell. To provide coverage, BS 103 may transmit one or more beams that cover at least a portion of geographic area 101. Each beam may include one or more carriers operating within a spectrum. For example, BS 103 may use one or more carriers associated with each beam to transmit data (e.g., PRS) in a downlink transmission to one or more UEs 104.

[0030] Additionally, each of the UEs 104 (e.g., UE 104A, UE 104B, UE 104C, UE 104D, UE 104E, and UE 104F) may detect one or more beams transmitted from the BS 103. Furthermore, each of the UEs 104 may deploy one or more processes on the detected one or more beams to generate corresponding measurement data. Figure 2 To facilitate performance of one or more of these exemplary processes, each of the UEs 104 (e.g., UE 104A) may include a computing device having one or more tangible, non-transitory memories (e.g., memory 212) storing data and / or software instructions, and one or more processors (e.g., processor 214) configured to execute the software instructions. In certain aspects, the one or more tangible, non-transitory memories may store applications, application engines or modules, and other code elements executable by one or more processors, such as, but not limited to, an executable web browser (e.g., Google Chrome) TM , Apple Safari TM , etc.), and executable applications associated with the wireless communication system 100 (e.g., application 212B). Figure 2 In some cases not shown in the figure, the memory 212 may also include one or more structured or unstructured data repositories or databases, and each of the UEs 104 may maintain one or more elements of device data in the one or more structured or unstructured data repositories or databases. For example, the elements of the device data may uniquely identify the UE 104 within the wireless communication system 100, and may include, but are not limited to, an Internet Protocol (IP) address assigned to the UE 104 or a medium access control (MAC) layer assigned to the UE 104.

[0031] Each of the UEs 104 may include an antenna unit (e.g., antenna unit 216A) configured to monitor and / or receive data transmissions or beams / resources from at least BS 103. In some examples, the antenna unit may include one or more antennas, and each of the one or more antennas may detect one or more beams transmitted from BS 103. As described herein, each of the UEs 104 may determine one or more measurements associated with each beam or resource transmitted from BS 103 (e.g., RSTD, RSRPP, RSRP, time difference between received and transmitted measurement values, and position estimate of the corresponding UE 104). Additionally, each of the UEs 104 may include a display unit (e.g., display unit 216B) configured to present interface elements to a corresponding user (e.g., user of UE 104), and an input unit (e.g., input unit 216C) configured to receive input from the user (e.g., in response to an interface element presented by the display unit). For example, the display unit may include but is not limited to an LCD display unit or other suitable types of display units, and the input unit 216C may include but is not limited to a key, a keyboard, a touch screen, a voice control technology or an suitable type of input unit. Figure 1 (not shown in the figure), the functions of the display unit and the input unit can be combined into a single device, for example, a pressure-sensitive touch screen display unit, which can present interface elements and receive input from the user. In addition, each of the UEs 104 may also include a communication interface (e.g., communication interface 216D), for example, a wireless transceiver device coupled to a processor (e.g., processor 214) of the corresponding UE 104, and a processor configured to communicate via one or more communication protocols (e.g., NFC, cellular communication protocols (e.g. etc.) or any other suitable communication protocol) to establish and maintain communications with a communication network.

[0032] Examples of UE 104 (e.g., UE 104A, UE 104B, UE 104C) may include, but are not limited to, personal computers, laptop computers, tablet computers, notebook computers, handheld computers, personal digital assistants, portable navigation devices, mobile phones, smart phones, wearable computing devices (e.g., smart watches, wearable activity monitors, wearable smart jewelry, and glasses and other optical devices including optical head-mounted displays (OHMDs)), embedded computing devices (e.g., communicating with smart textiles or electronic fabrics), and any other type of computing device that can be configured to store data and software instructions, execute software instructions to perform operations and / or display information on an interface device or unit (e.g., display unit 216B). In some examples, UE 104 may be a vehicle. In other examples, UE 104 may be an autonomous vehicle (e.g., a vehicle with autonomous driving capabilities). In some examples, UE 104 may also establish communications with one or more additional computing systems or devices operating within the wireless communication system 100 across a wired or wireless communication channel (e.g., via a communication interface using any appropriate communication protocol).

[0033] In some examples, each of the UEs 104 can perform operations to determine measurements of one or more detected beams transmitted by the BS 103. Additionally, each of the UEs 104 can generate measurement data based on the determined measurements. For example, each element in the measurement data can include measurement information of the determined measurement at a specific point in time when the measurement was performed. In addition, each element can be associated with a timestamp generated by the corresponding UE 104. The timestamp can indicate a specific point in time when the measurement was performed. In some examples, each UE 104 can implement UE-assisted or UE-based positioning methods, such as multi-cell round-trip time (multi-RTT) positioning, downlink arrival time difference (DL-TDOA) positioning, and downlink transmission angle (DL-AoD) positioning methods, to determine such measurements or generate such measurement data. For example, when BS 103 operates in a wireless communication system such as New Radio (NR), one or more UEs 104 (e.g., UE 104A, UE 104B, and UE 104C) may implement UE-assisted or UE-based positioning methods, such as multi-cell round trip time (multi-RTT) positioning, downlink time difference of arrival (DL-TDOA) positioning, and downlink angle of emission (DL-AoD) positioning methods, to generate measurement data. In various cases, the LMF calculation system 102 may utilize the measurement data to transmit additional data (e.g., assistance data), which each of the one or more UEs 104 may utilize to determine its own position.

[0034] In some cases, each of the UEs 104 may operate in a normal or first mode associated with normal operation. When each of the UEs 104 operates in the first mode, the corresponding UE 104 (e.g., UE 104A) may perform operations to determine measurements of one or more detected beams transmitted by the BS 103 by utilizing a process associated with the first mode. In some examples, the process may be a newly deployed trained machine learning process.

[0035] In other cases, each of the UEs 104 may be configured to deploy or implement multiple processes for the purpose of determining the performance of one of the multiple processes. In this case, each of the UEs 104 may implement a mode associated with determining the performance of one of the multiple processes or operate in this mode by comparing data (e.g., measurement data generated from one process and another process (e.g., a traditional process)). For example, a specific UE 104 (e.g., UE 104A) may be configured to deploy a first trained machine learning process and a second trained machine learning process. In this case, both the first trained machine learning process and the second trained machine learning process are associated with determining the measurement of one or more beams transmitted from BS 103 and / or generating measurement data of one or more beams transmitted from BS 103. Additionally, when the specific UE 104 operates in the second mode, the UE 104 may implement the first trained machine learning process to generate the first measurement data, and implement the second trained machine learning process to generate the second measurement data. Based on the first measurement data and the second measurement data, the specific UE 104 may determine the performance of the first trained machine learning model. In some cases, a process used by one or more UEs 104 to determine the performance of another process may be specified by an operator of the wireless communication system 100 as a legacy process.

[0036] In various cases, one or more UEs 104 may determine the performance of a particular process based on a quality or standard threshold. For example, a particular UE 104 (e.g., UE 104A) may be configured to deploy a first trained machine learning process to generate first measurement data of a beam sent from BS 103, and to deploy a second trained traditional machine learning process to generate second measurement data of the beam. Additionally, the first trained machine learning process may be identified by the LMF computing system 102 as a process to be monitored, and the quality or standard threshold may be a predetermined difference threshold, which, if exceeded, may indicate that the performance of the first trained machine learning process is poor. In addition, the particular UE 104 may determine the difference between at least one element or measured value of the first measurement data and at least one element or measured value of the second measurement data, and compare the determined difference with a predetermined difference threshold. In an example where the determined difference exceeds a predetermined difference threshold, the particular UE 104 may determine that the first trained machine learning process is performed below the quality / standard threshold. In examples where the determined difference matches or is below a predetermined difference threshold, the particular UE 104 may determine that the first trained machine learning process is executed at or above a quality / standard threshold, respectively.

[0037] In an example where a particular UE 104 determines that a process identified as being to be monitored (e.g., a first trained machine learning process) is below a quality / standard threshold for execution, the UE 104 may send reporting data to the LMF computing system 102 indicating that the process to be monitored is below a quality / standard threshold for execution. In such an example, the LMF computing device 102 may modify or change the configuration of the particular UE 104. For example, based on the reporting data, the LMF computing system 102 may cause the particular UE 104 to operate in a third mode. In some cases, when the particular UE 104 operates in the third mode, the particular UE 104 may utilize a process for determining the performance of a process to be monitored (e.g., a second trained traditional machine learning process), or another process designated by the operator of the wireless communication system 100 as robust and reliable (e.g., a third trained traditional machine learning process). For example, in response to the LMF computing device 102 determining based on the reported data that a process to be monitored is performed below a quality / standard threshold, the LMF computing device 102 can cause the specific UE 104 to operate in a third mode and automatically utilize a process for determining the performance of the process to be monitored (e.g., a second trained traditional machine learning process), or other processes designated by the operator of the wireless communication system 100 as robust and reliable (e.g., a third trained traditional machine learning process), instead of the process to be monitored.

[0038] In an example where a specific UE 104 determines that a process identified as a process to be monitored (e.g., a first trained machine learning process) is at or above a quality / standard threshold for execution, the UE 104 may send report data indicating that the process to be monitored is at or above a quality / standard threshold for execution to the LMF computing device 102. In such an example, the LMF computing system 102 may enable the specific UE 104 to implement the process to be monitored to generate measurement data. Therefore, the LMF computing system 102 may provide location services to the specific UE 104 based on the measurement data associated with the process to be monitored. In some examples, the specific UE 104 may determine that a process identified as a process to be monitored (e.g., a first trained machine learning process) is at or above a quality / standard threshold for execution. In such an example, the specific UE 104 may not generate and / or send report data indicating that the process to be monitored is at or above a quality / standard threshold for execution to the LMF computing system. Instead, the particular UE 104 may automatically begin a process of implementing monitoring to generate measurement data that the LMF computing system 102 may use to provide location services to the particular UE 104 .

[0039] In some cases, the UE 104 may automatically and without receiving instructions from the LMF computing device 102 fall back to a default mode, a secure mode, or a third mode when the UE 104 determines that the process to be monitored (e.g., the first trained machine learning process) is executed below a quality / standard threshold. For example, a specific UE 104 (e.g., UE 104A) may determine that the first trained machine learning process is executed below a quality / standard threshold based on measurement data of the first trained machine learning process and measurement data of a second traditional process (e.g., the second trained traditional machine learning process). When the UE 104 determines that the first trained machine learning process is executed below a quality / standard threshold, the UE 104 may automatically fall back to the default mode, the secure mode, or the third mode and operate in that mode. In this case, the UE 104 may automatically fall back to the default mode, the secure mode, or the third mode and operate in that mode without communicating with the LMF computing system 102. Additionally, when UE 104 is operating in a default mode, a secure mode, or a third mode, UE 104 may deploy a process for determining the performance of a process to be monitored (e.g., a second trained conventional machine learning process), or another process designated by an operator of the wireless communication system 100 as robust and reliable (e.g., a third trained conventional machine learning process), to determine measurements of one or more detected beams transmitted from BS 103.

[0040] In various cases, each of the UEs 104 may determine measurements associated with one or more beams detected by the corresponding UE 104 and transmitted by the BS 103 and / or generate measurement data associated with one or more beams detected by the corresponding UE 104 and transmitted by the BS 103 based on one or more monitoring parameters including a report request received from the LMF computing system 102. For example, the LMF computing system 102 may send a report request to one or more UEs 104 (e.g., UE 104A). As described herein, the report request may include parameter data, and the parameter data may include one or more monitoring parameters, including timing parameters, resource parameters, and model parameters. In some examples, the report request may include data characterizing a quality / standard threshold (e.g., a value of a difference threshold). Additionally, based on one or more monitoring parameters, one or more UEs 104 may determine which process to monitor (based on the model parameters), the resources, attributes, or data to be measured from one or more detected beams transmitted from the BS 103 (based on the resource parameters), and the measurement time interval or period (based on the timing parameters). In addition, based on such a determination, one or more UEs 104 may implement an identified process, such as a first trained machine learning process, to make such measurements and generate corresponding measurement data. In addition, one or more UEs 104 may implement another or second process that generates measurement data that can be utilized to determine the performance of the identified process to be monitored. For example, one or more UEs 104 may implement a second process (e.g., a second trained traditional machine learning process) to make measurements based on timing parameters and resource parameters and generate corresponding measurement data associated with one or more detected beams sent from BS 103. In some cases, the process or model parameters may also identify other processes or a second process.

[0041] B. Computer-implemented techniques for updating machine learning processes

[0042] As described herein, LMF computing systems 102 (e.g., LMF computing system 102A, LMF computing system 102B, LMF computing system 102C, LMF computing system 102D, LMF computing system 102E, and LMF computing system 102F) can each be configured to monitor a new process (e.g., a first trained machine learning process) implemented by one or more UEs 104 (e.g., UE 104A, UE 104B, UE 104C, UE 104D, UE 104E, and UE 104F). Additionally, LMF computing system 102 can determine the performance of the new process based on measurement data of the new process and measurement data of a second process. As described herein, the second process can be a trained traditional machine learning process.

[0043] Reference Figure 3 , the executed UE engine 208A may perform operations to generate a report request 302, the report request 302 including parameter data 304. Figure 3 As shown, parameter data 304 may include one or more monitoring parameters (e.g., timing parameter 304A, resource parameter 304B, and modeling or processing parameter 304C). As described herein, one or more parameters (e.g., timing parameter 304A (e.g., a parameter indicating a time period or measurement cycle for the corresponding UE 104 to measure from the new process and the traditional process), resource parameter 304B (e.g., a parameter identifying one or more resources, attributes, or data that the corresponding UE 104 can measure from one or more detected beams sent from BS 103), and modeling or processing parameter 304C (e.g., a parameter at least identifying a process to be monitored (e.g., a new process), and the parameter may identify a second process (e.g., a traditional process) with which measurement data is generated to compare the measurement data of the process to be monitored)), modeling or processing parameter 304C may enable each UE 104 in one or more UEs 104 (e.g., UE 104A) to generate measurement data from a new process (e.g., a first trained new machine learning process) and a second process (e.g., a traditional process). Additionally, UE 104 and / or LMF computing system 102 may compare measurement data associated with the new process with measurement data associated with the second process to determine performance of the new process.

[0044] For example, the executed UE engine 208A may access the data repository 204 and obtain the UE data 206A of the UE 104A. As described herein, the UE data of the UE 104 (e.g., the UE data 206A of the UE 104A) may include data associated with a new process (e.g., a trained new machine learning process) to be monitored by the executed UE engine 208A, and data associated with another process (e.g., a traditional process) used by the executed UE engine 208A to determine the performance of the new process. The data associated with the new process and the other process (e.g., the traditional process) may include data identifying the new process and the other process, and data identifying and characterizing parameters of the new process and the other process, such as model parameters in an example where one or both processes are trained machine learning processes. Additionally, the UE data of the UE 104 (e.g., the UE data 206A of the UE 104A) may include data identifying the corresponding UE 104 (e.g., UE 104A) (e.g., a corresponding serial number or identification number), and data indicating the performance status of one or more processes. Examples of performance states of one or more processes include a performance state indicating that the performance of a particular process is below a predetermined standard, and a performance state indicating that the performance of a particular trained machine learning process is above or at a predetermined standard. Additionally, the executed UE engine 208A may generate parameter data 304 including one or more portions of the UE data 206A of the UE 104A. For example, the parameter data 304 may include data identifying the UE 104A. In addition, the executed UE engine 208A may generate parameter data 304 including data based on one or more portions of the UE data of the UE 104. For example, one or more parameters derived by the UE engine 208A from the UE data 206A.

[0045] In some cases, the executing UE engine 208A may determine one or more parameters for a particular UE 104 (e.g., UE 104A) based on UE data of a new process to be monitored by the UE engine 208A (e.g., UE data 206A of UE 104A) and UE data of another process (e.g., a legacy process) (e.g., UE data 206A of UE 104A). In this case, the UE engine 208A may utilize the second process to determine the performance of the new process. For example, the UE engine 208A may determine the timing parameter 304A based on data of model parameters of the new process and the legacy process. In some cases, based on model parameters associated with measurement periods of the new process and the legacy process, the UE engine 208A may determine the timing parameter 304A so that the UE 104 (e.g., UE 104A) configures the new process and the legacy process to generate comparable measurement data. For example, one or more elements of the measurement data of the new process and one or more elements of the measurement data of the legacy process may have matching timestamps or timestamps within a predetermined time threshold or margin.

[0046] Additionally, the UE engine 208A may generate a report request 302 and may encapsulate one or more portions of the parameter data 304 into portions of the report request 302. Furthermore, the executed UE engine 208A may send the report request 302 to the UE 104A. Figure 3 The wireless communication system 100 is shown including one LMF computing system 102 (e.g., LMF computing system 102A) communicating with one UE 104 (e.g., UE 104A), but the LMF computing system 102 can send a report request (e.g., report request 302) to any number of UEs 104. In addition, the report request can be specific to the process that the LMF computing system 102 monitors on the corresponding UE 104. The LMF computing system 102 can initiate monitoring of a specific process on the corresponding UE 104 by sending a report request to the corresponding UE 104.

[0047] As described herein, the LMF computing system 102 (e.g., LMF computing system 102A) can monitor the performance of the new process identified in the report request 302 based on the first measurement data of the new process and the second measurement data of the traditional process. The first measurement data and the second measurement data can be sent from the corresponding UE 104 to the LMF computing system 102. In addition, the corresponding UE 104 (e.g., UE 104A) can generate the first measurement data by applying the new process to one or more detected beams sent from the BS 103, and generate the second measurement data by applying the traditional process to the one or more detected beams. In addition, the UE 104 (e.g., UE 104A) can apply the new process and the traditional process to the monitored one or more beams according to the parameter data 304 of the report request. In addition, the UE 104 (e.g., UE 104A) can generate report data associated with the first measurement data and the second measurement data. In various examples, the report data can indicate the performance of the process (e.g., the first process) to be monitored based on the first measurement data and the second measurement data. Figure 4 An example of UE 104A generating report data 415 is shown. Figure 4 Only UE 104A may be shown, but any number of UEs 104 may perform the operations described herein to generate report data (e.g., report data 415). Each UE 104 that receives a report request from an LMF computing system 102 (e.g., LMF computing system 102A) may each generate report data associated with the report request (e.g., report data 415).

[0048] like Figure 4 As shown, a programming interface established and maintained by the UE 104A (e.g., an application programming interface (API) 402 of the UE 104A) can receive a report request 302 including parameter data 304. The parameter data 304 can include one or more parameters (e.g., timing parameters 304A, resource parameters 304B, and model parameters 304C). As described herein, the UE 104A can receive the report request 302 from the LMF computing system 102A (e.g., the executed UE engine 208A) across a communication network via a communication channel programmatically established between the API 402 and the executed UE engine 208A.

[0049] In various examples, one or more applications 212B (e.g., the process module 404, the analysis module 406, and the notification module 408 of the UE 104A) executed by the processor 214 of the UE 104A may perform any of the exemplary processes described herein to generate report data 415 indicating the performance of a new process (e.g., a first new trained machine learning process). The UE engine 208A executed by the LMF computing system 102 may utilize the report data 415 obtained from the UE 104A to modify or change the configuration of the UE 104A. For example, when executed by the processor 214 of the UE 104A, the executed process module 404 may perform an operation of storing the parameter data 304 in the memory 212. In such an example, the parameter data 304 may include one or more parameters (e.g., the timing parameter 304A, the resource parameter 304B, and the model parameter 304C).

[0050] Additionally, the executed process module 404 may perform an operation of accessing the memory 212 to obtain process data 411. Portions of the process data 411 may be associated with one or more processes (e.g., new processes and legacy processes) that the UE 104A (or any UE 104) may implement. Each of the one or more processes that the UE 104A may implement may be associated with measurement data that generates one or more beams detected by the antenna unit 216A and transmitted from the BS 103. In addition, the executed process module 404 may perform an operation of accessing the memory 212 to obtain parameter data 304. Based on the model parameters 304C of the report request, the process module 404 may identify the process (e.g., new process) that the LMF computing device 102A is monitoring and the additional process to determine the performance of the process that the LMF computing device 102A is monitoring. In some examples, the corresponding UE 104 (e.g., UE 104A) may utilize the measurement data of the additional process (e.g., legacy process) when determining the performance of the monitored process. In other examples, the model parameters 304C may at least identify a process to be monitored (eg, a new process), and may identify additional processes.

[0051] In addition, the executed process module 404 can obtain a portion of the process data 411 associated with the process to be monitored (e.g., a new process) and a portion of the process data 411 associated with the additional process (e.g., a traditional process) based on the identified process to be monitored and the identified additional process. In some cases, the process to be monitored (e.g., the new process) is a new trained machine learning process. In this case, the portion of the process data 411 associated with the process to be monitored (e.g., the new process) may include one or more model parameters. Additionally, the process module 404 may deploy the new process based on one or more model parameters included in the portion of the associated process data 411. In other cases, the portion of the process data 411 associated with the traditional process includes one or more parameters (e.g., model parameters in an example where the traditional process is a trained traditional machine learning process), and the process module 404 may utilize these parameters to deploy the additional process or the traditional process.

[0052] Based on the portion of the obtained process data 411 associated with the new process or the process to be monitored, the portion of the obtained process data 411 associated with the additional process or the legacy process, and the obtained parameter data 304, the executed process module 404 may apply the new process and the legacy process to the beam data 412 of one or more beams detected from the antenna unit 216A and transmitted from the BS 103. In some examples, the executed process module 404 may apply the new process to the beam data 412 based on the portion of the process data 411 associated with the new process and the parameter data 304. In these examples, the executed process module 404 may generate first measurement data 414A associated with the new process. The first measurement data 414A may include one or more measurement values ​​of the detected beams in the beam data 412. Additionally, the executed process module 404 may apply the legacy process to the beam data 412 based on the portion of the process data 411 associated with the legacy process and the parameter data 304. In addition, the executed process module 404 may generate second measurement data 414B associated with the legacy process. The second measurement data 414B may include one or more measurements of the detected beam of the beam data 412. In some cases, the process module 404 may store the first measurement data 414A and the second measurement data 414B in the memory 212.

[0053] As described herein, parameter data 304 may make measurement data (e.g., first measurement data 414A and second measurement data 414B) generated by the process to be monitored and the additional process comparable for the purpose of determining the performance of the process to be monitored. For example, parameter data 304 may include timing parameters 304A that identify time periods or measurement cycles during which the corresponding UE 104 makes measurements from the new process and the legacy process. In this case, in accordance with the above example, the executed process module 404 may apply the new process to the beam data 412 according to the portion of the process data 411 associated with the new process and within the identified time period indicated in the timing parameters 304A. Alternatively, the executed process module 404 may apply the legacy process to the beam data 412 according to the portion of the process data 411 associated with the legacy process and within the identified time period indicated by the timing parameters 304A. In this way, the first and second measurement data of the new process and the legacy process, respectively generated, may include one or more elements that may have a timestamp or a timestamp within a predetermined time threshold or margin. As described herein, each of the one or more elements may be associated with a measurement of a beam of the beam data 412 determined by a corresponding process (eg, a new process or a legacy process).

[0054] In another case, the parameter data 304 may include resource parameters 304B, which identify one or more resources, attributes, or data that a corresponding process (e.g., a new process or a legacy process) can measure from beam data 412 of one or more beams detected by the antenna unit 216A and transmitted from the BS 103. In this case, such resources, attributes, or data may be associated with assistance data, PFL ID, PRS resource ID, TRP, and PRS resource set ID. Additionally, according to the above example, the executed process module 404 may apply the new process to the beam data 412 according to the portion of the process data 411 associated with the new process and for the identified resources indicated in the resource parameters 304B. In addition, the executed process module 404 may apply the legacy process to the beam data 412 according to the portion of the process data 411 associated with the legacy process and for the identified resources indicated in the resource parameters 304B. In this way, the first measurement data 414A and the second measurement data 414B generated by the new process and the traditional process, respectively, may include one or more elements, and each of the one or more elements may be associated with the same identified resource. As described herein, the one or more elements may also be associated with measurements or measurement information of beams of the beam data 412 determined by the corresponding process (e.g., the new process or the traditional process) and information of the same identified resource. For example, based on the identified resource indicated in the resource parameter 304B, each of the one or more elements of the first measurement data 414A and the second measurement data 414B may include measurement information associated with the RSRPP or measurements indicating association with the RSRPP.

[0055] In some examples, the resource parameter 304B may be associated with a particular positioning method (e.g., UE-assisted or UE-based). In these examples, the first measurement data 414A and the second measurement data 414B may be associated with the positioning method associated with the resource parameter 304B. For example, the report request 302 may include the resource parameter 304B associated with the DL-TDOA positioning method. Additionally, the corresponding UE 104 may implement a new process to generate the first measurement data 414A, and implement a traditional process to generate the second measurement data 414B. In such an example, the first measurement data 414A and the second measurement data 414B may each be associated with the DL-TDOA positioning method. For example, the first measurement data 414A and the second measurement data 414B may include data identifying RSTD, RSRP and / or RSRPP.

[0056] Please look back Figure 4, the executed process module 404 may provide the first measurement data 414A of the beam data 412 and the second measurement data 414B of the beam data 412 as inputs to the executed analysis module 406. The executed analysis module 406 may perform an operation of comparing the first measurement data 414A and the second measurement data 414B. For example, the executed analysis module 406 may access the memory 212 and obtain the first measurement data 414A and the second measurement data 414B. Additionally, the executed analysis module 406 may parse the first measurement data 414A and obtain one or more elements of the first measurement data 414A, and parse the second measurement data 414B and obtain one or more elements of the second measurement data 414B. As described herein, each of the one or more elements of the first measurement data 414A and the second measurement data 414B may each be associated with a specific measurement made by a corresponding process (e.g., a new process and a traditional process) from a specific resource, respectively. Additionally, the measurements may have been made by a corresponding process (e.g., a new process and a legacy process), and are made to beam data 412 for one or more beams detected by antenna unit 216A and transmitted from BS 103. Furthermore, each of the one or more elements of first measurement data 414A and second measurement data 414B may include a value of an associated measurement, and each of first measurement data 414A and second measurement data 414B may be associated with a timestamp indicating a time at which the associated measurement was made by UE 104A and made using a new process or a legacy process.

[0057] Additionally, the analysis module 406 executed can compare the value of each of the one or more elements of the first measurement data 414A with the value of each of the one or more elements of the second measurement data 414B. The values ​​of the elements of the first measurement data 414A compared with the values ​​of the elements of the second measurement data 414B by the analysis module 406 executed can each be associated with a timestamp that matches or is within a predetermined time threshold or margin. In addition, the analysis module 406 executed can determine a difference between the values ​​and can determine whether the determined difference exceeds a difference threshold. The analysis module 406 executed can determine whether the first process or the new process associated with the first measurement data 414A is executed above or below a quality / standard threshold based on the analysis module 406 executing to determine whether the determined difference exceeds the difference threshold.

[0058] In some examples, the analysis module 406 executed can determine that the determined difference exceeds the difference threshold. In this example, the analysis module 406 executed can determine that the first process (e.g., new process) is executed below the quality / standard threshold. In addition, the analysis module 406 executed can generate report data 415, and the report data 415 indicates that the monitored process or the first process (e.g., new process) is executed below the quality / standard threshold. In addition, the analysis module 406 executed can store the report data 415 in the memory 212. In some cases, when executed by the processor 214 of the UE 104A, the notification module 408 executed can generate a notification message 416. In addition, the notification module executed can include one or more parts of the report data 415 in the part of the notification message 416, and the report data 415 indicates that the monitored process or the first process (e.g., new process) is executed below the quality / standard threshold. In this case, the notification module 408 executed can send the notification message 416 to the LMF computing system 102A.

[0059] In other examples, the analysis module 406 executed can determine that the determined difference is lower than or at a difference threshold. In these examples, the analysis module 406 executed can determine that the first process (e.g., a new process) is executed above or at a quality / standard threshold. In addition, the analysis module 406 executed can generate report data 415, and the report data 415 indicates that the monitored process or the first process (e.g., a new process) is executed above or at a quality / standard threshold. In addition, the analysis module 406 executed can store the report data 415 in the memory 212. In some cases, when executed by the processor 214 of the UE104A, the notification module 408 executed can generate a notification message 416. In addition, the notification module executed can include one or more parts of the report data 415 in the part of the notification message 416, and the report data 415 indicates that the monitored process or the first process (e.g., a new process) is executed above or at a quality / standard threshold. In this case, the notification module 408 executed can send the notification message 416 to the LMF computing system 102A.

[0060] In various examples, when the analysis module 406 executed determines that the determined difference exceeds the difference threshold, the analysis module 406 executed may generate the report data 415. In these examples, when the analysis module 406 executed determines that the determined difference does not exceed the difference threshold, the analysis module 406 executed may not generate the report data 415. Alternatively, the notification module 408 executed may generate a notification message 416 for the report data 415, the report data 415 indicating that the monitored process or the first process (e.g., the new process) is executed below the quality / standard threshold. Additionally, the notification module 408 executed may not generate a notification message 416 for the report data, the report data indicating that the monitored process or the first process (e.g., the new process) is executed above or at the quality / standard threshold.

[0061] The LMF computing system 102 (e.g., the LMF computing system 102A) may receive a notification message including report data from one or more UAs 104, the report data indicating that the monitored process is performed below the quality / standard threshold (e.g., the notification message 416 including report data 415 indicating that the monitored process or the first process (e.g., the new process) is performed below the quality / standard threshold). Based on the report data, the LMF computing system 102 may modify or change the configuration of the corresponding UE 104. Figure 5 As shown, the UE 104A (e.g., the executed notification module 408) can send a notification message 416 across the communication network via a communication channel established between the executed notification module 408 and the API 502. As described herein, the notification message 416 can include report data 415, which indicates that the monitored process or the first process (e.g., the new process) is executed below the quality / standard threshold. The API 502 of the server 202 can receive the notification message 416 and can route the notification message 416 to the executed UE engine 208A. The executed UE engine 208A can implement the operation of parsing the notification message 416 and obtaining one or more parts of the report data 415. In addition, the executed UE engine 208A can store one or more parts of the report data 415 to a corresponding part of the data repository 204 (e.g., the UE data storage 206).

[0062] Additionally, the executed UE engine 208A may perform operations to modify or change the configuration of the corresponding UE 104 (e.g., UE 104A) based on one or more portions of the report data. Figure 5, the executed UE engine 208A may access the data repository 204 and obtain one or more portions of the report data 415. Based on the one or more portions of the report data 415, the UE engine 208A may generate instructions 510 associated with the UE 104A (or any corresponding UE 104 of the one or more portions of the obtained report data). Additionally, the executed UE engine 208A may store the instructions 510 in a corresponding portion of the data repository 204 (e.g., the UE data store 206). In some cases, the instructions 510 may cause the UE 104A to modify or change the configuration of the UE 104A (or any corresponding UE 104 of the report data). For example, one or more portions of the report data 415 may indicate that the performance of the first process (e.g., the new process) is below the quality / standard threshold (e.g., the difference between the first measurement data 414A and the second measurement data 414B of the new process exceeds the difference threshold). In such an example, the UE engine 208A may generate instructions 510 that may cause the UE 104A to switch to a default / safe mode or operate in a default / safe mode. In some cases, when the UE 104A operates in the default mode, the security mode, or the third mode, the UE 104A may be configured to implement or deploy an additional process (e.g., a traditional process), or another process (e.g., a third trained traditional machine learning process) designated as robust and reliable by the operator of the wireless communication system 100. In this case, the UE 104A may apply the additional process or other processes to generate measurement data, which the LMF computing system 102 may utilize to provide location services for the UE 104A.

[0063] As described herein, when UE 104A (or any other UE 104) is generating measurement data (e.g., measurement data 414) and report data (e.g., report data 415), UE 104A can operate in the reporting mode. Additionally, UE 104A (or any other UE 104) can be configured to operate in the reporting mode upon receiving a report request (e.g., report request 302) from LMF computing system 102 (e.g., LMF computing system 102A). In addition, when UE 104A receives or processes instruction 510, UE 104A can switch from the reporting mode to the default / security mode.

[0064] Furthermore, when executed by one or more processors of the server 202 of the LMF computing system 102A, the executed notification engine 520 may access the data repository and obtain the instructions 510. Furthermore, the executed notification engine 520 may generate a configuration message 522, and may encapsulate one or more portions of the instructions 510 into portions of the configuration message 522. Furthermore, the executed notification engine 520 may send the configuration message 522 to the UE 104A via a communication channel established between the UE 104A and the executed notification engine 520 over the communication network.

[0065] In some examples, the LMF computing system 102 (e.g., LMF computing system 102A) can process the measurement data of the process to be monitored and the measurement data of another process (e.g., a traditional process) to determine the performance of the process to be monitored. In some cases, the report data 415 may include the measurement data of the process to be monitored (e.g., the first measurement data 414A), and such a process may be a new machine learning process that has been trained. Additionally, the report data 415 may include the measurement data of the traditional process (e.g., the second measurement data 414B). Additionally, the measurement data of the new trained machine learning process may include a location estimate, and the measurement data of the traditional process may include raw data or measurements that the LMF computing system 102 (e.g., the executed UE engine 208A) can use to generate the corresponding location estimate. In this case, the LMF computing system 102 can compare the location estimate of the new and trained machine learning process with the location estimate determined from the traditional process to determine the accuracy and performance of the new and trained machine learning process. Based on such a determination, the LMF computing system 102 can generate instructions (e.g., instructions 510) for the corresponding UE 104 (e.g., UE 104A). As described herein, the instructions can cause the corresponding UE 104 to operate in different modes and implement processes associated with the different modes to generate measurement data. The LMF computing system 102 can then use the measurement data to provide location services to the corresponding UE.

[0066] For example, the LMF computing system 102 can determine the difference between the value associated with the location estimate of the new and trained machine learning process and the value associated with the location estimate determined from the traditional process. Additionally, the LMF computing system 102 can also determine whether the determined difference exceeds a difference threshold. In the example where the determined difference exceeds the difference threshold, the LMF computing system 102 can generate instructions that cause the corresponding UE 104 to operate in the default / secure mode described herein.

[0067] In other examples, the LMF computing system 102 (e.g., the LMF computing system 102A) can update the status of a process being implemented on the corresponding UE 104 for a particular UE 104. In these examples, the process can be a process being monitored by the corresponding LMF computing system 102. Additionally, the process can be a new process (e.g., a new and trained machine learning process). For example, referring to Figure 5 , the LMF computing system 102A may receive a notification message 416 including report data 415 from the UE 104A, the report data 415 indicating that the monitored new process is executed above or at a quality / standard threshold. In such an example, the UE engine 208A may access the UE data 206A of the UE 104A and update data related to the status of the new process based on the report data 415 (e.g., the current or when the new process determined by the UE 104A is executed above or at a quality / standard threshold, the new process is executed above or at a quality / standard threshold). In another example, the LMF computing system 102A may receive a notification message 416 including report data 415 from the UE 104A, the report data 415 indicating that the monitored new process is executed below a quality / standard threshold. In such an example, the UE engine 208A can access the UE data 206A of the UE 104A and update data related to the status of the new process based on the reporting data 415 (e.g., currently or when the UE 104A determines that the new process is to be performed below the quality / standard threshold, the new process is to be performed below the quality / standard threshold).

[0068] Reference Figure 6 , the API 602 of the UE 104A may receive a request from the UE 104A (e.g., Figure 5 The API 602 may route the configuration message 522 to the executed process module 404. Additionally, the executed process module 404 may parse the configuration message 522 and obtain one or more portions of the instructions 510. Furthermore, the executed process module 404 may store one or more portions of the instructions 510 in the memory 212.

[0069] In some examples, the executed process module 404 can configure the UE 104A according to the instructions 510. In such an example, the executed process module 404 can implement the operation of accessing the memory 212 to obtain the instructions 510. As described herein, the instructions 510 may include programming instructions for the UE 104A to switch to a default / safe mode or to operate in a default / safe mode. Additionally, the instructions 510 may identify a specific process associated with the default / safe mode. In addition, when the UE 104A operates in the default mode, the UE 104A may generate measurement data using a process associated with the default / safe mode. In some cases, the specific process identified in the instructions 510 may be a process for determining the performance of a traditional process (e.g., a traditional process), or another process designated as robust and reliable in the operation of the wireless communication system 100 (e.g., a third trained traditional machine learning process).

[0070] In various cases, in response to UE 104A determining that the performance of the process to be monitored or the new process is below the quality or standard threshold, UE 104A can automatically fall back and operate in default / safe mode. As described herein, when UE 104A (or any UE 104) determines whether the performance of the process to be monitored or the new process is below the quality or standard threshold, UE 104A can operate in another mode (e.g., second or reporting mode). Therefore, UE 104A (or any UE 104) can automatically switch to default / safe mode when determining that the performance of the process to be monitored or the new process is below the quality or standard threshold without receiving an instruction 510 from LMF computing system 102A (or any LMF computing system 102).

[0071] In addition, when the UE 104A is operating in the default / secure mode, and based on the instructions 510, the executed process module 404 can implement the process identified in the instructions 510 to generate the measurement data 604. For example, based on the instructions 510, the executed process module 404 can identify a specific process (e.g., a traditional process or other process). In addition, the executed process module 404 can access the memory 212 to obtain a portion of the process data 411 associated with the identified specific process. As described herein, the portion of the process data 411 associated with the identified specific process may include one or more parameters so that the executed process module 404 can configure and deploy the identified specific process. For example, the specific process can be a trained traditional machine learning process, and the portion of the obtained process data 411 can include one or more model parameters of the trained traditional machine learning process. Therefore, the executed process module 404 can configure and deploy the trained traditional machine learning process according to the one or more model parameters. In addition, the executed process module 404 can apply the identified specific process to the beam data 312 of one or more beams detected by the antenna unit 216A and transmitted by the BS 103. Additionally, the executed process module 404 may generate the measurement data 604 based on applying the identified particular process to the beam data 312. In some cases, the executed process module 404 may store the measurement data 604 in the memory 212.

[0072] As described herein, the LMF computing system 102A (or any LMF computing system 102) can utilize measurement data (e.g., measurement data 604) to provide location services for the UE 104A. Figure 6 , the executed notification module 408 can access the memory 212 to obtain the measurement data 604. Additionally, the executed notification module 408 can generate a measurement message 606 and encapsulate one or more portions of the measurement data 604 into portions of the measurement message 606. In addition, the executed notification module 408 can send the measurement message 606 to the LMF computing system 102A. The LMF computing system 102A (e.g., the executed UE engine 208A) can parse the measurement message 606; obtain one or more portions of the measurement data 604 from the parsed measurement message 606; and implement operations for providing location services to the UE 104A based on the obtained one or more portions of the measurement data 604 ( Figure 6 not shown).

[0073] In some examples, UE 104A may be configured to operate in one or more modes (e.g., a first or normal mode, a second or reporting mode, and a third or default / security mode). Each mode may be associated with the deployment of one or more processes (e.g., new processes and legacy processes). In some cases, prior to receiving report request 302, UE 104A (or any UE 104) may be configured to operate in a first or normal mode. When UE 104A operates in the first or normal mode, UE 104A may apply a first process (e.g., a new process) to one or more detected beams transmitted from BS 103 according to one or more parameters of the first process included in a corresponding portion of the obtained process data 411. Additionally, UE 104A may determine one or more measurements and generate measurement data (e.g., first measurement data 414A), including one or more measurements based on applying the first process to one or more detected beams (e.g., beam data 412). In addition, UE 104A may transmit the measurement data (e.g., first measurement data 414A) to LMF computing system 102A. The LMF computing system 102A may provide location services to the UE 104A based on the measurement data.

[0074] In other cases, in response to receiving the report request 302, the UE 104A (or any UE 104) can be configured to operate in a second or reporting mode. When the UE 104A operates in the second or reporting mode, the UE 104A can apply the first process (e.g., the new process) and the second process (e.g., the legacy process) to one or more detected beams transmitted from the BS 103 according to one or more parameters of the first and second processes included in the corresponding parts of the obtained process data 411 and the parameter data (e.g., the parameter data 304 included in the report request 302). As described herein, the UE 104A can determine one or more measurements and generate measurement data (e.g., the first measurement data 414A), including one or more measurements based on applying the first process to the one or more detected beams (e.g., the beam data 412). Additionally, the UE 104A can determine one or more measurements and generate measurement data (e.g., the second measurement data 414B), including one or more measurements based on applying the second process to the one or more detected beams (e.g., the beam data 412). In addition, the UE 104A may transmit an indication of the performance of the first process to the LMF computing system 102A based on the measurement data of the first and second processes. The LMF computing system 102A may modify or change the configuration of the UE 104A based on the indication. As described herein, if the UE 104A operates in another mode (e.g., the first mode or the normal mode), the UE 104A may switch to the reporting mode upon receiving the report request 302. In some cases, when the UE 104A operates in the reporting mode, the UE 104A may simultaneously or concurrently deploy the first process (e.g., the new process (e.g., the new and trained machine learning process)) and the second process (e.g., the traditional process).

[0075] Figure 7 700 is a flow chart of an exemplary process 700 for determining the performance of a process (e.g., a new process) deployed by a user equipment (UE) 104. For example, one or more LMF computing systems 102 (e.g., LMF computing system 102A) may perform one or more steps of the exemplary process 700, as described below with reference to Figure 7 As described. Figure 7 , the LMF computing system 102A may perform any of the processes described herein to send the report request 302 to the UE 104A (e.g., at Figure 7702 of ). As described herein, the report request 302 may include parameter data 304. In some cases, the parameter data 304 may include one or more monitoring parameters (e.g., timing parameters 304A, resource parameters 304B, and modeling or processing parameters 304C). Additionally, the executing UE engine 208A may utilize the UE data 206A of the UE 104A to determine the one or more monitoring parameters included in the parameter data 304. Additionally, as described herein, the UE data 206A of the UE 104A may include data associated with a new process (e.g., a trained new machine learning process) to be monitored by the executing UE engine 208A, and data associated with another process (e.g., a traditional process) used by the executing UE engine 208A to determine the performance of the new process. The data associated with the new process and the other process (e.g., the traditional process) may include data identifying the new process and the other process, as well as data identifying and characterizing parameters of each of the new process and the other process (e.g., model parameters in examples where one or both processes are trained machine learning processes). Additionally, the UE data 206A of the UE 104A may include data identifying the UE 104A (e.g., a corresponding serial number or identification number), and data indicating a performance status of one or more processes. Examples of performance statuses of one or more processes include a performance status indicating that the performance of a particular process is below a predetermined standard, and a performance status indicating that the performance of a particular trained machine learning process is below a predetermined standard.

[0076] For example, the executed UE engine 208A may access the data repository 204 and obtain the UE data 206A of the UE 104A. Additionally, the executed UE engine 208A may generate parameter data 304 including one or more portions of the UE data 206A of the UE 104A. For example, the parameter data 304 may include data identifying the UE 104A. Furthermore, the executed UE engine 208A may generate parameter data 304 including data based on one or more portions of the UE data of the UE 104. For example, one or more parameters (e.g., timing parameters 304A) derived from the UE data 206A by the UE engine 208A. Furthermore, the executed UE engine 208A may generate a report request 302 and encapsulate one or more portions of the parameter data 304 into portions of the report request 302. Furthermore, the executed UE engine 208A may send the report request 302 to the API 402 of the UE 104A.

[0077] In response to receiving the report request 302, the UE 104A may operate in a reporting mode, and the executed process module 404 may implement the operations described herein to generate measurement data (e.g., first measurement data 414A and second measurement data 414B) according to the report request 302. Additionally, when the UE 104A operates in the reporting mode, the executed process module 404 may determine the performance of the first process (e.g., a new and trained machine learning process) identified in the report request 302 based on the measurement data. Based on the determined performance, the executed process module 404 may generate report data 415 that may indicate the performance of the first process. In some cases, the report data 415 may indicate a comparison between the measurement data generated by the first process (e.g., the first measurement data 414A) and the measurement data of the second process used to determine the performance of the first process. In this case, the comparison may indicate the performance of the first process. In addition, the executed process module 404 may send the report data 415 to the LMF computing system 102A (e.g., API 502).

[0078] Reference Figure 7 , the LMF computing system 102A may perform any of the processes described herein to receive report data 415 from the UE 104A (e.g., in Figure 7 In step 704 of ). Additionally, based on the report data 415, the LMF computing system 102A may generate instructions for causing the user device to implement at least one of the first process and the second process (e.g., in Figure 7 506 of the execution). For example, the UE 104A (e.g., the executed notification module 408) can send the notification message 416 across the communication network via a communication channel established between the executed notification module 408 and the API 502. As described herein, the notification message 416 can include report data 415, which indicates the performance of the monitored process or the first process (e.g., the new process). The API 502 of the server 202 can receive the notification message 416 and can route the notification message 416 to the executed UE engine 208A. The executed UE engine 208A can implement operations of parsing the notification message 416 and obtaining one or more portions of the report data 415. Additionally, the executed UE engine 208A can generate instructions 510 based on one or more portions of the report data 415.

[0079] In some cases, the reporting data 415 may indicate that the first process is performed below a quality / standard threshold. Based on the reporting data 415, the executed UE engine 208A may generate instructions 510 associated with the UE 104A. The instructions 510 may cause the UE 104A to modify or change the configuration of the UE 104A (or any corresponding UE 104 of the reporting data). For example, the instructions 510 may cause the UE 104A to switch from a reporting mode to a default / security mode. In this case, when the UE 104A operates in the default mode, the security mode, or the third mode, the UE 104A may be configured to implement or deploy an additional process (e.g., a traditional process), or another process designated by the operator of the wireless communication system 100 as robust and reliable (e.g., a third trained traditional machine learning process).

[0080] In other cases, the reporting data 415 may indicate that the first process is performed above or at a quality / standard threshold. Based on the reporting data 415, the executing UE engine 208A may generate instructions 510 associated with the UE 104A. The instructions 510 may cause the UE 104A to modify or change the configuration of the UE 104A (or any corresponding UE 104 of the reporting data). For example, the instructions 510 may cause the UE 104A to switch from the reporting mode to the first mode or normal mode. In this case, when the UE 104A operates in the normal mode or the first mode, the UE 104A may be configured to implement or deploy the first process (e.g., a new and trained traditional machine learning process).

[0081] Additionally, the LMF computing system 102A may send instructions 510 (e.g., Figure 7 As described herein, when executed by one or more processors of the server 202 of the LMF computing system 102A, the executed notification engine 520 can generate a configuration message 522 and can encapsulate one or more portions of the instructions 510 into portions of the configuration message 522. In addition, the executed notification engine 520 can send the configuration message 522 to the UE 104A via a communication channel established between the UE 104A and the executed notification engine 520 through a communication network. Additionally, the API 602 of the UE 104A can receive from the UE 104A (e.g., Figure 5 The notification engine 520 of the execution of the API 602 receives the configuration message 522. The API 602 can route the configuration message 522 to the execution process module 404. Additionally, the execution process module 404 can parse the configuration message 522 and obtain one or more parts of the instruction 510.

[0082] In the event that the instruction 510 is associated with the reporting data 415 indicating that the first process is performed below the quality / standard threshold, the executed process module 404 can cause the UE 104A to switch from operating in the reporting mode to the default / secure mode. As described herein, the default / secure mode can be associated with a specific process that the UE 104A can use to measure data, such as a traditional process or another process designated as robust and reliable by the operator of the wireless communication system 100 (e.g., a third trained traditional machine learning process). Additionally, when the UE 104A is operating in the default / secure mode, the UE 104A can implement operations as described herein to generate measurement data (e.g., measurement data 604) that the LMF computing system 102A can use to provide location services for the UE 104A.

[0083] For example, the executed process module 404 may identify a specific process associated with the default / safe mode based on one or more portions of the instructions 510. Additionally, the executed process module 404 may access the memory 212 to obtain a portion of the process data 411 associated with the identified specific process. As described herein, the portion of the process data 411 associated with the identified process may include one or more parameters that enable the executed process module 404 to configure and deploy the identified specific process. For example, the specific process may be a trained traditional machine learning process, and the portion of the obtained process data 411 may include one or more model parameters of the trained traditional machine learning process. Therefore, the executed process module 404 may configure and deploy the trained traditional machine learning process based on the one or more model parameters. In addition, the executed process module 404 may apply the identified specific process to the beam data 312 of one or more beams detected by the antenna unit 216A and transmitted by the BS 103. In addition, the executed process module 404 may generate measurement data 604 based on applying the identified specific process to the beam data 312.

[0084] As described herein, the LMF computing system 102A may obtain measurement data (e.g., measurement data 604) and utilize the measurement data to provide location services for the UE 104A. For example, the executed notification module 408 may access the memory 212 to obtain the measurement data 604. Additionally, the executed notification module 408 may generate a measurement message 606 and encapsulate one or more portions of the measurement data 604 into portions of the measurement message 606. Furthermore, the executed notification module 408 may send the measurement message 606 to the LMF computing system 102A. The LMF computing system 102A (e.g., the executed UE engine 208A) may parse the measurement message 606; obtain one or more portions of the measurement data 604 from the parsed measurement message 606; and implement operations for providing location services to the UE 104A based on the obtained one or more portions of the measurement data 604 ( Figure 6 not shown).

[0085] In the case where the instruction 510 is associated with the reporting data 415 indicating that the first process is performed above the quality / standard threshold, the executed process module 404 can cause the UE 104A to switch from operating in the reporting mode to the first mode or normal mode. As described herein, the first / normal mode can be associated with a specific process that the UE 104A can utilize measurement data (e.g., the first process). In some cases, the first process can be a new trained machine learning process. Additionally, when the UE 104A is operating in the first / normal mode, the UE 104A can implement operations as described herein to generate measurement data, for example, the first measurement data 414A that the LMF computing system 102A can use to provide location services for the UE 104A.

[0086] For example, the executed process module 404 may determine a first process associated with the first / normal mode based on one or more portions of the instructions 510. Additionally, the executed process module 404 may access the memory 212 to obtain a portion of the process data 411 associated with the first process. As described herein, the portion of the process data 411 associated with the first process may include one or more parameters that enable the executed process module 404 to configure and deploy the first process. For example, the first process may be a new trained traditional machine learning process, and the portion of the obtained process data 411 may include one or more model parameters of the new and trained traditional machine learning process. Thus, the executed process module 404 may configure and deploy the new and trained machine learning process based on the one or more model parameters. In addition, the executed process module 404 may apply the identified first process to the beam data 312 of one or more beams detected by the antenna unit 216A and transmitted by the BS 103. In addition, the executed process module 404 may generate the first measurement data 414A based on applying the first process to the beam data 312. In some cases, the executed process module 404 may store in the memory 212 the measurement data 414A generated when the UE 104A operates in the first mode.

[0087] As described herein, the LMF computing system 102A can obtain measurement data (e.g., first measurement data 414A) and use the measurement data to provide location services for the UE 104A. For example, the executed notification module 408 can access the memory 212 to obtain the measurement data of the first process generated when the UE 104A operates in the first mode (e.g., measurement data 414A). Additionally, the executed notification module 408 can generate a measurement message (e.g., measurement message 606) and encapsulate one or more portions of the obtained measurement data into portions of the measurement message. In addition, the executed notification module 408 can send the measurement message to the LMF computing system 102A. The LMF computing system 102A (e.g., the executed UE engine 208A) can parse the measurement message; obtain one or more portions of the measurement data from the parsed measurement message 606; and implement operations for providing location services to the UE 104A based on the one or more portions of the obtained measurement data.

[0088] Figure 8 800 for generating report data according to some exemplary embodiments. For example, one or more UEs 104 (e.g., UE 104A) may perform one or more steps of the exemplary process 800, as described below with reference to Figure 8 As described. Figure 8, a first UE 104A in the plurality of UEs 104 (or any UE 104) may perform any of the processes described herein to obtain a report request 302 (e.g., in Figure 8 As described herein, report request 302 may include parameter data 304. In some cases, parameter data 304 may include one or more monitoring parameters (eg, timing parameters 304A, resource parameters 304B, and modeling or processing parameters 304C).

[0089] Based on the report request 302, the UE 104A may utilize the first trained machine learning process identified in the report request 302 to generate one or more elements of the first measurement data 414A (e.g., Figure 8 Step 804 of the report request 302). As described herein, the first trained machine learning process can be a new process identified in the report request 302 (e.g., the modeling or processing parameters 304C from the report request 302). Additionally, the executed process module 404 can identify the first trained machine learning process from the report request 302 and access the memory 212 to obtain a portion of the process data 411 associated with the first trained machine learning process. In some cases, the portion of the process data 411 associated with the first trained machine learning process may include one or more modeling parameters. Based on the obtained portion of the process data 411 associated with the first trained machine learning process and the parameter data 304 of the report request 302, the executed process module 404 can apply the first trained machine learning process to beam data 412 of one or more beams detected by the slave antenna unit 216A and transmitted from the BS 103. In addition, the executed process module 404 can generate first measurement data 414A based on applying the first trained machine learning process to the beam data 412.

[0090] Additionally, based on the report request 302, while the UE 104A is generating one or more elements of the first measurement data 414A, the UE 104A may utilize a second process to generate one or more elements of the second measurement data 414B (eg, Figure 8Step 806 of ). As described herein, the second process can be a traditional process and / or a trained machine learning process. Additionally, the second process can be utilized to determine the performance of the first trained machine learning process. In some cases, the report request 302 can identify the second process. In this case, the executed process module 404 can identify the second process from the report request 302 and access the memory 212 to obtain the portion of the process data 411 associated with the second process. In some cases, the portion of the process data 411 associated with the second process can include one or more parameters, and the executed process module 404 can use these parameters to deploy the second process. Based on the portion of the obtained process data 411 associated with the second process and the parameter data 304 of the report request 302, the executed process module 404 can apply the second process to the beam data 412. In addition, the executed process module 404 can generate second measurement data 414B based on applying the second process to the beam data 412.

[0091] In addition, UE 104A may compare first measurement data 414A and second measurement data 414B (eg, Figure 8Step 808 of ). As described herein, the UE 104A can determine the performance of the first trained machine learning process based on the comparison. For example, the analysis module 406 executed can perform an operation of comparing the first measurement data 414A and the second measurement data 414B. For example, the analysis module 406 executed can parse the first measurement data 414A and obtain one or more elements of the first measurement data 414A, and parse the second measurement data 414B and obtain one or more elements of the second measurement data 414B. Each of the one or more elements of the first measurement data 414A and the second measurement data 414B can include the value of the associated measurement, and each of the first measurement data 414A and the second measurement data 414B can be associated with a timestamp indicating the time when the associated measurement was performed by the UE 104A and the first trained machine learning process and the second process, respectively. Additionally, the analysis module 406 executed can compare the values ​​of the elements of the first measurement data 414A and the values ​​of the elements of the second measurement data 414B, each of which is associated with a timestamp that matches or is within a predetermined time threshold or margin. In addition, the analysis module 406 executed can determine the difference between the values, and can determine whether the determined difference exceeds the difference threshold. In addition, based on the analysis module 406 executed determining whether the determined difference exceeds the difference threshold, the analysis module 406 executed can determine that the first trained machine learning process associated with the first measurement data 414A is executed above or below the quality / standard threshold. For example, the analysis module 406 executed determines that the determined difference is at or below the difference threshold. In this case, the analysis module 406 executed can determine that the first trained machine learning process is executed above or at the quality / standard threshold. In another case, the analysis module 406 executed determines that the determined difference exceeds the difference threshold. In this case, the analysis module 406 executed can determine that the first trained machine learning process is executed below the quality / standard threshold.

[0092] Examples of implementations are further described in the following numbered clauses:

[0093] 1. A device comprising:

[0094] a non-transitory machine-readable storage medium storing instructions; and

[0095] At least one processor coupled to a non-transitory machine-readable storage medium, the at least one processor configured to execute instructions to:

[0096] Sending a report request for positioning data to a user equipment, wherein the user equipment receiving the report request causes the user equipment

[0097] The device performs operations in a first mode, the operations comprising:

[0098] Implementing a first process to generate first positioning data, and implementing a second process to generate second positioning data; and

[0099] generating report data indicating a comparison between the first positioning data and the second positioning data;

[0100] receiving reporting data from a user device; and

[0101] Based on the report data, an instruction is generated and sent to the user equipment, wherein the instruction causes the user equipment to implement at least one of the first process or the second process.

[0102] 2. An apparatus as claimed in clause 1, wherein the reporting request comprises a timing parameter identifying a time interval for the user equipment to operate in the first mode.

[0103] 3. An apparatus according to any of clauses 1-2, wherein the report request comprises a resource parameter identifying one or more resources from which the user equipment is to generate the positioning data.

[0104] 4. An apparatus according to clause 3, wherein the one or more resources include resources selected from the group consisting of: resources associated with an auxiliary data ID, resources associated with a positioning frequency layer (PFL) ID, resources associated with a transmit receive (TRP) ID, resources associated with a positioning reference signal (PRS) set ID, a set of positioning reference signal (PRS) resources, or a combination thereof.

[0105] 5. Apparatus according to any of clauses 1-4, wherein the first positioning data is associated with a first subset of resources and the second positioning data is associated with a second subset of resources.

[0106] 6. An apparatus as described in clause 5, wherein the second subset of resources includes the first subset of resources.

[0107] 7. An apparatus according to any of clauses 1-6, wherein the first positioning data and the second positioning data each comprise one or more elements associated with the same resource.

[0108] 8. An apparatus as described in any of clauses 1 to 7, wherein the report request causes the user equipment to:

[0109] generating a first timestamp associated with a first element of the one or more elements of the first positioning data and a second timestamp associated with a second element of the one or more elements of the second positioning data, wherein the first element corresponds to the second element;

[0110] Comparing the first timestamp and the second timestamp;

[0111] determining that the first timestamp and the second timestamp are within a predetermined time interval; and

[0112] Report data is generated based on the determination.

[0113] 9. An apparatus as claimed in clause 8, wherein the report request further causes the user equipment to:

[0114] It is determined that the first timestamp and the second timestamp match.

[0115] 10. An apparatus as claimed in any of clauses 1 to 9, wherein the report request further causes the user equipment to:

[0116] One or more elements of first positioning data are generated concurrently using a first process, and one or more elements of second positioning data are generated using a second process.

[0117] 11. An apparatus as claimed in clause 10, wherein the report request further causes the user equipment to:

[0118] comparing the first positioning data and the second positioning data;

[0119] Based on the comparison, determining that a difference between a first element of the first positioning data and a corresponding first element of the second positioning data satisfies a difference threshold; and

[0120] Based on the determined difference satisfying the difference threshold, report data is sent to the device.

[0121] 12. An apparatus according to clause 11, wherein, prior to receiving the report request from the apparatus, the user equipment operates in the second mode and implements the first process to generate the first positioning data, and wherein the receipt of the report request by the user equipment causes the user equipment to:

[0122] switching from operating in the second mode to the first mode;

[0123] determining that at least a first element of the first positioning data and a corresponding first element of the second positioning data are within a predetermined time margin;

[0124] comparing at least a first element of the first positioning data with a corresponding first element of the second positioning data; and

[0125] In the event that the comparison is greater than a predetermined threshold, a switch is made to operate in a third mode to generate additional positioning data.

[0126] 13. Apparatus according to clause 12, wherein, when the user equipment operates in the third mode, the user equipment implements a second process to generate additional positioning data.

[0127] 14. Apparatus according to clause 12, wherein, when the user equipment operates in the third mode, the user equipment implements a third process to generate additional positioning data.

[0128] 15. An apparatus according to any of clauses 1-14, wherein the first process is a trained machine learning process.

[0129] 16. An apparatus according to any of clauses 1-15, wherein the second process is a trained machine learning process.

[0130] 17. An apparatus according to any of clauses 1-16, wherein the report request comprises model parameters, wherein the model parameters identify one or more processes to be monitored, the one or more processes comprising a first process and a second process.

[0131] 18. A non-transitory machine-readable storage medium storing instructions that, when executed by at least one processor of a location server, cause the at least one processor to perform operations comprising:

[0132] Sending a report request for positioning data to a user equipment, wherein receipt of the report request by the user equipment causes the user equipment to perform operations in a first mode, the operations comprising:

[0133] Implementing a first process to generate first positioning data, and implementing a second process to generate second positioning data; and

[0134] generating report data indicating a comparison between the first positioning data and the second positioning data; and

[0135] receiving reporting data from a user device; and

[0136] Based on the report data, an instruction is generated and sent to the user equipment, wherein the instruction causes the user equipment to implement at least one of the first process and / or the second process.

[0137] 19. The non-transitory machine-readable storage medium of clause 18, wherein the reporting request comprises a timing parameter identifying a time interval for the user equipment to operate in the first mode.

[0138] 20. The non-transitory machine-readable storage medium of any of clauses 18-19, wherein the report request comprises a resource parameter identifying one or more resources from which the user equipment generates the positioning data.

[0139] 21. A non-transitory machine-readable storage medium as described in clause 20, wherein the one or more resources include resources selected from the group consisting of: resources associated with an auxiliary data ID, resources associated with a positioning frequency layer (PFL) ID, resources associated with a transmit receive (TRP) ID, resources associated with a positioning reference signal (PRS) set ID, a set of positioning reference signal (PRS) resources, or a combination thereof.

[0140] 22. The non-transitory machine-readable storage medium of any of clauses 18-22, wherein the first positioning data is associated with a first subset of resources and the second positioning data is associated with a second subset of resources.

[0141] 23. The non-transitory machine-readable storage medium of clause 22, wherein the second subset of resources includes the first subset of resources.

[0142] 24. The non-transitory machine-readable storage medium of any of clauses 18-23, wherein the first positioning data and the second positioning data each comprise one or more elements associated with the same resource.

[0143] 25. A non-transitory machine-readable storage medium as claimed in any of clauses 18 to 24, wherein the report request causes the user equipment to:

[0144] generating a first timestamp associated with a first element of the one or more elements of the first positioning data and a second timestamp associated with a second element of the one or more elements of the second positioning data, wherein the first element corresponds to the second element;

[0145] Comparing the first timestamp and the second timestamp;

[0146] determining that the first timestamp and the second timestamp are within a predetermined time interval; and

[0147] Report data is generated based on the determination.

[0148] 26. The non-transitory machine-readable storage medium of clause 25, wherein the report request further causes the user device to:

[0149] It is determined that the first timestamp and the second timestamp match.

[0150] 27. A non-transitory machine-readable storage medium as described in any of clauses 18-26, wherein the report request further causes the user device to:

[0151] One or more elements of first positioning data are generated concurrently using a first process, and one or more elements of second positioning data are generated using a second process.

[0152] 28. The non-transitory machine-readable storage medium of clause 27, wherein the report request further causes the user device to:

[0153] comparing the first positioning data and the second positioning data;

[0154] Based on the comparison, determining that a difference between a first element of the first positioning data and a corresponding first element of the second positioning data satisfies a difference threshold; and

[0155] Based on the determined difference satisfying the difference threshold, report data is sent to the device.

[0156] 29. A non-transitory machine-readable storage medium as described in clause 28, wherein, prior to receiving the report request from the apparatus, the user equipment operates in the second mode and implements the first process to generate the first positioning data, and wherein the receipt of the report request by the user equipment causes the user equipment to:

[0157] switching from operating in the second mode to the first mode;

[0158] determining that at least a first element of the first positioning data and a corresponding first element of the second positioning data are within a predetermined time margin;

[0159] comparing at least a first element of the first positioning data with a corresponding first element of the second positioning data; and

[0160] In the event that the comparison is greater than a predetermined threshold, a switch is made to operate in a third mode to generate additional positioning data.

[0161] 30. The non-transitory machine-readable storage medium of clause 29, wherein when the user equipment operates in the third mode, the user equipment implements a second process to generate additional positioning data.

[0162] 31. The non-transitory machine-readable storage medium of clause 29, wherein when the user equipment operates in the third mode, the user equipment implements a third process to generate additional positioning data.

[0163] 32. A non-transitory machine-readable storage medium as described in any of clauses 18-31, wherein the first process is a trained machine learning process.

[0164] 33. A non-transitory machine-readable storage medium as described in any of clauses 18-32, wherein the second process is a trained machine learning process.

[0165] 34. The non-transitory machine-readable storage medium of any of clauses 18-33, wherein the report request comprises model parameters, wherein the model parameters identify one or more processes to be monitored, the one or more processes comprising a first process and a second process.

[0166] 35. A computer-implemented method comprising:

[0167] A processor of the location server sends a report request for positioning data to a user equipment, wherein receipt of the report request by the user equipment causes the user equipment to perform operations in a first mode, the operations comprising:

[0168] Implementing a first process to generate first positioning data, and implementing a second process to generate second positioning data; and

[0169] generating report data indicating a comparison between the first positioning data and the second positioning data; and

[0170] receiving, by the processor, reporting data from a user device; and

[0171] Based on the report data, instructions are generated and sent to the user equipment, wherein the instructions cause the user equipment to implement at least one of the first process and the second process.

[0172] 36. A computer-implemented method as recited in clause 35, wherein the reporting request comprises a timing parameter, wherein the timing parameter identifies a time interval for the user equipment to operate in the first mode.

[0173] 37. A computer-implemented method according to any of clauses 35-36, wherein the report request comprises a resource parameter identifying one or more resources from which the user equipment generates the positioning data.

[0174] 38. A computer-implemented method according to clause 37, wherein the one or more resources include resources selected from the group consisting of: resources associated with assistance, resources associated with a positioning frequency layer (PFL), resources associated with a transmit receive (TRP) ID, resources associated with a positioning reference signal (PRS) resource, or a combination thereof.

[0175] 39. A computer-implemented method according to any of clauses 35-38, wherein the first positioning data is associated with a first subset of resources and the second positioning data is associated with a second subset of resources.

[0176] 40. The computer-implemented method of clause 39, wherein the second subset of resources comprises the first subset of resources.

[0177] 41. A computer-implemented method according to any of clauses 35-40, wherein the report request causes the user equipment to:

[0178] generating a first timestamp associated with a first element of the one or more elements of the first positioning data and a second timestamp associated with a second element of the one or more elements of the second positioning data, wherein the first element corresponds to the second element;

[0179] Comparing the first timestamp and the second timestamp;

[0180] determining that the first timestamp and the second timestamp are within a predetermined time interval; and

[0181] Report data is generated based on the determination.

[0182] 42. The computer-implemented method of clause 41, wherein the report request further causes the user device to determine that the first timestamp and the second timestamp match.

[0183] 43. A computer-implemented method according to any of clauses 35-42, wherein the reporting request further causes the user device to simultaneously generate one or more elements of the first positioning data using a first process and to generate one or more elements of the second positioning data using a second process.

[0184] 44. The computer-implemented method of clause 43, wherein the report request further causes the user device to:

[0185] comparing the first positioning data and the second positioning data;

[0186] Based on the comparison, determining that a difference between a first element of the first positioning data and a corresponding first element of the second positioning data satisfies a difference threshold; and

[0187] Based on the determined difference satisfying the difference threshold, reporting data is sent to a processor of the location server.

[0188] 45. A computer-implemented method as recited in clause 44, wherein prior to receiving the report request from the processor of the location server, the user equipment operates in the second mode, and wherein receipt of the report request by the user equipment causes the user equipment to:

[0189] switching from operating in the second mode to the first mode;

[0190] determining that at least a first element of the first positioning data and a corresponding first element of the second positioning data are within a predetermined time margin;

[0191] comparing at least a first element of the first positioning data with a corresponding first element of the second positioning data; and

[0192] In the event that the comparison is greater than a predetermined threshold, a switch is made to operate in a third mode to generate additional positioning data.

[0193] 46. ​​A computer-implemented method as described in clause 45, wherein when the user equipment operates in the third mode, the user equipment implements a second process to generate additional positioning data.

[0194] 47. A computer-implemented method as recited in clause 45, wherein, when the user equipment operates in the third mode, the user equipment implements a third process to generate additional positioning data.

[0195] 48. A computer-implemented method according to any of clauses 35-47, wherein the first process is a trained machine learning process.

[0196] 49. A computer-implemented method according to any of clauses 35-48, wherein the second process is a trained machine learning process.

[0197] 50. The computer-implemented method of any of clauses 35-49, wherein the report request comprises model parameters, wherein the model parameters identify one or more processes to be monitored, the one or more processes comprising a first process and a second process.

[0198] 51. A positioning computing device, comprising:

[0199] Means for sending, by a processor of the location server, a report request for positioning data to a user equipment, wherein receipt of the report request by the user equipment causes the user equipment to perform operations in a first mode, the operations comprising:

[0200] Implementing a first process to generate first positioning data, and implementing a second process to generate second positioning data; and

[0201] generating report data indicating a comparison between the first positioning data and the second positioning data; and

[0202] means for receiving, by a processor, report data from a user device; and

[0203] Means for generating instructions based on the report data and sending the instructions to the user equipment, wherein the instructions cause the user equipment to implement at least one of the first process and the second process.

[0204] 52. The positioning computing device of clause 51, wherein the reporting request comprises a timing parameter, wherein the timing parameter identifies a time interval for the user equipment to operate in the first mode.

[0205] 53. A positioning computing device according to any of clauses 51-52, wherein the report request comprises a resource parameter identifying one or more resources from which the user equipment generates the positioning data.

[0206] 54. A positioning computing device according to clause 53, wherein the one or more resources include resources selected from the group consisting of: resources associated with assistance, resources associated with a positioning frequency layer (PFL), resources associated with a transmit receive (TRP) ID, resources associated with a positioning reference signal (PRS) resource, or a combination thereof.

[0207] 55. A positioning computing device according to any of clauses 51-54, wherein the first positioning data is associated with a first subset of resources and the second positioning data is associated with a second subset of resources.

[0208] 56. The method of positioning a computing device according to clause 55, wherein the second subset of resources includes the first subset of resources.

[0209] 57. A positioning computing device according to any of clauses 51 to 56, wherein the reporting request causes the user equipment to:

[0210] generating a first timestamp associated with a first element of the one or more elements of the first positioning data and a second timestamp associated with a second element of the one or more elements of the second positioning data, wherein the first element corresponds to the second element;

[0211] Comparing the first timestamp and the second timestamp;

[0212] determining that the first timestamp and the second timestamp are within a predetermined time interval; and

[0213] Report data is generated based on the determination.

[0214] 58. The positioning computing device of clause 57, wherein the reporting request further causes the user device to determine that the first timestamp and the second timestamp match.

[0215] 59. A positioning computing device according to any of clauses 51-58, wherein the report request further causes the user device to simultaneously generate one or more elements of the first positioning data using a first process and to generate one or more elements of the second positioning data using a second process.

[0216] 60. The location computing device of clause 59, wherein the report request further causes the user device to:

[0217] comparing the first positioning data and the second positioning data;

[0218] Based on the comparison, determining that a difference between a first element of the first positioning data and a corresponding first element of the second positioning data satisfies a difference threshold; and

[0219] Based on the determined difference satisfying the difference threshold, reporting data is sent to a processor of the location server.

[0220] 61. The positioning computing device of clause 60, wherein prior to receiving the report request from the processor of the location server, the user device operates in the second mode, and wherein receipt of the report request by the user device causes the user device to:

[0221] switching from operating in the second mode to the first mode;

[0222] determining that at least a first element of the first positioning data and a corresponding first element of the second positioning data are within a predetermined time margin;

[0223] comparing at least a first element of the first positioning data with a corresponding first element of the second positioning data; and

[0224] In the event that the comparison is greater than a predetermined threshold, a switch is made to operate in a third mode to generate additional positioning data.

[0225] 62. The positioning computing device of clause 61, wherein when the user device operates in the third mode, the user device implements a second process to generate additional positioning data.

[0226] 63. The positioning computing device of clause 61, wherein when the user device operates in the third mode, the user device implements a third process to generate additional positioning data.

[0227] 64. A positioning computing device according to any of clauses 51-63, wherein the first process is a trained machine learning process.

[0228] 65. A positioning computing device according to any of clauses 51-64, wherein the second process is a trained machine learning process.

[0229] 66. The positioning computing device of any of clauses 51-65, wherein the report request comprises a model parameter, wherein the model parameter identifies one or more processes to be monitored, the one or more processes comprising a first process and a second process.

[0230] 67. A device comprising:

[0231] a non-transitory machine-readable storage medium storing instructions; and

[0232] At least one processor coupled to a non-transitory machine-readable storage medium, the at least one processor configured to execute instructions to:

[0233] sending a report request for positioning data to a user equipment;

[0234] receiving report data indicating a comparison between first positioning data generated from a first process and second positioning data generated from a second process; and

[0235] Based on the report data, instructions are generated and sent to the user equipment, the instructions causing the user equipment to implement a first process.

[0236] 68. The apparatus of clause 67, wherein the at least one processor is configured to further execute the instructions to:

[0237] receiving third positioning data from the user equipment;

[0238] Based on the third positioning data, a second instruction is generated and sent to the user equipment, so that the user equipment implements a second process.

[0239] 69. Apparatus as described in any of clauses 67-68, wherein the report request comprises a timing parameter identifying a time interval for the user equipment to operate in the first mode.

[0240] 70. Apparatus according to any of clauses 67-69, wherein the report request comprises a resource parameter, the resource parameter identifying one or more resources from which the user equipment is to generate the positioning data.

[0241] 71. An apparatus according to clause 70, wherein the one or more resources include resources selected from the group consisting of: resources associated with an auxiliary data ID, resources associated with a positioning frequency layer (PFL) ID, resources associated with a transmit receive (TRP) ID, resources associated with a positioning reference signal (PRS) set ID, a set of positioning reference signal (PRS) resources, or a combination thereof.

[0242] 72. Apparatus according to any of clauses 67-71, wherein the first positioning data is associated with a first subset of resources and the second positioning data is associated with a second subset of resources.

[0243] 73. An apparatus as described in clause 72, wherein the second subset of resources includes the first subset of resources.

[0244] 74. An apparatus as described in any of clauses 67-74, wherein the first positioning data and the second positioning data each include one or more elements associated with the same resource.

[0245] 75. An apparatus according to any of clauses 67-75, wherein the first process is a trained machine learning process.

[0246] 76. An apparatus as described in any of clauses 67-75, wherein the second process is a trained machine learning process.

[0247] 77. An apparatus as described in any of clauses 67-76, wherein the report request includes model parameters, wherein the model parameters identify one or more processes to be monitored, the one or more processes comprising a first process and a second process.

[0248] 78. A non-transitory machine-readable storage medium storing instructions that, when executed by at least one processor of a server, cause the at least one processor to perform operations comprising:

[0249] sending a report request for positioning data to a user equipment;

[0250] receiving report data indicating a comparison between first positioning data generated from a first process and second positioning data generated from a second process; and

[0251] Based on the report data, instructions are generated and sent to the user equipment, the instructions causing the user equipment to implement a first process.

[0252] 79. The non-transitory machine-readable storage medium of clause 78, further comprising:

[0253] receiving third positioning data from the user equipment;

[0254] Based on the third positioning data, a second instruction is generated and sent to the user equipment, so that the user equipment implements a second process.

[0255] 80. A non-transitory machine-readable storage medium as described in any of clauses 78-79, wherein the reporting request comprises a timing parameter identifying a time interval for the user equipment to operate in the first mode.

[0256] 81. A non-transitory machine-readable storage medium as described in any of clauses 78-80, wherein the report request includes a resource parameter, the resource parameter identifying one or more resources from which the user equipment generates the positioning data.

[0257] 82. A non-transitory machine-readable storage medium as described in clause 81, wherein the one or more resources include resources selected from the group consisting of: resources associated with an auxiliary data ID, resources associated with a positioning frequency layer (PFL) ID, resources associated with a transmit receive (TRP) ID, resources associated with a positioning reference signal (PRS) set ID, a set of positioning reference signals (PRS), or a combination thereof.

[0258] 83. The non-transitory machine-readable storage medium of any of clauses 78-82, wherein the first positioning data is associated with a first subset of resources and the second positioning data is associated with a second subset of resources.

[0259] 84. The non-transitory machine-readable storage medium of clause 83, wherein the second subset of resources includes the first subset of resources.

[0260] 85. The non-transitory machine-readable storage medium of any of clauses 78-84, wherein the first positioning data and the second positioning data each comprise one or more elements associated with the same resource.

[0261] 86. A non-transitory machine-readable storage medium as described in any of clauses 78-85, wherein the first process is a trained machine learning process.

[0262] 87. A non-transitory machine-readable storage medium as described in any of clauses 78-86, wherein the second process is a trained machine learning process.

[0263] 88. The non-transitory machine-readable storage medium of any of clauses 78-87, wherein the report request comprises model parameters, wherein the model parameters identify one or more processes to be monitored, the one or more processes comprising a first process and a second process.

[0264] 89. A computer-implemented method comprising:

[0265] sending a report request for positioning data to a user equipment;

[0266] receiving report data indicating a comparison between first positioning data generated from a first process and second positioning data generated from a second process; and

[0267] Based on the report data, instructions are generated and sent to the user equipment, the instructions causing the user equipment to implement a first process.

[0268] 90. The computer-implemented method of clause 89, further comprising:

[0269] receiving third positioning data from the user equipment;

[0270] Based on the third positioning data, a second instruction is generated and sent to the user equipment, so that the user equipment implements a second process.

[0271] 91. A computer-implemented method as described in any of clauses 89-90, wherein the reporting request comprises a timing parameter identifying a time interval for the user equipment to operate in the first mode.

[0272] 92. A computer-implemented method as described in any of clauses 89-91, wherein the report request includes a resource parameter, the resource parameter identifying one or more resources from which the user equipment generates the positioning data.

[0273] 93. A computer-implemented method according to clause 92, wherein the one or more resources include resources selected from the group consisting of: resources associated with an auxiliary data ID, resources associated with a positioning frequency layer (PFL) ID, resources associated with a transmit receive (TRP) ID, resources associated with a positioning reference signal (PRS) set ID, a set of positioning reference signal (PRS) resources, or a combination thereof.

[0274] 94. A computer-implemented method as recited in any of clauses 89-93, wherein the first positioning data is associated with a first subset of resources and the second positioning data is associated with a second subset of resources.

[0275] 95. The computer-implemented method of clause 94, wherein the second subset of resources comprises the first subset of resources.

[0276] 95. A computer-implemented method as recited in any of clauses 89-95, wherein the first positioning data and the second positioning data each comprise one or more elements associated with the same resource.

[0277] 96. A computer-implemented method according to any of clauses 89-95, wherein the first process is a trained machine learning process.

[0278] 97. A computer-implemented method according to any of clauses 89-96, wherein the second process is a trained machine learning process.

[0279] 98. A computer-implemented method as described in any of clauses 89-97, wherein the report request includes model parameters, wherein the model parameters identify one or more processes to be monitored, the one or more processes including a first process and a second process.

[0280] 99. A positioning computing device, comprising:

[0281] means for sending a report request for positioning data to a user equipment;

[0282] means for receiving report data indicating a comparison between first positioning data generated from a first process and second positioning data generated from a second process; and

[0283] Means for generating instructions based on the report data and sending the instructions to the user equipment, the instructions causing the user equipment to implement a first process.

[0284] 100. The positioning computing device of clause 99, further comprising:

[0285] means for receiving third positioning data from a user equipment;

[0286] The invention also provides a unit for generating a second instruction based on the third positioning data and sending the second instruction to the user equipment, wherein the second instruction enables the user equipment to implement a second process.

[0287] 101. A positioning computing device according to any of clauses 99-100, wherein the report request includes a timing parameter, the timing parameter identifying a time interval for the user equipment to operate in the first mode.

[0288] 102. A positioning computing device according to any of clauses 99-101, wherein the report request includes a resource parameter, the resource parameter identifying one or more resources from which the user device generates positioning data.

[0289] 103. A positioning computing device according to clause 102, wherein the one or more resources include resources selected from the group consisting of: resources associated with an auxiliary data ID, resources associated with a positioning frequency layer (PFL) ID, resources associated with a transmit receive (TRP) ID, resources associated with a positioning reference signal (PRS) set ID, a set of positioning reference signal (PRS) resources, or a combination thereof.

[0290] 104. A positioning computing device according to any of clauses 99-103, wherein the first positioning data is associated with a first subset of resources and the second positioning data is associated with a second subset of resources.

[0291] 105. The positioning computing device of clause 104, wherein the second subset of resources includes the first subset of resources.

[0292] 106. A positioning computing device according to any of clauses 99-105, wherein the first positioning data and the second positioning data each include one or more elements associated with the same resource.

[0293] 107. A positioning computing device according to any of clauses 99-105, wherein the first process is a trained machine learning process.

[0294] 108. A positioning computing device according to any of clauses 99-106, wherein the second process is a trained machine learning process.

[0295] 109. A positioning computing device as described in any of clauses 99-108, wherein the report request includes a model parameter, wherein the model parameter identifies one or more processes to be monitored, the one or more processes including a first process and a second process.

[0296] C. Exemplary Hardware and Software Implementations

[0297] Embodiments of the subject matter and functional operations described in the present disclosure may be implemented in digital electronic circuit systems, tangible embedded computer software or firmware, computer hardware (including the structures disclosed in this specification and their structural equivalents), or in a combination of one or more of them. Embodiments of the subject matter described in the present disclosure, including user equipment (UE) engine 208A, application 212B, application program interface (API) 402, process module 404, analysis module 406, notification module 408, API 502, notification engine 520, and API 602, may be implemented as one or more computer programs, i.e., one or more computer program instruction modules encoded on a tangible non-transitory program carrier for execution by a data processing device (or computing system) or for controlling the operation of the data processing device (or computing system). Additionally or alternatively, the program instructions may be encoded on artificially generated propagation signals (e.g., machine-generated electrical, optical, or electromagnetic signals) that are generated to encode information for transmission to a suitable receiver device for execution by the data processing device. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access storage device, or a combination of one or more of them.

[0298] The terms "device," "equipment," and "system" refer to data processing hardware and include various devices, equipment, and machines for processing data, including, for example, a programmable processor, a computer, or multiple processors or computers. A device, equipment, or system may also be or further include a dedicated logic circuit system (e.g., an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit)). In addition to hardware, a device, equipment, or system may optionally include code that creates an execution environment for a computer program (e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them).

[0299] A computer program may also be referred to or described as a program, software, software application, application, engine, module, software module, script or code, and may be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages, and may be deployed in any form, including as a standalone program or as a module, component, subroutine or other unit suitable for use in a computing environment. A computer program may, but need not, correspond to a file in a file system. A program may be stored in a portion of a file that stores other programs or data, such as one or more scripts stored in a markup language document; or in a single file dedicated to the program in question; or in multiple coordinated files (e.g., files storing one or more modules, subroutines or portions of code). A computer program may be deployed for execution on one or more computers that are located at the same location, or distributed across multiple locations and interconnected by a communication network.

[0300] The processes and logic flows described in this specification can be performed by one or more programmable computers that execute one or more computer programs to perform functions by operating input data and generating output. The processes and logic flows can also be performed by a dedicated logic circuit system, and the device can also be implemented as a dedicated logic circuit system (for example, FPGA (field programmable gate array) or ASIC (application-specific integrated circuit)).

[0301] For example, a computer suitable for executing a computer program includes a general or special microprocessor or both, or a central processing unit of any other type. Usually, the central processing unit will receive instructions and data from a read-only memory or a random access memory or both. The basic element of a computer is a central processing unit for executing or running instructions and one or more memory devices for storing instructions and data. Usually, a computer will also include, or be operably coupled to one or more large-capacity storage devices (for example, magnetic disks, magneto-optical disks or optical disks) for storing data, to receive data from these devices or to transmit data thereto, or both. However, a computer does not need to have these devices. In addition, a computer can also be embedded in other devices (for example, mobile phones, personal digital assistants (PDAs), mobile audio or video players, game consoles, global positioning systems (GPS) or assisted global positioning systems (AGPS) receivers, or portable storage devices (for example, universal serial bus (USB) flash drives, etc.)).

[0302] Computer-readable media suitable for storing computer program instructions and data include various forms of nonvolatile memory, media, and memory devices (for example, including semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices); magnetic disks (e.g., internal hard disks or removable disks); magneto-optical disks; and CD-ROM and DVD-ROM disks). The processor and memory may be supplemented by, or incorporated in, special purpose logic circuitry.

[0303] To enable interaction with a user, embodiments of the subject matter described in this specification may be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) display) and a keyboard and pointing device (e.g., a mouse or trackball), the display device being used to display information to the user, and the user being able to provide input to the computer through the keyboard and pointing device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and the user's input may be received in any form, including sound, voice, or tactile input. In addition, the computer may also interact with the user by sending documents to and receiving documents from a device used by the user; for example, sending a web page to a web browser on a user's device in response to a request received from the web browser.

[0304] Implementations of the subject matter described in this specification may be implemented in a computing system that includes a back-end component (e.g., a data server), or includes a middleware component (e.g., an application server), or includes a front-end component (e.g., a client computer with a graphical user interface or a web browser through which a user can interact with implementations of the subject matter described in this specification), or any combination of one or more such back-end, middleware, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks (LANs) and wide area networks (WANs) (e.g., the Internet).

[0305] A computing system may include a client and a server. The client and the server are usually remote from each other and typically interact through a communication network. The relationship between the client and the server is generated by a computer program running on a corresponding computer, and there is a relationship between the client and the server. In some implementations, the server sends data (e.g., an HTML page) to a user device, for example, to display data to a user interacting with the user device acting as a client and to receive user input from the user. Data generated at the user device (e.g., the result of the user interaction) can be received at the server from the user device receiver.

[0306] Although this specification includes many specific details, these details should not be interpreted as limitations on the scope of the present disclosure or the content that can be claimed, but should be interpreted as descriptions of features specific to a particular embodiment of the present disclosure. Certain features described in the context of the respective embodiments of this specification may also be implemented in combination in a single embodiment. Conversely, the various features described in the context of a single embodiment may also be implemented in multiple embodiments, respectively, or in any suitable sub-combination. In addition, although the above-mentioned features may be described as working in certain combinations, and even initially claimed in this way, in some cases, one or more features in the claimed combination may be deleted from the combination, and the claimed combination may be directed to a variant of a sub-combination or a sub-combination.

[0307] Similarly, although the operations are depicted in a particular order in the accompanying drawings, this should not be understood as requiring that the operations be performed in the particular order shown or in a sequential order, or that all of the operations shown be performed, in order to achieve the desired effect. In some cases, multitasking and parallel processing may be advantageous. In addition, the separation of various system components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0308] Each time an HTML file is mentioned, other file types or formats may be substituted. For example, an HTML file may be substituted with XML, JSON, plain text, or other types of files. Additionally, when a table or hash table is mentioned, other data structures (e.g., a spreadsheet, a relational database, or a structured file) may also be used.

[0309] Various embodiments have been described herein with reference to the accompanying drawings. It will, however, be evident that various modifications and changes may be made thereto, and additional embodiments may be implemented, without departing from the broader scope of the disclosed embodiments as set forth in the claims below.

[0310] In addition, unless otherwise expressly defined herein, all terms should be given the broadest possible interpretation, including the meaning implied in the specification and the meaning understood by those skilled in the art and / or the meaning defined in dictionaries, papers, etc. It should also be noted that the singular forms "a", "an", and "the" used in this specification and the appended claims include plural references unless otherwise specified, and when used in this specification, the terms "comprises" and / or "comprising" refer to the presence or addition of one or more other features, aspects, steps, operations, elements, components and / or groups thereof. In addition, the terms "coupled", "coupled", "operably coupled", "operably connected", etc. should be broadly understood to connect devices or components together mechanically, electrically, wired, wirelessly or in other ways, so that the connection allows the related devices or components to interoperate (e.g., communicate) as expected according to this relationship. In the present disclosure, "or" is used to mean "and / or" unless otherwise specified. Furthermore, the use of the term "including" and other forms such as "includes" and "included" is not limiting. In addition, terms such as "element" or "component" encompass both elements and components that include one unit and elements and components that include more than one subunit, unless otherwise specifically stated. Additionally, the section headings used herein are for organizational purposes only and are not to be construed as limitations on the subject matter described.

[0311] The foregoing is provided to illustrate, explain and describe embodiments of the present disclosure. Modifications and adjustments to the embodiments will be apparent to those skilled in the art and may be made without departing from the scope or spirit of the present disclosure.

Claims

1. A device, include: a non-transitory machine-readable storage medium storing instructions; as well as at least one processor coupled to the non-transitory machine-readable storage medium, the at least one processor configured to execute the instructions to perform the following operations: Sending a report request for positioning data to a user equipment, wherein receipt of the report request by the user equipment causes the user equipment to perform operations in a first mode, the operations comprising: Implementing a first process to generate first positioning data, and implementing a second process to generate second positioning data; and generating report data indicating a comparison between the first positioning data and the second positioning data; receiving the report data from the user equipment; and Based on the report data, an instruction is generated and sent to the user equipment, wherein the instruction causes the user equipment to implement at least one of the first process or the second process.

2. The device according to claim 1, in, The reporting request comprises a timing parameter identifying a time interval for the user equipment to operate in the first mode.

3. The device according to claim 1, in, The report request includes a resource parameter identifying one or more resources from which the user equipment generates the positioning data.

4. The device according to claim 3, in, The one or more resources include resources selected from the group consisting of: resources associated with an assistance data ID, resources associated with a positioning frequency layer (PFL) ID, resources associated with a transmit receive (TRP) ID, resources associated with a positioning reference signal (PRS) set ID, a set of positioning reference signal (PRS) resources, or a combination thereof.

5. The device according to claim 1, in, The first positioning data is associated with a first subset of resources and the second positioning data is associated with a second subset of resources.

6. The device according to claim 5, in, The second subset of resources includes the first subset of resources.

7. The device according to claim 1, in, The first positioning data and the second positioning data each include one or more elements associated with the same resource.

8. The device according to claim 7, in, The report request causes the user equipment to perform the following operations: generating a first timestamp associated with a first element of the one or more elements of the first positioning data and a second timestamp associated with a second element of the one or more elements of the second positioning data, wherein the first element corresponds to the second element; comparing the first timestamp and the second timestamp; determining that the first timestamp and the second timestamp are within a predetermined time interval; and The reporting data is generated based on the determination.

9. The device according to claim 8, in, The report request further causes the user equipment to perform the following operations: It is determined that the first timestamp and the second timestamp match.

10. The device according to claim 1, in, The report request further causes the user equipment to perform the following operations: The first process is concurrently used to generate one or more elements of the first positioning data, and the second process is used to generate one or more elements of the second positioning data.

11. The device according to claim 10, in, The report request further causes the user equipment to perform the following operations: comparing the first positioning data and the second positioning data; Based on the comparison, determining that a difference between a first element of the first positioning data and a corresponding first element of the second positioning data satisfies a difference threshold; as well as Based on the determined difference satisfying the difference threshold, the report data is sent to the device.

12. The device according to claim 11, in, Prior to receiving the report request from the apparatus, the user equipment operates in a second mode and implements the first process to generate the first positioning data, and wherein the reception of the report request by the user equipment causes the user equipment to: switching from operating in the second mode to the first mode; determining that at least the first element of the first positioning data and the corresponding first element of the second positioning data are within a predetermined time margin; comparing at least the first element of the first positioning data with the corresponding first element of the second positioning data; and In the event that the comparison is greater than a predetermined threshold, switching to operation in a third mode to generate additional positioning data.

13. The device according to claim 12, in, When the user equipment operates in the third mode, the user equipment implements the second process to generate the additional positioning data.

14. The device according to claim 12, in, When the user equipment operates in the third mode, the user equipment implements a third process to generate the additional positioning data.

15. The device according to claim 1, in, The first process is a trained machine learning process.

16. The device according to claim 15, in, The second process is a trained machine learning process.

17. The device according to claim 1, in, The report request includes model parameters, wherein the model parameters identify one or more processes to be monitored, the one or more processes including the first process and the second process.

18. A non-transitory machine-readable storage medium storing instructions that, when executed by at least one processor of a location server, cause the at least one processor to perform operations comprising: sending a report request for positioning data to a user equipment, in, Receipt of the report request by the user equipment causes the user equipment to perform operations in a first mode, the operations comprising: Implementing a first process to generate first positioning data, and implementing a second process to generate second positioning data; and generating report data indicating a comparison between the first positioning data and the second positioning data; and receiving the report data from the user equipment; and Based on the report data, an instruction is generated and sent to the user equipment, wherein the instruction causes the user equipment to implement at least one of the first process and / or the second process.

19. A computer-implemented method, include: A processor of the location server sends a report request for positioning data to a user equipment, wherein receipt of the report request by the user equipment causes the user equipment to perform operations in a first mode, the operations comprising: Implementing a first process to generate first positioning data, and implementing a second process to generate second positioning data; and generating report data indicating a comparison between the first positioning data and the second positioning data; and receiving, by the processor, the report data from the user equipment; and Based on the report data, instructions are generated and sent to the user equipment, wherein the instructions cause the user equipment to implement at least one of the first process and the second process.

20. The computer-implemented method of claim 19, in, The report request comprises a timing parameter, wherein the timing parameter identifies a time interval for the user equipment to operate in the first mode.

21. The computer-implemented method of claim 19, in, The report request includes a resource parameter identifying one or more resources from which the user equipment generates positioning data.

22. The computer-implemented method of claim 21, in, The one or more resources include resources selected from the group consisting of: resources associated with assistance, resources associated with a positioning frequency layer (PFL), resources associated with a transmit receive (TRP) ID, resources associated with a positioning reference signal (PRS) resource, or a combination thereof.

23. The computer-implemented method of claim 19, in, The first positioning data is associated with a first subset of resources and the second positioning data is associated with a second subset of resources.

24. The computer-implemented method of claim 23, in, The second subset of resources includes the first subset of resources.

25. The computer-implemented method of claim 24, in, The report request causes the user equipment to perform the following operations: generating a first timestamp associated with a first element of the one or more elements of the first positioning data and a second timestamp associated with a second element of the one or more elements of the second positioning data, wherein the first element corresponds to the second element; comparing the first timestamp and the second timestamp; determining that the first timestamp and the second timestamp are within a predetermined time interval; and The reporting data is generated based on the determination.

26. The computer-implemented method of claim 25, in, The report request also causes the user equipment to determine that the first timestamp and the second timestamp match.

27. The computer-implemented method of claim 19, in, The report request further causes the user equipment to simultaneously generate one or more elements of the first positioning data using the first process and to generate one or more elements of the second positioning data using the second process.

28. The computer-implemented method of claim 27, in, The report request further causes the user equipment to perform the following operations: comparing the first positioning data and the second positioning data; Based on the comparison, determining that a difference between a first element of the first positioning data and a corresponding first element of the second positioning data satisfies a difference threshold; as well as Based on the determined difference satisfying the difference threshold, the report data is sent to the processor of the location server.

29. The computer-implemented method of claim 28, in, Prior to receiving the report request from the processor of the location server, the user equipment operates in a second mode, and wherein the receipt of the report request by the user equipment causes the user equipment to: switching from operating in the second mode to the first mode; determining that at least the first element of the first positioning data and the corresponding first element of the second positioning data are within a predetermined time margin; comparing at least the first element of the first positioning data with the corresponding first element of the second positioning data; and In the event that the comparison is greater than a predetermined threshold, switching to operation in a third mode to generate additional positioning data.

30. The computer-implemented method of claim 29, in, When the user equipment operates in the third mode, the user equipment implements the second process to generate the additional positioning data.

31. The computer-implemented method of claim 29, in, When the user equipment operates in the third mode, the user equipment implements a third process to generate the additional positioning data.

32. A device, include: a non-transitory machine-readable storage medium storing instructions; as well as at least one processor coupled to the non-transitory machine-readable storage medium, the at least one processor configured to execute the instructions to perform the following operations: sending a report request for positioning data to a user equipment; receiving report data indicating a comparison between first positioning data generated from a first process and second positioning data generated from a second process; as well as Based on the report data, an instruction is generated and sent to the user equipment so that the user equipment implements the first process.

33. The device according to claim 32, in, The at least one processor is configured to further execute the instructions to perform the following operations: receiving third positioning data from the user equipment; Based on the third positioning data, a second instruction is generated and sent to the user equipment, so that the user equipment implements the second process.

34. A computer-implemented method, include: sending a report request for positioning data to a user equipment; receiving report data indicating a comparison between first positioning data generated from a first process and second positioning data generated from a second process; as well as Based on the report data, an instruction is generated and sent to the user equipment so that the user equipment implements the first process.

35. The computer-implemented method of claim 34, further comprising: include: receiving third positioning data from the user equipment; Based on the third positioning data, a second instruction is generated and sent to the user equipment, so that the user equipment implements the second process.

Images