Information acquisition method and apparatus, and communication device
Through information interaction between devices, the model accuracy information is obtained and transmitted, and the problem of difficult to determine the accuracy of the model in intelligent data analysis in the prior art is solved, and the accuracy and effectiveness of data analysis are improved.
Patent Information
- Application Number
- PCT/CN2024/091307
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-12
- Filing Date
- 2024-05-07
- Publication Date
- 2024-11-21
AI Technical Summary
In the existing communication systems, some devices involved in intelligent data analysis cannot accurately understand the accuracy of the model, resulting in poorer intelligent data analysis.
Through an information acquisition method, information used to determine the accuracy of the model is interactively obtained and transmitted between devices, ensuring that the devices participating in intelligent data analysis can accurately obtain and determine the accuracy of the model, thereby improving the data analysis effect.
Through the information interaction between devices, the device can accurately obtain and determine the accuracy of the model, improve the effect of intelligent data analysis and ensure the accuracy and reliability of data analysis results.
Smart Images

Figure CN2024091307_21112024_PF_FP_ABST
Abstract
Description
Information acquisition method, device and communication equipment
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS
[0002] This application claims priority to Chinese patent application No. 202310536698.8 filed in China on May 12, 2023, the entire contents of which are incorporated herein by reference. Technical Field
[0003] The present application belongs to the field of communication technology, and specifically relates to an information acquisition method, device and communication equipment. Background Art
[0004] In communications systems, some network elements can assist other devices in making strategic decisions through intelligent data analysis, thereby leveraging artificial intelligence (AI) or machine learning (ML) to enhance the intelligence of these decisions. However, in related technologies, some devices involved in intelligent data analysis are unable to determine the accuracy of the model, resulting in poor results.
[0005] Summary of the Invention
[0006] The embodiments of the present application provide an information acquisition method, apparatus, and communication device, which can solve the problem of poor intelligent data analysis results.
[0007] In a first aspect, a method for obtaining information is provided, comprising:
[0008] The first device obtains first information, where the first information is used to determine the accuracy of the first model.
[0009] In a second aspect, a method for obtaining information is provided, comprising:
[0010] The second device sends first information to the first device, where the first information is used to determine the accuracy of the first model.
[0011] A third aspect provides an information acquisition method, comprising:
[0012] The third device receives the first information or the second information sent by the first device, where the first information or the second information is used to determine the accuracy of the requested first model.
[0013] In a fourth aspect, an information acquisition device is provided. A first device includes the information acquisition device, including:
[0014] The acquisition module is used to acquire first information, where the first information is used to determine the accuracy of the first model.
[0015] In a fifth aspect, an information acquisition device is provided. The second device includes the information acquisition device, including:
[0016] The sending module is used to send first information to the first device, where the first information is used to determine the accuracy of the first model.
[0017] In a sixth aspect, an information acquisition device is provided. The third device includes the information acquisition device, including:
[0018] The first receiving module is configured to receive first information or second information sent by a first device, where the first information or second information is used to determine the accuracy of the requested first model.
[0019] In the seventh aspect, a communication device is provided, which includes a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the method described in the first aspect, the second aspect, or the third aspect are implemented.
[0020] In an eighth aspect, a first device is provided, comprising a processor and a communication interface, wherein the processor is used to: obtain first information, and the first information is used to determine the accuracy of a first model.
[0021] In a ninth aspect, a second device is provided, comprising a processor and a communication interface, wherein the communication interface is used to send first information to the first device, the first information being used to determine the accuracy of the first model.
[0022] In the tenth aspect, a third device is provided, comprising a processor and a communication interface, wherein the communication interface is used to: receive first information or second information sent by the first device, wherein the first information or second information is used to determine the accuracy of the requested first model.
[0023] In the eleventh aspect, an information acquisition system is provided, comprising: a first device, a second device and a third device, wherein the first device can be used to execute the steps of the method described in the first aspect, the second device can be used to execute the steps of the method described in the second aspect, and the third device can be used to execute the steps of the method described in the third aspect.
[0024] In the twelfth aspect, a readable storage medium is provided, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented, or the steps of the method described in the second aspect are implemented, or the steps of the method described in the third aspect are implemented.
[0025] In the thirteenth aspect, a chip is provided, which includes a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the method as described in the first aspect, or implement the method as described in the second aspect, or implement the steps of the method as described in the third aspect.
[0026] In the fourteenth aspect, a computer program / program product is provided, which is stored in a storage medium and is executed by at least one processor to implement the steps of the method described in the first aspect, or the steps of the method described in the second aspect, or the steps of the method described in the third aspect.
[0027] In an embodiment of the present application, a first device obtains first information, which is used to determine the accuracy of a first model, and can support the first device to obtain the accuracy of the first model more accurately, thereby improving the effect of intelligent data analysis; through the interaction of the first device, the second device and the third device, the information used to determine the accuracy of the first model is used. When the first device, the second device and the third device participate in intelligent data analysis, the information used to determine the accuracy of the first model can support the devices participating in the intelligent data analysis to unify the method of determining the accuracy of the model, thereby further improving the effect of intelligent data analysis. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] FIG1 is a block diagram of a wireless communication system to which embodiments of the present application may be applied;
[0029] FIG2 is a schematic diagram of a data analysis process in the related art;
[0030] FIG3 is a flowchart of an information acquisition method according to an embodiment of the present application;
[0031] FIG4 is a second flowchart of an information acquisition method provided in an embodiment of the present application;
[0032] FIG5 is a third flowchart of an information acquisition method provided in an embodiment of the present application;
[0033] FIG6 is a fourth flowchart of an information acquisition method provided in an embodiment of the present application;
[0034] FIG7 is a schematic diagram of a structure of an information acquisition device according to an embodiment of the present application;
[0035] FIG8 is a second structural diagram of an information acquisition device provided in an embodiment of the present application;
[0036] FIG9 is a third structural diagram of an information acquisition device provided in an embodiment of the present application;
[0037] FIG10 is a schematic structural diagram of a communication device provided in an embodiment of the present application;
[0038] FIG11 is a schematic structural diagram of a terminal provided in an embodiment of the present application;
[0039] FIG12 is a schematic diagram of a structure of a network side device according to an embodiment of the present application;
[0040] FIG13 is a second structural diagram of a network-side device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0041] The following will be combined with the accompanying drawings in the embodiments of this application to clearly describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of this application.
[0042] The terms "first", "second", etc. in this application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same type, and do not limit the number of objects, for example, the first object can be one or more. In addition, "or" in this application represents at least one of the connected objects. For example, "A or B" covers three options, namely, Option 1: including A but not including B; Option 2: including B but not including A; Option 3: including both A and B. The character " / " generally indicates that the objects associated before and after are in an "or" relationship.
[0043] The term "indication" in this application can be either a direct indication (or explicit indication) or an indirect indication (or implicit indication). A direct indication can be understood as the sender explicitly informing the receiver of specific information, the operation to be performed, or the requested result, etc. in the instruction sent; an indirect indication can be understood as the receiver determining the corresponding information based on the instruction sent by the sender, or making a judgment and determining the operation to be performed or the requested result, etc. based on the judgment result.
[0044] It is worth noting that the technology described in the embodiments of the present application is not limited to the Long Term Evolution (LTE) / LTE-Advanced (LTE-A) system, but can also be used in other wireless communication systems, such as Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), Orthogonal Frequency Division Multiple Access (OFDMA), Single-carrier Frequency-Division Multiple Access (SC-FDMA) or other systems. The terms "system" and "network" in the embodiments of the present application are often used interchangeably, and the technology described can be used for the systems and radio technologies mentioned above, as well as for other systems and radio technologies. The following description describes a New Radio (NR) system for illustrative purposes, and NR terminology is used in most of the following description, but these technologies can also be applied to systems other than NR systems, such as 6th generation (6G) systems. th Generation, 6G) communication system.
[0045] FIG1 is a block diagram of a wireless communication system applicable to an embodiment of the present application. The wireless communication system includes a terminal 11 and a network-side device 12. The terminal 11 may be a mobile phone, a tablet computer (Tablet Personal Computer), a laptop computer (Laptop Computer), a notebook computer, a personal digital assistant (PDA), a handheld computer, a netbook, an ultra-mobile personal computer (UMPC), a mobile internet device (MID), an augmented reality (AR), a virtual reality (VR) device, a robot, a wearable device (Wearable Device), an aircraft (Flight Vehicle), a vehicle-mounted device (VUE), a ship-mounted device, a pedestrian user equipment (PUE), a smart home (home appliances with wireless communication capabilities, such as refrigerators, televisions, washing machines, or furniture), a game console, a personal computer (PC), an ATM, or a self-service machine, or other terminal-side devices. Wearable devices include: smart watches, smart bracelets, smart headphones, smart glasses, smart jewelry (smart bracelets, smart bracelets, smart rings, smart necklaces, smart anklets, smart anklets, etc.), smart wristbands, smart clothing, etc. Among them, the vehicle-mounted device can also be called a vehicle-mounted terminal, a vehicle-mounted controller, a vehicle-mounted module, a vehicle-mounted component, a vehicle-mounted chip or a vehicle-mounted unit, etc. In addition to the above-mentioned terminal devices, it can also be a chip in the terminal, such as a modem chip, a system-on-chip (SoC). It should be noted that the specific type of the terminal 11 is not limited in the embodiment of the present application. The network side device 12 may include an access network device or a core network device, wherein the access network device may also be called a radio access network (RAN) device, a radio access network function or a radio access network unit. The access network device may include a base station, a wireless local area network (WLAN) access point (AS) or a wireless fidelity (WiFi) node, etc.Among them, the base station can be referred to as Node B (NB), Evolved Node B (eNB), the next generation Node B (gNB), New Radio Node B (NR Node B), access point, Relay Base Station (RBS), Serving Base Station (SBS), Base Transceiver Station (BTS), radio base station, radio transceiver, Basic Service Set (BSS), Extended Service Set (ESS), Home Node B (HNB), Home Evolved Node B (home evolved Node B), Transmission Reception Point (TRP) or other appropriate terms in the field. As long as the same technical effect is achieved, the base station is not limited to specific technical vocabulary. It should be noted that in the embodiment of the present application, only the base station in the NR system is used as an example for introduction, and the specific type of the base station is not limited.
[0046] The core network equipment may include but is not limited to at least one of the following: core network node, core network function, mobility management entity (MME), access mobility management function (AMF), session management function (SMF), user plane function (UPF), policy control function (PCF), policy and charging rules function unit (PCRF), edge application service discovery function (EASDF), unified data management (UDM), unified data repository (UDR), home user server (HSS), centralized network configuration (CNC), network storage function (NRF), network exposure function (NEF), local NEF (L-NEF), binding support function (BSF), application function ( It should be noted that in the embodiments of the present application, only the core network device in the NR system is introduced as an example, and the specific type of the core network device is not limited.
[0047] In order to facilitate a better understanding of the embodiments of the present application, the following technical points are first introduced below.
[0048] The Network Data Analytics Function (NWDAF) can be functionally divided into two network elements, as follows:
[0049] Model Training Logical Function (MTLF), used to generate models and perform model training.
[0050] Analytics Logical Function (AnLF) is used to perform inference and generate prediction information or statistical information.
[0051] MTLF and AnLF can be deployed independently as different network element devices, or deployed together in the same network element device, such as NWDAF. In this case, NWDAF can provide both AI model training and model inference functions.
[0052] Among them, the AI model can also be described as an ML model or an analytics model.
[0053] In related technologies, as shown in Figure 2, the process by which a client (consumer Network Function (NF)) obtains AI data analysis results (analytics) from the NWDAF is as follows:
[0054] (11) MTLF collects training data from the training data source device. The MTLF is NWDAF with MTLF function (NWDAF containing MTLF).
[0055] (12). MTLF trains the first model based on the training data.
[0056] (13) Consumer NF sends a request message to AnLF, which is NWDAF containing AnLF.
[0057] (14). AnLF requests the model from MTLF according to the request message.
[0058] (15). MTLF sends the model information and model accuracy to AnLF.
[0059] (16). AnLF determines the source device of the inference input data, the type information of the input data, and the type information of the output data based on the received request message.
[0060] (17).AnLF obtains the reasoning input data corresponding to the task.
[0061] (18). AnLF performs reasoning based on the acquired first model and reasoning input data to obtain reasoning result data.
[0062] (19). AnLF sends the inference result data obtained through inference to the consumer NF.
[0063] (110). AnLF obtains label data corresponding to the inference result data.
[0064] (111).AnLF calculates the first accuracy of the first model based on the inference result data and the label data.
[0065] (112). AnLF sends the first instruction to the consumer NF.
[0066] (113). The consumer NF may perform corresponding operations according to the first instruction.
[0067] (114). AnLF sends a second instruction to MTLF.
[0068] (115).MTLF can perform corresponding operations according to the second instruction.
[0069] (116). When MTLF retrains the model, MTLF sends the trained second model to AnLF.
[0070] In related technologies, some devices involved in intelligent data analysis are unable to obtain the accuracy of the model, resulting in poor results of intelligent data analysis.
[0071] The information acquisition method provided by the embodiments of the present application is described in detail below through some embodiments and their application scenarios in conjunction with the accompanying drawings.
[0072] Referring to FIG3 , FIG3 is a flow chart of an information acquisition method provided in an embodiment of the present application. As shown in FIG3 , the information acquisition method includes the following steps:
[0073] Step 101: A first device obtains first information, where the first information is used to determine the accuracy of a first model.
[0074] The first device may be used to perform inference to generate prediction information or generate statistical information. The first device may be an AnLF network element device, and the AnLF may be a NWDAF containing AnLF. The first model may be an AI model, an ML model, or an analytical model, etc., having prediction or statistical functions.
[0075] Accuracy can also be described as accuracy information, ML accuracy information, ML model accuracy information, or analytics accuracy information. The following descriptions are the same and will not be repeated here.
[0076] In one embodiment, the first device can receive a first request sent by the second device, where the first request is used to request the accuracy of the first model, and the first request carries the first information, so that the first device can obtain the first information from the second device; or, the first device can determine the first information. For example, the first device can determine the first information according to its own needs.
[0077] The first request may also be described as a first request message.
[0078] The accuracy of the first model can also be described as the accuracy corresponding to the first model.
[0079] The first information may be used to indicate information used to obtain the accuracy of the first model. The first information may be used to indicate an accuracy deviation, an allowable error deviation, an allowable error percentage, or an allowable value range used to obtain the accuracy of the first model. The first information may include at least one of the following:
[0080] Information indicating differences in precision;
[0081] Information indicating permissible errors;
[0082] Information indicating the allowed values.
[0083] The accuracy deviation can also be described as the acceptable accuracy deviation, and the allowed error can also be described as the allowed prediction error deviation.
[0084] The first information may be used to indicate information used to obtain the accuracy of the first model, and may also be described as the first information being used to help the first device determine or judge what is a correct prediction.
[0085] It should be noted that determining the accuracy of the first model may include: determining that the inference result of the first model is correct if the difference between the inference result of the first model and the true value corresponding to the inference result satisfies the first information; or determining that the inference result of the first model is incorrect if the difference between the inference result of the first model and the true value corresponding to the inference result does not satisfy the first information. Thus, the first information can be used to transform a regression problem into a classification problem.
[0086] Taking the example of the first information including information for indicating the accuracy difference, if the difference between the inference result of the first model and the true value corresponding to the inference result satisfies the first information, it can be considered that the difference between the inference result of the first model and the true value corresponding to the inference result is less than or equal to the accuracy difference; if the difference between the inference result of the first model and the true value corresponding to the inference result does not satisfy the first information, it can be considered that the difference between the inference result of the first model and the true value corresponding to the inference result is greater than the accuracy difference.
[0087] Taking the example of the first information including information for indicating the allowable error, if the difference between the inference result of the first model and the true value corresponding to the inference result satisfies the first information, it can be considered that the difference between the inference result of the first model and the true value corresponding to the inference result is less than or equal to the allowable error; if the difference between the inference result of the first model and the true value corresponding to the inference result does not satisfy the first information, it can be considered that the difference between the inference result of the first model and the true value corresponding to the inference result is greater than the allowable error.
[0088] Taking the example of the first information including information for indicating allowed values, if the difference between the inference result of the first model and the true value corresponding to the inference result satisfies the first information, it can be considered that the difference between the inference result of the first model and the true value corresponding to the inference result is within the allowed value range; if the difference between the inference result of the first model and the true value corresponding to the inference result does not satisfy the first information, it can be considered that the difference between the inference result of the first model and the true value corresponding to the inference result is not within the allowed value range.
[0089] It should be noted that in the communication network system, some network elements are introduced to perform intelligent data analysis and generate data analysis results (analytics) (or inference data results) of some tasks. These data analysis results can assist devices inside and outside the network in making policy decisions. The purpose is to use artificial intelligence (AI) or machine learning (ML) methods to improve the intelligence level of device policy decisions.
[0090] NWDAF can train AI models or machine learning (ML) models based on training data to obtain a model suitable for a certain AI task. The inference input data of a certain AI task is inferred based on the AI model or ML model to obtain the inference result data corresponding to a specific AI task. The Policy Control Function (PCF) performs intelligent policy control and charging (PCC) based on the inference result data, such as formulating intelligent user retention strategies based on the inference result data of the user's business behavior to improve the user's business experience; or, the Access and Mobility Management Function (AMF) performs intelligent mobility management operations based on the inference result data of a certain AI task, such as intelligently paging users based on the inference result data of the user's mobile trajectory to improve the paging reachability.
[0091] Devices inside and outside the network can make accurate and optimized policy decisions based on AI data analysis results, but this is premised on accurate data analysis results. If the accuracy of the data analysis results is low, they will be provided as erroneous information to devices inside and outside the network for reference, ultimately leading to incorrect policy decisions or inappropriate operations. Therefore, ensuring the accuracy of data analysis results is essential.
[0092] It should be noted that the accuracy requirement information is used to indicate the accuracy that the requested model must meet. This accuracy can refer to the accuracy of the correct prediction, or it can refer to the prediction error value, such as the Mean Absolute Error (MAE). The accuracy requirement information can be a specific value or percentage, or it can include the value / percentage and its corresponding calculation method. For example, the accuracy requirement information: 70%.
[0093] The accuracy of a correct prediction can be calculated as the ratio of the number of correct predictions to the total number of predictions. This is calculated by dividing the number of correct predictions by the total number of predictions. This method is also called classification and is suitable for classification methods, such as determining whether a destination has been reached or whether it is raining. The following formula can be used to calculate accuracy:
[0094] Among them, the number of correct predictions refers to the number of accurate predictions, and all predictions refers to the total number of predictions.
[0095] Alternatively, the predicted error can be the mean squared error between the model's predicted value and the actual value (i.e., the true value). This method, also known as regression, is used to calculate the accuracy of the model's prediction. For example, in navigation software, the difference between the predicted arrival time and the actual arrival time is calculated to determine whether the model's prediction is accurate. The following formula can be used to calculate the predicted error:
[0096] Among them, yi-predicted refers to the model prediction value, y i-truth Refers to the actual value, n refers to the total number of predictions, and i is a positive integer.
[0097] In related technologies, there are multiple methods for calculating accuracy requirements, which complicates network design. Network element devices not only need to know how to calculate the accuracy requirements, but also need to know the calculation method of the accuracy requirements.
[0098] This embodiment unifies the accuracy requirement calculation method into a classification approach. For regression problems, the first information, such as the acceptable accuracy deviation or the allowable prediction error deviation, is sent to convert the regression problem into a classification problem. For example, if the predicted arrival time is within 5 minutes of the actual arrival time, the prediction is considered correct if the accuracy deviation is within 5 minutes; if the error is greater than 5 minutes, the task prediction is considered incorrect.
[0099] For example, when the consumer NF (second device) sends a first request to the AnLF (first device), it carries the accuracy difference for the regression problem; when the AnLF calculates the accuracy, it considers the inference result with an error value less than or equal to the accuracy difference to be correct, and considers the inference result with an error value greater than the accuracy difference to be incorrect.
[0100] In an embodiment of the present application, the first device obtains first information, which is used to determine the accuracy of the first model, and can support the first device to more accurately obtain the accuracy of the first model, thereby improving the effect of intelligent data analysis.
[0101] Optionally, the first device acquiring the first information includes:
[0102] The first device receives a first request sent by the second device, where the first request is used to request the accuracy of the first model, and the first request carries the first information.
[0103] The second device may be a Consumer NF network element device. The first request may also carry at least one of the following:
[0104] Analytics ID information;
[0105] Analytics Filter Information;
[0106] Analytics Target information.
[0107] It should be noted that the first request may be a Nnwdaf_AnalyticsSubscription_Subscribe message.
[0108] In this embodiment, the first device receives a first request sent by the second device, where the first request is used to request the accuracy of the first model. The first request carries the first information, so that the first information can be transmitted by the second device when sending the accuracy requirement, thereby facilitating the first device and the second device to unify the method of determining the accuracy of the model.
[0109] Optionally, the method further includes:
[0110] The first device sends a second request to a third device, where the second request is used to request the first model;
[0111] The second request carries the first information or the second information, the second information is used to determine the accuracy of the requested first model, and the second information is determined based on the first information.
[0112] The second request may also be described as a second request message.
[0113] The third device may be used to generate a model and perform model training. The third device may be a MTLF network element device, where the MTLF is a NWDAF containing MTLF. The first device and the third device may be deployed as separate devices, or the first device and the third device may be deployed as a single device.
[0114] The first device may send the second request to the third device based on the received first request; or, the first device may directly send the second request to the third device. For example, the first device may directly send the second request to the third device according to its own needs.
[0115] In addition, the second request may carry the first information, or the second request may carry the second information.
[0116] The second information may be used to indicate information used to obtain the accuracy of the requested first model. The second information may be used to indicate the accuracy difference, allowable error value, allowable error percentage, or allowable value range used when obtaining the accuracy of the requested first model. The second information may include at least one of the following:
[0117] Information indicating differences in precision;
[0118] Information indicating permissible errors;
[0119] Information indicating the allowed values.
[0120] In addition, the first device may determine the second information based on the first information, and the second information may be the same as or different from the first information. The first information and the second information may be of the same information type, for example, the first information and the second information may both include information indicating a precision difference; or the first information and the second information may be of different information types, for example, the first information may include information indicating a precision difference, and the second information may include information indicating an allowable error.
[0121] If the first information and the second information are of the same information type, the value of the second information may be the same as or different from the value of the first information. For example, if both the first information and the second information include precision difference values, the precision difference value in the second information may be the same as the precision difference value in the first information, or the precision difference value in the second information may be greater than the precision difference value in the first information, or the precision difference value in the second information may be less than the precision difference value in the first information.
[0122] If the first information and the second information are of different information types, the information type of the second information can be converted to the information type of the first information, and the value of the second information after conversion can be the same as or different from the value of the first information. For example, the first information may include information indicating a precision difference, and the second information may include information indicating an allowable error. The second information can be converted to a precision difference, and the converted second information can be greater than, less than, or equal to the first information.
[0123] It should be noted that the second request may be a Nnwdaf_MLModelProvision_Subscribe message or a Nnwdaf_MLModelInfo_Request message.
[0124] The second request may also include at least one of the following:
[0125] Analytics ID, which is used to determine the first model to use;
[0126] Analytics Filter Information.
[0127] In one embodiment, when AnLF sends a second request to MTLF, it carries the accuracy difference for the regression problem; when MTLF calculates the accuracy, it considers the inference result with an error value less than or equal to the accuracy difference to be correct, and considers the inference result with an error value greater than the accuracy difference to be incorrect.
[0128] In one implementation, when the MTLF sends the accuracy information to the AnLF, the accuracy difference value obtained is carried.
[0129] In this embodiment, the first device sends a second request to the third device, where the second request is used to request the first model. The second request carries the first information or the second information, and the second information is used to determine the accuracy of the requested first model, with the second information determined based on the first information. This allows the first device to transmit the first information or the second information when sending a model request, facilitating a unified method for determining model accuracy between the first and third devices.
[0130] Optionally, the method further includes:
[0131] The first device receives the accuracy of the first model and third information sent by the third device, where the third information is used to indicate information used to obtain the accuracy of the first model.
[0132] The third information may be the same as or different from the second information.
[0133] The third information may be used to indicate information used to obtain the accuracy of the first model. The third information may be used to indicate the accuracy difference, allowable error value, allowable error percentage, or allowable value range used to obtain the accuracy of the first model. The third information may include at least one of the following:
[0134] Information indicating differences in precision;
[0135] Information indicating permissible errors;
[0136] Information indicating the allowed values.
[0137] In addition, the information types of the second information and the third information may be the same, for example, the second information and the third information may both include information for indicating the accuracy difference; or, the information types of the second information and the third information may be different, for example, the second information may include information for indicating the accuracy difference, and the third information may include information for indicating the allowable error.
[0138] If the second information and the third information are of the same information type, the value of the third information may be the same as or different from the value of the second information. For example, if both the second information and the third information include precision differences, the precision difference in the third information may be the same as the precision difference in the second information, or the precision difference in the third information may be greater than the precision difference in the second information, or the precision difference in the third information may be less than the precision difference in the second information.
[0139] If the second information and the third information are of different information types, the information type of the third information may be converted to the information type of the second information, and the value of the converted third information may be the same as or different from the value of the second information. For example, the second information may include information indicating a precision difference, and the third information may include information indicating an allowable error. The third information may be converted to a precision difference, and the converted third information may be greater than, less than, or equal to the second information.
[0140] It should be noted that after the first device sends the second request to the third device, the first device may receive the accuracy of the first model and the third information sent by the third device. After receiving the second request, the third device may also send the first model to the first device, so that the first device can receive the first model, the accuracy of the first model, and the third information sent by the third device.
[0141] In this embodiment, the first device receives the first model accuracy and third information sent by the third device, where the third information indicates information used to determine the accuracy of the first model. Thus, the third device can transmit the third information when sending the model accuracy, facilitating the first and third devices to unify their methods for determining model accuracy.
[0142] Optionally, the method further includes:
[0143] The first device determines the accuracy of the first model based on at least one of the first information and fourth information, an inference result of the first model, and a true value corresponding to the inference result, wherein the fourth information is determined based on the first information.
[0144] In one embodiment, the first device determines the accuracy of the first model based on the inference result of the first model, the ground truth corresponding to the inference result, and the first information.
[0145] In one embodiment, the first device determines the accuracy of the first model based on the inference result of the first model, the true value corresponding to the inference result, and the fourth information.
[0146] Among them, the inference results can also be understood as prediction results.
[0147] The fourth information may be used to indicate information used to obtain the accuracy of the first model. The fourth information may be used to indicate the accuracy difference, allowable error value, allowable error percentage, or allowable value range used to obtain the accuracy of the first model. The fourth information may include at least one of the following:
[0148] Information indicating differences in precision;
[0149] Information indicating permissible errors;
[0150] Information indicating the allowed values.
[0151] In addition, the first device may determine the fourth information based on the first information, and the fourth information may be the same as or different from the first information. The first information and the fourth information may be of the same information type, for example, the first information and the fourth information may both include information indicating a precision difference; or the first information and the fourth information may be of different information types, for example, the first information may include information indicating a precision difference, and the fourth information may include information indicating an allowable error.
[0152] If the first information and the fourth information are of the same information type, the value of the fourth information may be the same as or different from the value of the first information. For example, if both the first information and the fourth information include precision difference values, the precision difference value in the fourth information may be the same as the precision difference value in the first information, or the precision difference value in the fourth information may be greater than the precision difference value in the first information, or the precision difference value in the fourth information may be less than the precision difference value in the first information.
[0153] If the first information and the fourth information are of different information types, the fourth information may be converted to the same type as the first information, and the value of the converted fourth information may be the same as or different from the value of the first information. For example, the first information may include information indicating a precision difference, and the fourth information may include information indicating an allowable error. The fourth information may be converted to a precision difference, and the converted fourth information may be greater than, less than, or equal to the first information.
[0154] It should be noted that after the first device obtains the first information, the first device can determine the accuracy of the first model based on at least one of the first information and the fourth information, the inference result of the first model and the true value corresponding to the inference result.
[0155] In this embodiment, the first device determines the accuracy of the first model based on at least one of the first information and the fourth information, an inference result of the first model, and a truth value corresponding to the inference result, wherein the fourth information is determined based on the first information. Thus, the accuracy of the first model can be determined using the same method for determining the accuracy of the acquired first model.
[0156] Optionally, the first device determines the accuracy of the first model based on at least one of the first information and the fourth information, an inference result of the first model, and a true value corresponding to the inference result, including any one of the following:
[0157] If the difference between the inference result of the first model and the true value corresponding to the inference result satisfies the first information, determining that the inference result of the first model is correct; or if the difference between the inference result and the true value corresponding to the inference result does not satisfy the first information, determining that the inference result of the first model is incorrect;
[0158] When the difference between the inference result of the first model and the true value corresponding to the inference result satisfies the fourth information, the inference result of the first model is determined to be correct; or, when the difference between the inference result and the true value corresponding to the inference result does not satisfy the fourth information, the inference result of the first model is determined to be incorrect.
[0159] The difference between the inference result and the true value corresponding to the inference result may be the difference obtained by directly subtracting the inference result from the true value, or may be the Euler distance between the inference result and the true value, or the Euclidean metric (also known as the Euclidean distance), or the Manhattan distance, or the Minkowski distance, or the adjacent space cosine similarity, or the adjusted cosine similarity (Pearson correlation coefficient), or the Jaccard similarity coefficient, etc.
[0160] In one embodiment, the first device determines the accuracy of the first model based on the inference result of the first model, the true value corresponding to the inference result, and the first information, including: when the difference between the inference result of the first model and the true value corresponding to the inference result satisfies the first information, determining that the inference result of the first model is correct; or, when the difference between the inference result of the first model and the true value corresponding to the inference result does not satisfy the first information, determining that the inference result of the first model is incorrect.
[0161] In one embodiment, the first device determines the accuracy of the first model based on the inference result of the first model, the true value corresponding to the inference result, and fourth information, including: when the difference between the inference result of the first model and the true value corresponding to the inference result satisfies the fourth information, determining that the inference result of the first model is correct; or, when the difference between the inference result of the first model and the true value corresponding to the inference result does not satisfy the fourth information, determining that the inference result of the first model is incorrect.
[0162] Optionally, the method further comprises at least one of the following:
[0163] The first device sends at least one of the first information and fourth information and the accuracy of the first model to the second device;
[0164] The first device sends at least one of the first information and fourth information and the accuracy of the first model to a third device.
[0165] In one embodiment, the first device transmits the accuracy of the first model and the first information to the second device. For example, when the first device determines the accuracy of the first model based on an inference result of the first model, a truth value corresponding to the inference result, and the first information, the first device transmits the accuracy of the first model and the first information to the second device. This allows the first device to transmit the first information when transmitting the accuracy of the model, facilitating the first and second devices to unify their methods for determining the accuracy of the model.
[0166] In one embodiment, the first device transmits the accuracy of the first model and the fourth information to the second device. For example, when the first device determines the accuracy of the first model based on the inference result of the first model, the truth value corresponding to the inference result, and the fourth information, the first device transmits the accuracy of the first model and the fourth information to the second device. This allows the first device to transmit the fourth information when transmitting the model accuracy, facilitating the first and second devices to unify their methods for determining model accuracy.
[0167] In one embodiment, the first device transmits the accuracy of the first model and the first information to a third device. For example, when the first device determines the accuracy of the first model based on an inference result of the first model, a truth value corresponding to the inference result, and the first information, the first device transmits the accuracy of the first model and the first information to the third device. This allows the first device to transmit the first information when transmitting the accuracy of the model, facilitating the first and third devices to unify their methods for determining the accuracy of the model.
[0168] In one embodiment, the first device transmits the accuracy of the first model and the fourth information to a third device. For example, when the first device determines the accuracy of the first model based on an inference result of the first model, a truth value corresponding to the inference result, and the fourth information, the first device transmits the accuracy of the first model and the fourth information to the third device. This allows the first device to transmit the fourth information when transmitting the accuracy of the model, facilitating the first and third devices to unify their methods for determining the accuracy of the model.
[0169] In one implementation, when the AnLF sends the accuracy information to the consumer NF, it carries the accuracy difference value obtained.
[0170] For example, if the accuracy difference sent by the consumer NF to AnLF is 5 minutes, AnLF can use 2 minutes to calculate the accuracy and inform the consumer NF of the accuracy difference used to obtain the accuracy when sending the accuracy information.
[0171] Optionally, the method further includes:
[0172] The first device receives the second model and the first information sent by the third device; or
[0173] The first device receives the second model and fifth information sent by the third device, where the fifth information is used to indicate information used to determine the accuracy of the second model;
[0174] The second model is a model obtained by retraining the first model.
[0175] In one embodiment, when the third device uses the first information to determine the accuracy of the second model, the third device sends the second model and the first information to the first device. The first device then receives the second model and the first information sent by the third device. This allows the third device to transmit the first information when sending the retrained model, facilitating a unified approach for determining model accuracy between the first and third devices.
[0176] In one embodiment, when the third device uses the fifth information to determine the accuracy of the second model, the third device sends the second model and the fifth information to the first device. The first device then receives the second model and the fifth information sent by the third device. This allows the third device to transmit the fifth information when sending the retrained model, facilitating a unified approach for determining model accuracy between the first and third devices.
[0177] The fifth information may be used to indicate information used to determine the accuracy of the second model. The fifth information may be used to indicate the accuracy difference, allowable error value, allowable error percentage, or allowable value range used to determine the accuracy of the second model. The fifth information may include at least one of the following:
[0178] Information indicating differences in precision;
[0179] Information indicating permissible errors;
[0180] Information indicating the allowed values.
[0181] It should be noted that after the first device sends at least one of the first information and the fourth information and the accuracy of the first model to the third device, the first device receives the second model and the first information sent by the third device; or, the first device receives the second model and the fifth information sent by the third device, and the fifth information is used to indicate the information used to determine the accuracy of the second model.
[0182] Optionally, the first information, the second information, the third information, the fourth information, or the fifth information includes at least one of the following:
[0183] Information indicating differences in precision;
[0184] Information indicating permissible errors;
[0185] Information indicating the allowed values.
[0186] The allowable error may include an allowable error value or an allowable error percentage. The allowable value may refer to a specific value or a range of values.
[0187] Optionally, the first model includes at least one of the following:
[0188] Artificial intelligence AI model; machine learning ML model; analytical model.
[0189] Optionally, the first device is an analysis logic function; or,
[0190] The second device is a consumer network function; or,
[0191] The third device is a model training logic function.
[0192] Referring to FIG4 , FIG4 is a flow chart of an information acquisition method provided in an embodiment of the present application. As shown in FIG4 , the information acquisition method includes the following steps:
[0193] Step 201: The second device sends first information to the first device, where the first information is used to determine the accuracy of the first model.
[0194] Optionally, the first information is carried in a first request, and the first request is used to request the accuracy of the first model; or, the first request is used to request the first device to perform inference.
[0195] Optionally, the method further includes:
[0196] The second device receives at least one of the first information and fourth information sent by the first device and the accuracy of the first model, wherein the fourth information is determined based on the first information.
[0197] Optionally, when the difference between the inference result of the first model and the true value corresponding to the inference result satisfies the first information, the inference result of the first model is correct; or, when the difference between the inference result and the true value corresponding to the inference result does not satisfy the first information, the inference result of the first model is wrong; or
[0198] When the difference between the inference result of the first model and the true value corresponding to the inference result satisfies the fourth information, the inference result of the first model is correct; or, when the difference between the inference result and the true value corresponding to the inference result does not satisfy the fourth information, the inference result of the first model is incorrect.
[0199] Optionally, the first information or the fourth information includes at least one of the following:
[0200] Information indicating differences in precision;
[0201] Information indicating permissible errors;
[0202] Information indicating the allowed values.
[0203] It should be noted that this embodiment is an implementation of the second device corresponding to the embodiment shown in Figure 3. Its specific implementation can refer to the relevant description of the embodiment shown in Figure 3. To avoid repeated description, this embodiment will not be repeated.
[0204] Referring to FIG5 , FIG5 is a flow chart of an information acquisition method provided in an embodiment of the present application. As shown in FIG5 , the information acquisition method includes the following steps:
[0205] Step 301: A third device receives first information or second information sent by a first device, where the first information or second information is used to determine the accuracy of a requested first model.
[0206] Optionally, the first information or the second information is carried in a second request, and the second request is used to request the first model.
[0207] Optionally, the method further includes:
[0208] The third device sends the accuracy of the first model and third information to the first device, where the third information is used to indicate information used to obtain the accuracy of the first model.
[0209] Optionally, the method further includes:
[0210] The third device receives at least one of the first information and fourth information sent by the first device and the accuracy of the first model, wherein the fourth information is determined based on the first information.
[0211] Optionally, if the difference between the inference result of the first model and the true value corresponding to the inference result satisfies the first information, the inference result of the first model is correct; or, if the difference between the inference result and the true value corresponding to the inference result does not satisfy the first information, the inference result of the first model is wrong; or,
[0212] When the difference between the inference result of the first model and the true value corresponding to the inference result satisfies the fourth information, the inference result of the first model is correct; or, when the difference between the inference result and the true value corresponding to the inference result does not satisfy the fourth information, the inference result of the first model is incorrect.
[0213] Optionally, the method further includes:
[0214] The third device sends the second model and the first information to the first device; or
[0215] The third device sends the second model and fifth information to the first device, where the fifth information is used to indicate information used to determine the accuracy of the second model;
[0216] The second model is a model obtained by retraining the first model.
[0217] Optionally, the first information, the second information, the third information, the fourth information, or the fifth information includes at least one of the following:
[0218] Information indicating differences in precision;
[0219] Information indicating permissible errors;
[0220] Information indicating the allowed values.
[0221] It should be noted that this embodiment is an implementation of the third device corresponding to the embodiment shown in Figure 3. Its specific implementation can refer to the relevant description of the embodiment shown in Figure 3. To avoid repeated description, this embodiment will not be repeated.
[0222] In the related art, there are many ways to determine the accuracy of the model used for intelligent data analysis. For example, the accuracy of the model can be represented by the ratio of the number of accurate model predictions to the total number of model predictions, or the mean square error between the model prediction value and the true value can be used to represent the accuracy of the model. Due to the inconsistent methods for determining the accuracy of the model, multiple devices participating in the intelligent data analysis cannot accurately know the accuracy of the model, resulting in poor intelligent data analysis results. This embodiment can support the first device, the second device, and the third device to accurately obtain the accuracy of the model through the interaction between the first device, the second device, and the third device. When the first device, the second device, and the third device participate in the intelligent data analysis, the first information, the second information, the third information, the fourth information, or the fifth device can support the devices participating in the intelligent data analysis to unify the method for determining the accuracy of the model, thereby improving the effect of intelligent data analysis.
[0223] In one embodiment, device A (Consumer NF or AnLF) sends first information corresponding to a first model to device B (AnLF or MTLF), where the first information is used to obtain accuracy information corresponding to the first model; and device A receives the accuracy information sent by device B.
[0224] The first information includes at least one of the following: indication information indicating an accuracy difference, an allowable error value, an allowable error percentage, or an allowable error range.
[0225] In addition, the accuracy information received by device A from device B may include:
[0226] The second information sent by receiving device B is used to indicate the accuracy difference, the allowable error value, the allowable error percentage or the allowable error range used to obtain the accuracy information.
[0227] The second information may include at least one of the following: indication information indicating an accuracy difference, an allowable error value, an allowable error percentage, or an allowable error range.
[0228] It should be noted that device A is a Consumer NF and device B is an AnLF; or, device A is an AnLF and device B is a MTLF. When device A is a Consumer NF and device B is an AnLF, device A sends a first message to device B. The first message is used to obtain accuracy information corresponding to the first model, and the first message includes the first information. When device A is an AnLF and device B is an MTLF, device A sends a second message to device B. The second message is used to obtain the first model, and the second message includes the first information.
[0229] As shown in FIG6 , the information acquisition method provided in the embodiment of the present application is described through a specific embodiment:
[0230] (21) The MTLF (i.e., the third device) collects training data from the training data source device. The MTLF is a NWDAF with MTLF function (NWDAF containing MTLF).
[0231] (22). MTLF trains the first model based on the training data.
[0232] (23) The Consumer NF (i.e., the second device) sends a first request to the AnLF (i.e., the first device). The AnLF is a NWDAF containing AnLF. The first request is used to request the AnLF to perform analysis or reasoning on a specific task. The first request includes:
[0233] Analytics ID information;
[0234] Analytics Filter Information;
[0235] Analytics Target information;
[0236] The first information may be used to indicate an accuracy difference, an allowable error value, an allowable error percentage, an allowable error range, or the like.
[0237] The first request may be a Nnwdaf_AnalyticsSubscription_Subscribe message or a Nnwdaf_AnalyticsInfo_Request message.
[0238] (24) AnLF requests the model (Model) from MTLF through a second request based on the first request;
[0239] The second request may be a Nnwdaf_MLModelProvision_Subscribe message or a Nnwdaf_MLModelInfo_Request message.
[0240] The second request includes:
[0241] Analytics ID, which is used to determine the first model to use;
[0242] Analytics Filter Information;
[0243] The second information is used to indicate the accuracy difference, the allowable error value, the allowable error percentage or the allowable value range, etc.
[0244] It should be noted that the second information may be the same as or different from the first information.
[0245] (25) MTLF sends the information of the first model and the accuracy of the first model to AnLF. The message sent may be Nnwdaf_MLModelProvision_Notify or Nnwdaf_MLModelInfo_Request Response message.
[0246] It should be noted that steps (21) and (22) can also be performed after step (24).
[0247] Optionally, the MTLF sends third information corresponding to the accuracy of the first model to the AnLF, where the third information is used to indicate the precision difference, allowable error value, allowable error percentage or allowable value range used when obtaining the accuracy.
[0248] It should be noted that the third information may be the same as or different from the second information.
[0249] (26). AnLF determines the source device of the inference input data, the type information of the input data, the type information of the output data, etc. based on the received first request.
[0250] (27). AnLF obtains the reasoning input data corresponding to the task. Specifically, AnLF can send a request message for the reasoning input data to the source device of the reasoning input data determined in step (6) to collect the reasoning input data corresponding to the task.
[0251] (28). AnLF performs reasoning based on the acquired first model and reasoning input data to obtain reasoning result data.
[0252] For example, AnLF performs inference on inference input data (such as User Equipment (UE) ID, time, UE current service status, and other numerical values) based on the first model corresponding to analytics ID = UE mobility, and obtains inference result data as output data of UE location.
[0253] AnLF can obtain multiple output result values by executing one inference calculation process; or, AnLF can obtain multiple inference output result values by executing multiple inferences.
[0254] (29). AnLF sends the inference result data obtained through inference to the consumer NF.
[0255] The inference result data can be used to inform the consumer NF of the statistical or predicted values obtained through inference by the model corresponding to the analytics ID, which can be used to assist the consumer NF in executing corresponding policy decisions. For example, the statistical or predicted value corresponding to UE mobility can be used to assist the AMF in user paging optimization.
[0256] (210) AnLF obtains label data corresponding to the inference result data. The message sent by AnLF to request to obtain label data may be an Nnf_EventExposure_Subscribe message.
[0257] Specifically, AnLF can determine which label data to feed back to the source device of the label data determined in step (6) by sending a request message for label data, which includes type information of the label data, object information corresponding to the label data, time information (such as timestamp, time period), etc.
[0258] Among them, the label data can also be described as the ground truth.
[0259] (211).AnLF calculates the first accuracy of the first model based on the inference result data and the label data (i.e., the true value corresponding to the inference result data).
[0260] When calculating accuracy, AnLF considers the results whose error values conform to the first information to be correct, for example, the results whose error values are less than or equal to the first information; and considers the results whose error values do not conform to the first information, for example, the results whose error values are greater than the first information to be incorrect.
[0261] (212). When AnLF needs to send the first accuracy to the consumer NF (for example, when it is determined that the first accuracy does not meet the accuracy requirement or the accuracy has decreased, or when the periodic sending time arrives), AnLF sends a first indication to the consumer NF. The first indication is used to notify the consumer NF that the accuracy of the first model does not meet the accuracy requirement or the accuracy of the first model has decreased, or the first indication is used to notify the consumer NF of the accuracy of the first model.
[0262] Optionally, the AnLF further sends fourth information corresponding to the first accuracy to the consumer NF, where the fourth information is used to indicate the accuracy difference, allowable error value, allowable error percentage or allowable value range used when obtaining the first accuracy.
[0263] It should be noted that the fourth information may be the same as or different from the first information.
[0264] The first accuracy and the fourth information may be carried in a Nnwdaf_AnalyticsSubscription_Notify or Nnwdaf_AnalyticsInfo_Request response message.
[0265] (213). The consumer NF may perform corresponding operations based on the accuracy of the first model, for example, stop using the inference result data corresponding to the task.
[0266] (214). When AnLF needs to send the first accuracy to MTLF (for example, when it is determined that the first accuracy does not meet the accuracy requirement or the accuracy has decreased, or when the periodic sending time arrives), AnLF sends a second indication to MTLF. The second indication is used to notify MTLF that the accuracy of the first model does not meet the accuracy requirement or the accuracy of the first model has decreased, or the second indication is used to notify the consumer NF of the accuracy of the first model.
[0267] The second indication may be the same as or different from the first indication.
[0268] Optionally, when AnLF needs to send the first accuracy to MTLF, it can send fourth information corresponding to the first accuracy to MTLF, where the fourth information is used to indicate the accuracy difference, allowable error value, allowable error percentage or allowable value range used when obtaining the first accuracy.
[0269] It should be noted that the fourth information may be the same as or different from the second information or the third information.
[0270] (215).MTLF can perform corresponding operations based on the accuracy of the first model, for example, retraining the model. During retraining, the fourth information can be used for training, that is, the accuracy of the model during training is determined using the fourth information.
[0271] (216). When MTLF retrains the model, MTLF sends the trained second model to AnLF.
[0272] Optionally, MTLF sends fifth information corresponding to the accuracy of the second model to AnLF, where the fifth information is used to indicate the accuracy difference, allowable error value, allowable error percentage or allowable value range used when obtaining the accuracy of the second model.
[0273] The embodiments of the present application can simplify the evaluation model.
[0274] The information acquisition method provided in the embodiment of the present application can be executed by an information acquisition device. In the embodiment of the present application, the information acquisition device performing the information acquisition method is taken as an example to illustrate the information acquisition device provided in the embodiment of the present application.
[0275] Please refer to FIG. 7 , which is a structural diagram of an information acquisition device provided in an embodiment of the present application. The first device includes the information acquisition device. As shown in FIG. 7 , the information acquisition device 400 includes:
[0276] The acquisition module 401 is configured to acquire first information, where the first information is used to determine the accuracy of the first model.
[0277] Optionally, the acquisition module is specifically configured to:
[0278] A first request sent by a second device is received, where the first request is used to request the accuracy of the first model, and the first request carries the first information.
[0279] Optionally, the device further comprises:
[0280] A first sending module is configured to send a second request to a third device, where the second request is used to request the first model;
[0281] The second request carries the first information or the second information, the second information is used to determine the accuracy of the requested first model, and the second information is determined based on the first information.
[0282] Optionally, the device further comprises:
[0283] The first receiving module is used to receive the accuracy of the first model and third information sent by the third device, where the third information is used to indicate information used to obtain the accuracy of the first model.
[0284] Optionally, the device further comprises:
[0285] A determination module is used to determine the accuracy of the first model based on at least one of the first information and the fourth information, the inference result of the first model and the true value corresponding to the inference result, wherein the fourth information is determined based on the first information.
[0286] Optionally, the determining module is specifically used for any of the following:
[0287] If the difference between the inference result of the first model and the true value corresponding to the inference result satisfies the first information, determining that the inference result of the first model is correct; or if the difference between the inference result and the true value corresponding to the inference result does not satisfy the first information, determining that the inference result of the first model is incorrect;
[0288] When the difference between the inference result of the first model and the true value corresponding to the inference result satisfies the fourth information, the inference result of the first model is determined to be correct; or, when the difference between the inference result and the true value corresponding to the inference result does not satisfy the fourth information, the inference result of the first model is determined to be incorrect.
[0289] Optionally, the apparatus further includes a second sending module, wherein the second sending module is configured to perform at least one of the following:
[0290] sending at least one of the first information and the fourth information and the accuracy of the first model to a second device;
[0291] At least one of the first information and the fourth information and the accuracy of the first model are sent to a third device.
[0292] Optionally, the device further includes a second receiving module, configured to:
[0293] receiving a second model and the first information sent by a third device; or,
[0294] receiving a second model and fifth information sent by a third device, where the fifth information is used to indicate information used to determine the accuracy of the second model;
[0295] The second model is a model obtained by retraining the first model.
[0296] Optionally, the first information, the second information, the third information, the fourth information, or the fifth information includes at least one of the following:
[0297] Information indicating differences in precision;
[0298] Information indicating permissible errors;
[0299] Information indicating the allowed values.
[0300] Optionally, the first model includes at least one of the following:
[0301] Artificial intelligence AI model; machine learning ML model; analytical model.
[0302] Optionally, the first device is an analysis logic function; or,
[0303] The second device is a consumer network function; or,
[0304] The third device is a model training logic function.
[0305] The information acquisition device in the embodiments of the present application can be an electronic device, such as an electronic device with an operating system, or a component in an electronic device, such as an integrated circuit or chip. The electronic device can be a terminal, or it can be other devices other than a terminal. For example, the terminal can include but is not limited to the types of terminal 11 listed above, and other devices can be servers, network attached storage (NAS), etc., which are not specifically limited in the embodiments of the present application.
[0306] The information acquisition device provided in the embodiment of the present application can implement each process implemented in the method embodiment of Figure 3 and achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0307] The information acquisition method provided in the embodiment of the present application can be executed by an information acquisition device. In the embodiment of the present application, the information acquisition device performing the information acquisition method is taken as an example to illustrate the information acquisition device provided in the embodiment of the present application.
[0308] Please refer to FIG8 , which is a structural diagram of an information acquisition device provided in an embodiment of the present application. The second device includes the information acquisition device. As shown in FIG8 , the information acquisition device 500 includes:
[0309] The sending module 501 is configured to send first information to a first device, where the first information is used to determine the accuracy of the first model.
[0310] Optionally, the first information is carried in a first request, and the first request is used to request the accuracy of the first model; or, the first request is used to request the first device to perform inference.
[0311] Optionally, the device further comprises:
[0312] A receiving module is used to receive at least one of the first information and fourth information sent by the first device and the accuracy of the first model, wherein the fourth information is determined based on the first information.
[0313] Optionally, when the difference between the inference result of the first model and the true value corresponding to the inference result satisfies the first information, the inference result of the first model is correct; or, when the difference between the inference result and the true value corresponding to the inference result does not satisfy the first information, the inference result of the first model is wrong; or
[0314] When the difference between the inference result of the first model and the true value corresponding to the inference result satisfies the fourth information, the inference result of the first model is correct; or, when the difference between the inference result and the true value corresponding to the inference result does not satisfy the fourth information, the inference result of the first model is incorrect.
[0315] Optionally, the first information or the fourth information includes at least one of the following:
[0316] Information indicating differences in precision;
[0317] Information indicating permissible errors;
[0318] Information indicating the allowed values.
[0319] The information acquisition device in the embodiments of the present application can be an electronic device, such as an electronic device with an operating system, or a component in an electronic device, such as an integrated circuit or chip. The electronic device can be a terminal, or it can be other devices other than a terminal. For example, the terminal can include but is not limited to the types of terminal 11 listed above, and other devices can be servers, network attached storage (NAS), etc., which are not specifically limited in the embodiments of the present application.
[0320] The information acquisition device provided in the embodiment of the present application can implement each process implemented in the method embodiment of Figure 4 and achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0321] The information acquisition method provided in the embodiment of the present application can be executed by an information acquisition device. In the embodiment of the present application, the information acquisition device performing the information acquisition method is taken as an example to illustrate the information acquisition device provided in the embodiment of the present application.
[0322] Please refer to FIG9 , which is a structural diagram of an information acquisition device provided in an embodiment of the present application. The third device includes the information acquisition device. As shown in FIG9 , the information acquisition device 600 includes:
[0323] The first receiving module 601 is configured to receive first information or second information sent by a first device, where the first information or second information is used to determine the accuracy of the requested first model.
[0324] Optionally, the first information or the second information is carried in a second request, and the second request is used to request the first model.
[0325] Optionally, the device further comprises:
[0326] The first sending module is used to send the accuracy of the first model and third information to the first device, where the third information is used to indicate information used to obtain the accuracy of the first model.
[0327] Optionally, the device further comprises:
[0328] The second receiving module is used to receive at least one of the first information and fourth information sent by the first device and the accuracy of the first model, wherein the fourth information is determined based on the first information.
[0329] Optionally, if the difference between the inference result of the first model and the true value corresponding to the inference result satisfies the first information, the inference result of the first model is correct; or, if the difference between the inference result and the true value corresponding to the inference result does not satisfy the first information, the inference result of the first model is wrong; or,
[0330] When the difference between the inference result of the first model and the true value corresponding to the inference result satisfies the fourth information, the inference result of the first model is correct; or, when the difference between the inference result and the true value corresponding to the inference result does not satisfy the fourth information, the inference result of the first model is incorrect.
[0331] Optionally, the device further includes a second sending module, configured to:
[0332] sending a second model and the first information to the first device; or
[0333] sending a second model and fifth information to the first device, where the fifth information is used to indicate information used to determine the accuracy of the second model;
[0334] The second model is a model obtained by retraining the first model.
[0335] Optionally, the first information, the second information, the third information, the fourth information, or the fifth information includes at least one of the following:
[0336] Information indicating differences in precision;
[0337] Information indicating permissible errors;
[0338] Information indicating the allowed values.
[0339] The information acquisition device in the embodiments of the present application can be an electronic device, such as an electronic device with an operating system, or a component in an electronic device, such as an integrated circuit or chip. The electronic device can be a terminal, or it can be other devices other than a terminal. For example, the terminal can include but is not limited to the types of terminal 11 listed above, and other devices can be servers, network attached storage (NAS), etc., which are not specifically limited in the embodiments of the present application.
[0340] The information acquisition device provided in the embodiment of the present application can implement each process implemented in the method embodiment of Figure 5 and achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0341] Optionally, as shown in Figure 10, an embodiment of the present application further provides a communication device 700, including a processor 701 and a memory 702, and the memory 702 stores a program or instruction that can be run on the processor 701. For example, when the communication device 700 is a first device, when the program or instruction is executed by the processor 701, it implements the various steps of the embodiment of the information acquisition method applied to the first device, and can achieve the same technical effect. To avoid repetition, it is not repeated here. When the communication device 700 is a second device, when the program or instruction is executed by the processor 701, it implements the various steps of the embodiment of the information acquisition method applied to the second device, and can achieve the same technical effect. To avoid repetition, it is not repeated here. When the communication device 700 is a third device, when the program or instruction is executed by the processor 701, it implements the various steps of the embodiment of the information acquisition method applied to the third device, and can achieve the same technical effect. To avoid repetition, it is not repeated here.
[0342] An embodiment of the present application further provides a first device, including a processor and a communication interface, wherein the processor is used to: obtain first information, and the first information is used to determine the accuracy of a first model.
[0343] An embodiment of the present application also provides a second device, including a processor and a communication interface, wherein the communication interface is used to: send first information to the first device, and the first information is used to determine the accuracy of the first model.
[0344] An embodiment of the present application also provides a third device, including a processor and a communication interface, wherein the communication interface is used to: receive first information or second information sent by the first device, and the first information or second information is used to determine the accuracy of the requested first model.
[0345] The first device, the second device or the third device in the embodiment of the present application may be a terminal. Specifically, FIG11 is a schematic diagram of the hardware structure of a terminal implementing the embodiment of the present application.
[0346] The terminal 800 includes but is not limited to: a radio frequency unit 801, a network module 802, an audio output unit 803, an input unit 804, a sensor 805, a display unit 806, a user input unit 807, an interface unit 808, a memory 809 and at least some of the components of the processor 810.
[0347] Those skilled in the art will appreciate that the terminal 800 may also include a power supply (such as a battery) to power various components. The power supply may be logically connected to the processor 810 through a power management system, thereby implementing functions such as charging, discharging, and power consumption management through the power management system. The terminal structure shown in Figure 11 does not constitute a limitation of the terminal. The terminal may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be described in detail here.
[0348] It should be understood that in an embodiment of the present application, the input unit 804 may include a graphics processing unit (GPU) 8041 and a microphone 8042, and the graphics processor 8041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 806 may include a display panel 8061, and the display panel 8061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 807 includes a touch panel 8071 and at least one of other input devices 8072. The touch panel 8071 is also called a touch screen. The touch panel 8071 may include two parts: a touch detection device and a touch controller. Other input devices 8072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.
[0349] In the embodiment of the present application, after receiving downlink data from a network-side device, the radio frequency unit 801 may transmit the data to the processor 810 for processing. Furthermore, the radio frequency unit 801 may send uplink data to the network-side device. Typically, the radio frequency unit 801 includes, but is not limited to, an antenna, an amplifier, a transceiver, a coupler, a low-noise amplifier, a duplexer, and the like.
[0350] The memory 809 can be used to store software programs or instructions and various data. The memory 809 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, applications or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 809 may include a volatile memory or a non-volatile memory, or the memory 809 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DRRAM). The memory 809 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.
[0351] Processor 810 may include one or more processing units. Optionally, processor 810 integrates an application processor and a modem processor. The application processor primarily handles operations related to the operating system, user interface, and application programs, while the modem processor primarily processes wireless communication signals, such as a baseband processor. It is understood that the modem processor may not be integrated into processor 810.
[0352] When the terminal is the first device:
[0353] The processor 810 is configured to:
[0354] First information is obtained, where the first information is used to determine the accuracy of the first model.
[0355] Optionally, the radio frequency unit 801 is configured to:
[0356] A first request sent by a second device is received, where the first request is used to request the accuracy of the first model, and the first request carries the first information.
[0357] Optionally, the radio frequency unit 801 is configured to: send a second request to a third device, where the second request is used to request the first model;
[0358] The second request carries the first information or the second information, the second information is used to determine the accuracy of the requested first model, and the second information is determined based on the first information.
[0359] Optionally, the radio frequency unit 801 is configured to:
[0360] The accuracy of the first model and third information sent by the third device are received, where the third information is used to indicate information used to obtain the accuracy of the first model.
[0361] Optionally, the processor 810 is used to determine the accuracy of the first model based on at least one of the first information and the fourth information, the inference result of the first model, and the true value corresponding to the inference result, wherein the fourth information is determined based on the first information.
[0362] Optionally, the processor 810 is specifically configured to perform any one of the following:
[0363] If the difference between the inference result of the first model and the true value corresponding to the inference result satisfies the first information, determining that the inference result of the first model is correct; or if the difference between the inference result and the true value corresponding to the inference result does not satisfy the first information, determining that the inference result of the first model is incorrect;
[0364] When the difference between the inference result of the first model and the true value corresponding to the inference result satisfies the fourth information, the inference result of the first model is determined to be correct; or, when the difference between the inference result and the true value corresponding to the inference result does not satisfy the fourth information, the inference result of the first model is determined to be incorrect.
[0365] Optionally, the radio frequency unit 801 is used for at least one of the following:
[0366] sending at least one of the first information and the fourth information and the accuracy of the first model to a second device;
[0367] At least one of the first information and the fourth information and the accuracy of the first model are sent to a third device.
[0368] Optionally, the radio frequency unit 801 is further configured to:
[0369] receiving a second model and the first information sent by a third device; or,
[0370] receiving a second model and fifth information sent by a third device, where the fifth information is used to indicate information used to determine the accuracy of the second model;
[0371] The second model is a model obtained by retraining the first model.
[0372] Optionally, the first information, the second information, the third information, the fourth information, or the fifth information includes at least one of the following:
[0373] Information indicating differences in precision;
[0374] Information indicating permissible errors;
[0375] Information indicating the allowed values.
[0376] Optionally, the first model includes at least one of the following:
[0377] Artificial intelligence AI model; machine learning ML model; analytical model.
[0378] Optionally, the first device is an analysis logic function; or,
[0379] The second device is a consumer network function; or,
[0380] The third device is a model training logic function.
[0381] When the terminal is the second device:
[0382] The radio frequency unit 801 is configured to send first information to the first device, where the first information is used to determine the accuracy of the first model.
[0383] Optionally, the first information is carried in a first request, and the first request is used to request the accuracy of the first model; or, the first request is used to request the first device to perform inference.
[0384] Optionally, the radio frequency unit 801 is further used to: receive at least one of the first information and fourth information sent by the first device and the accuracy of the first model, wherein the fourth information is determined based on the first information.
[0385] Optionally, when the difference between the inference result of the first model and the true value corresponding to the inference result satisfies the first information, the inference result of the first model is correct; or, when the difference between the inference result and the true value corresponding to the inference result does not satisfy the first information, the inference result of the first model is wrong; or
[0386] When the difference between the inference result of the first model and the true value corresponding to the inference result satisfies the fourth information, the inference result of the first model is correct; or, when the difference between the inference result and the true value corresponding to the inference result does not satisfy the fourth information, the inference result of the first model is incorrect.
[0387] Optionally, the first information or the fourth information includes at least one of the following:
[0388] Information indicating differences in precision;
[0389] Information indicating permissible errors;
[0390] Information indicating the allowed values.
[0391] When the terminal is a third device:
[0392] The radio frequency unit 801 is configured to receive first information or second information sent by a first device, where the first information or second information is used to determine the accuracy of the requested first model.
[0393] Optionally, the first information or the second information is carried in a second request, and the second request is used to request the first model.
[0394] Optionally, the radio frequency unit 801 is further used to: send the accuracy of the first model and third information to the first device, where the third information is used to indicate information used to obtain the accuracy of the first model.
[0395] Optionally, the radio frequency unit 801 is further used to: receive at least one of the first information and fourth information sent by the first device and the accuracy of the first model, wherein the fourth information is determined based on the first information.
[0396] Optionally, if the difference between the inference result of the first model and the true value corresponding to the inference result satisfies the first information, the inference result of the first model is correct; or, if the difference between the inference result and the true value corresponding to the inference result does not satisfy the first information, the inference result of the first model is wrong; or,
[0397] When the difference between the inference result of the first model and the true value corresponding to the inference result satisfies the fourth information, the inference result of the first model is correct; or, when the difference between the inference result and the true value corresponding to the inference result does not satisfy the fourth information, the inference result of the first model is incorrect.
[0398] Optionally, the radio frequency unit 801 is further configured to:
[0399] sending a second model and the first information to the first device; or
[0400] sending a second model and fifth information to the first device, where the fifth information is used to indicate information used to determine the accuracy of the second model;
[0401] The second model is a model obtained by retraining the first model.
[0402] Optionally, the first information, the second information, the third information, the fourth information, or the fifth information includes at least one of the following:
[0403] Information indicating differences in precision;
[0404] Information indicating permissible errors;
[0405] Information indicating the allowed values.
[0406] The embodiments of the present application can improve the effect of intelligent data analysis.
[0407] It can be understood that the implementation process of each implementation method mentioned in this embodiment can refer to the relevant description of the method embodiment Figure 3 or Figure 4 or Figure 5, and achieve the same or corresponding technical effects. To avoid repetition, it will not be repeated here.
[0408] Specifically, the terminal of the embodiment of the present application also includes: instructions or programs stored in the memory 809 and executable on the processor 810. The processor 810 calls the instructions or programs in the memory 809 to execute the methods executed by the modules shown in FIG. 7 , FIG. 8 or FIG. 9 , and achieves the same technical effect. To avoid repetition, it will not be described here.
[0409] The present application also provides a network-side device, including a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is configured to execute a program or instruction to implement the steps of the method embodiments shown in Figures 3, 4, and 5. This network-side device embodiment corresponds to the aforementioned method embodiment, and each implementation process and implementation method of the aforementioned method embodiment is applicable to this network-side device embodiment and can achieve the same technical effects.
[0410] Specifically, an embodiment of the present application further provides a network-side device. The network-side device can be a first device, a second device, or a third device. As shown in Figure 12, the network-side device 900 includes: an antenna 901, a radio frequency device 902, a baseband device 903, a processor 904, and a memory 905. The antenna 901 is connected to the radio frequency device 902. In the uplink direction, the radio frequency device 902 receives information via the antenna 901 and sends the received information to the baseband device 903 for processing. In the downlink direction, the baseband device 903 processes the information to be sent and sends it to the radio frequency device 902. The radio frequency device 902 processes the received information and sends it through the antenna 901.
[0411] The method executed by the network-side device in the above embodiment may be implemented in the baseband device 903 , which includes a baseband processor.
[0412] The baseband device 903 may include, for example, at least one baseband board, on which multiple chips are arranged, as shown in Figure 12, one of which is a baseband processor, for example, which is connected to the memory 905 through a bus interface to call the program in the memory 905 and execute the network device operations shown in the above method embodiment.
[0413] The network side device may further include a network interface 906, which is, for example, a Common Public Radio Interface (CPRI).
[0414] Specifically, the network side device 900 of an embodiment of the present invention also includes: instructions or programs stored in the memory 905 and executable on the processor 904. The processor 904 calls the instructions or programs in the memory 905 to execute the methods executed by the modules shown in FIG. 7 , FIG. 8 or FIG. 9 , and achieves the same technical effect. To avoid repetition, they will not be elaborated here.
[0415] Specifically, the embodiment of the present application further provides a network-side device. As shown in FIG13 , the network-side device 1000 includes a processor 1001, a network interface 1002, and a memory 1003. The network interface 1002 is, for example, a common public radio interface (CPRI).
[0416] Specifically, the network side device 1000 of an embodiment of the present invention also includes: instructions or programs stored in the memory 1003 and executable on the processor 1001. The processor 1001 calls the instructions or programs in the memory 1003 to execute the methods executed by the modules shown in FIG. 7 , FIG. 8 or FIG. 9 , and achieves the same technical effect. To avoid repetition, it will not be described here.
[0417] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned information acquisition method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
[0418] The processor is the processor in the terminal described in the above embodiment. The readable storage medium includes a computer-readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk. In some examples, the readable storage medium may be a non-transitory readable storage medium.
[0419] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned information acquisition method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0420] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.
[0421] An embodiment of the present application further provides a computer program / program product, which is stored in a storage medium. The computer program / program product is executed by at least one processor to implement the various processes of the above-mentioned information acquisition method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0422] An embodiment of the present application also provides an information acquisition system, including: a first device, a second device and a third device, wherein the first device can be used to execute the steps of the information acquisition method applied to the first device as described above, the second device can be used to execute the steps of the information acquisition method applied to the second device as described above, and the third device can be used to execute the steps of the information acquisition method applied to the third device as described above.
[0423] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the opposite order according to the functions involved. For example, the described method may be performed in an order different from that described, and various steps may also be added, omitted or combined. In addition, the features described with reference to certain examples may be combined in other examples.
[0424] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of a computer software product plus a necessary general-purpose hardware platform, or of course, by hardware. The computer software product is stored in a storage medium (such as ROM, RAM, magnetic disk, optical disk, etc.) and includes a number of instructions for enabling a terminal or network-side device to execute the methods described in each embodiment of the present application.
[0425] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms of implementation methods without departing from the purpose of this application and the scope of protection of the claims. These implementation methods are all within the protection of this application.
Claims
1. A method for obtaining information, comprising: The first device acquires first information, where the first information is used to determine the accuracy of the first model.
2. The method according to claim 1, wherein: The first device acquires first information, including: The first device receives a first request sent by the second device, where the first request is used to request the accuracy of the first model, and the first request carries the first information.
3. The method according to claim 1 or 2, further comprising: The first device sends a second request to a third device, where the second request is used to request the first model; The second request carries the first information or the second information, the second information is used to determine the accuracy of the requested first model, and the second information is determined based on the first information.
4. The method according to claim 3, further comprising: The first device receives the accuracy of the first model and third information sent by the third device, where the third information is used to indicate information used to obtain the accuracy of the first model.
5. The method according to any one of claims 1 to 4, further comprising: The first device determines the accuracy of the first model based on at least one of the first information and fourth information, an inference result of the first model, and a true value corresponding to the inference result, wherein the fourth information is determined based on the first information.
6. The method according to claim 5, wherein: The first device determines the accuracy of the first model according to at least one of the first information and the fourth information, the inference result of the first model, and the true value corresponding to the inference result, including any one of the following: In a case where a difference between the inference result of the first model and a true value corresponding to the inference result satisfies the first information, determining that the inference result of the first model is correct; or, in a case where the difference between the inference result and the true value corresponding to the inference result does not satisfy the first information, determining that the inference result of the first model is wrong; In a case where a difference between the inference result of the first model and a true value corresponding to the inference result satisfies the fourth information, determining that the inference result of the first model is correct; Or, when the difference between the inference result and the true value corresponding to the inference result does not satisfy the fourth information, it is determined that the inference result of the first model is wrong.
7. The method according to claim 5 or 6, further comprising at least one of the following: The first device sends at least one of the first information and fourth information and the accuracy of the first model to the second device; The first device sends at least one of the first information and fourth information and the accuracy of the first model to a third device.
8. The method according to claim 3, 4 or 7, further comprising: The first device receives the second model and the first information sent by the third device; or, The first device receives the second model and fifth information sent by the third device, and the fifth information is used to indicate the determination information used to determine the accuracy of the second model; The second model is a model obtained by retraining the first model.
9. The method according to any one of claims 1 to 8, wherein: The first information, the second information, the third information, the fourth information or the fifth information includes at least one of the following: Information indicating differences in precision; Information indicating permissible errors; Information indicating the allowed values.
10. The method according to any one of claims 1 to 9, wherein: The first model includes at least one of the following: Artificial intelligence AI model; machine learning ML model; analytical model.
11. The method according to any one of claims 1 to 10, wherein: The first device is an analytical logic function; or, The second device is a consumer network function; or, The third device is the model training logic function.
12. A method for obtaining information, comprising: The second device sends first information to the first device, where the first information is used to determine the accuracy of the first model.
13. The method according to claim 12, wherein: The first information is carried in a first request, and the first request is used to request the accuracy of the first model; or, the first request is used to request the first device to perform inference.
14. The method according to claim 12 or 13, further comprising: The second device receives at least one of the first information and fourth information sent by the first device and the accuracy of the first model, wherein the fourth information is determined based on the first information.
15. The method according to claim 14, wherein: If the difference between the inference result of the first model and the true value corresponding to the inference result satisfies the first information, the inference result of the first model is correct; or, in the case where the difference between the inference result and the true value corresponding to the inference result does not satisfy the first information, the inference result of the first model is wrong; or In the case where the difference between the inference result of the first model and the true value corresponding to the inference result satisfies the fourth information, the inference result of the first model is correct; Alternatively, when the difference between the inference result and the true value corresponding to the inference result does not satisfy the fourth information, the inference result of the first model is wrong.
16. The method according to any one of claims 12 to 15, wherein: The first information or the fourth information includes at least one of the following: Information indicating differences in precision; Information indicating permissible errors; Information indicating the allowed values.
17. A method for obtaining information, comprising: The third device receives the first information or the second information sent by the first device, where the first information or the second information is used to determine the accuracy of the requested first model.
18. The method according to claim 17, wherein: The first information or the second information is carried in a second request, and the second request is used to request the first model.
19. The method according to claim 17 or 18, further comprising: The third device sends the accuracy of the first model and third information to the first device, where the third information is used to indicate information used to obtain the accuracy of the first model.
20. The method according to claim 19, further comprising: The third device receives at least one of the first information and fourth information sent by the first device and the accuracy of the first model, wherein the fourth information is determined based on the first information.
21. The method according to claim 20, wherein: If the difference between the inference result of the first model and the true value corresponding to the inference result satisfies the first information, the inference result of the first model is correct; or, in the case where the difference between the inference result and the true value corresponding to the inference result does not satisfy the first information, the inference result of the first model is wrong; or, In the case where the difference between the inference result of the first model and the true value corresponding to the inference result satisfies the fourth information, the inference result of the first model is correct; Alternatively, when the difference between the inference result and the true value corresponding to the inference result does not satisfy the fourth information, the inference result of the first model is wrong.
22. The method according to claim 20 or 21, further comprising: The third device sends the second model and the first information to the first device; or The third device sends the second model and fifth information to the first device, where the fifth information is used to indicate information used to determine the accuracy of the second model; The second model is a model obtained by retraining the first model.
23. The method according to any one of claims 17 to 22, wherein: The first information, the second information, the third information, the fourth information or the fifth information includes at least one of the following: Information indicating differences in precision; Information indicating permissible errors; Information indicating the allowed values.
24. An information acquisition device, wherein a first device comprises the information acquisition device, comprising: The acquisition module is used to acquire first information, where the first information is used to determine the accuracy of the first model.
25. An information acquisition device, wherein the second device comprises the information acquisition device, comprising: The sending module is used to send first information to the first device, where the first information is used to determine the accuracy of the first model.
26. An information acquisition device, wherein a third device comprises the information acquisition device, comprising: The first receiving module is used to receive first information or second information sent by a first device, where the first information or the second information is used to determine the accuracy of the requested first model.
27. A communication device, comprising a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, it implements the steps of the information acquisition method as described in any one of claims 1 to 11, or implements the steps of the information acquisition method as described in any one of claims 12 to 16, or implements the steps of the information acquisition method as described in any one of claims 17 to 23.
28. A chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the steps of the information acquisition method as described in any one of claims 1 to 11, or to implement the steps of the information acquisition method as described in any one of claims 12 to 16, or to implement the steps of the information acquisition method as described in any one of claims 17 to 23.
29. A readable storage medium storing a program or instruction, wherein the program or instruction, when executed by a processor, implements the steps of the information acquisition method as described in any one of claims 1 to 11, or implements the steps of the information acquisition method as described in any one of claims 12 to 16, or implements the steps of the information acquisition method as described in any one of claims 17 to 23.
Citation Information
Patent Citations
Method, apparatus, and computer program
WO2023016653A1
