Data processing method and related device
By storing multiple sets of parameter values and dynamically setting hyperparameters according to device and business needs, the problem of insufficient flexibility in machine learning model training in wireless communication is solved, the stability and accuracy of the model are improved, and the needs of different communication scenarios are adapted.
Patent Information
- Application Number
- PCT/CN2024/095695
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-06-02
- Filing Date
- 2024-05-28
- Publication Date
- 2024-12-05
AI Technical Summary
In the field of wireless communications, the lack of flexibility in setting hyperparameters of machine learning models, resulting in insufficient stability and accuracy of the training process, and it is unable to adapt to the needs of different devices and communication services.
Provide a data processing method to improve the training flexibility and adaptability of machine learning models by storing multiple sets of parameter values and dynamically determining hyperparameter values based on device level, type and QoS level of communication services.
It enhances the training stability and accuracy of machine learning models in different devices and communication scenarios, improves communication quality, and adapts to changes in device levels and business needs.
Smart Images

Figure CN2024095695_05122024_PF_FP_ABST
Abstract
Description
A data processing method and related equipment
[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on June 2, 2023, with application number 202310652873.X and invention name “A data processing method and related equipment”, the entire contents of which are incorporated by reference into this application. Technical Field
[0002] The present application relates to the field of artificial intelligence, and in particular to a data processing method and related equipment. Background Art
[0003] Artificial Intelligence (AI) is the theory, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a branch of computer science that seeks to understand the essence of intelligence and develop new intelligent machines that can respond in a manner similar to human intelligence. AI also studies the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making.
[0004] Applying AI technology to wireless communications is one application of AI. Wireless communication devices can be configured with machine learning models, which can then allocate communication resources within the wireless communication system. In practice, these machine learning models can be deployed and trained on multiple devices with wireless communication capabilities. However, these models are trained using the same set of hyperparameters across all devices, resulting in a lack of flexibility.
[0005] Summary of the Invention
[0006] An embodiment of the present application provides a data processing method and related devices, in which multiple groups of parameter values corresponding to a machine learning model are stored on a first device or other devices. When the first device trains the machine learning model, the group of parameter values that can be obtained by the first device is one of the aforementioned multiple groups of parameter values. That is, the values of the hyperparameters used when training the machine learning model have more possibilities, thereby improving the flexibility of training the machine learning model.
[0007] To solve the above technical problems, the embodiments of the present application provide the following technical solutions:
[0008] In a first aspect, an embodiment of the present application provides a data processing method, which can apply artificial intelligence technology to the field of communications, and the method includes: a first device obtains a set of parameter values corresponding to a first machine learning model; the first device trains the first machine learning model based on a set of parameter values, wherein the aforementioned set of parameter values includes the value of at least one hyperparameter used when training the first machine learning model, each set of parameter values in the multiple sets of parameter values corresponding to the first machine learning model includes the value of at least one hyperparameter used when training the first machine learning model, the aforementioned set of parameter values is one set of multiple sets of parameter values corresponding to the first machine learning model, and the multiple sets of parameter values corresponding to the machine learning model are stored on the first device or other devices.
[0009] In this implementation, multiple groups of parameter values corresponding to the machine learning model will be stored on the first device or other devices, and each group of parameter values includes the value of at least one hyperparameter used when training the machine learning model. The set of parameter values that the first device can obtain when training the machine learning model is one of the aforementioned multiple groups of parameter values, that is, the values of the hyperparameters used when training the machine learning model have more possibilities, which improves the flexibility when training the machine learning model.
[0010] In one possible implementation, if the first machine learning model is obtained by deep reinforcement learning, in one implementation, the hyperparameters used in each training of the first machine learning model include utilization probability, learning rate and discount factor, and the above-mentioned set of parameter values may include the values of any one or more of the following parameters: utilization probability, learning rate or discount factor value. In another implementation, the hyperparameters used in each training of the first machine learning model include learning rate, discount factor, utilization probability, penalty factor, first threshold value and second threshold value, the first threshold value is used to measure the quality of the first action in the first state, and the second threshold value is used to measure the quality of the second action other than the first action in the first state, the second action is different from the first action; wherein, the first training sample includes information of the first state and the first action, and the first training sample is any training sample used when training the first machine learning model; the above-mentioned set of parameters may include the values of any one or more of the following parameters: learning rate, discount factor, utilization probability, penalty factor, first threshold value or second threshold value.
[0011] In one possible implementation, before the first device obtains a set of parameter values corresponding to the machine learning model, the method further includes: the first device receiving a first signaling. The first device obtaining the set of parameter values corresponding to the machine learning model includes: the first device determining the set of parameter values corresponding to the machine learning model based on the first signaling. In this implementation, directly instructing the first device via signaling which set of parameter values to use when training the first machine learning model is beneficial for improving the efficiency of the parameter value transmission process and also for reducing the communication resources consumed during the parameter value transmission process.
[0012] In one possible implementation, the first device stores multiple sets of parameter values corresponding to the machine learning model, and the first signaling carries index information for a set of parameter values; alternatively, the first signaling carries a set of parameter values corresponding to the machine learning model. In this implementation, the first signaling can directly carry a set of parameter values used to train the machine learning model, or it can carry index information for the aforementioned set of parameter values, providing different implementation solutions and increasing the implementation flexibility of this solution.
[0013] In one possible implementation, the first signaling is NG interface signaling, Xn interface signaling, or radio resource control (RRC) signaling. For example, if the first device is a terminal device, and the first signaling is sent by a base station to the first device, the first signaling may adopt RRC signaling; or, if the first device is a base station, and the first signaling is sent by another base station to the first device, the first signaling may adopt Xn interface signaling; or, if the first device is a base station, and the first signaling is sent by a network element of a core network to the first device, the first signaling may adopt NG interface signaling, and so on. The examples are not exhaustive here. In this implementation, a variety of expressions of the first signaling are provided, which is conducive to making this solution compatible with various communication scenarios of "a set of parameter values corresponding to a machine learning model".
[0014] In one possible implementation, the set of parameter values used by the first device is changeable. In this implementation, since the communication task to be completed by the communication device that performs the inference operation of the first machine learning model may change during use, or the importance of the device may also change, that is, the communication device that performs the inference operation of the first machine learning model may undergo various changes, setting the set of parameter values used by the first device when training the first machine learning model to be changeable is conducive to making the "training process of the first machine learning model" more flexible, thereby making the "set of parameter values used when training the first machine learning model" and the "training process of the first machine learning model" more adaptable.
[0015] In one possible implementation, factors determining one set of parameter values in the multiple sets of parameter values include: the level of the first device, the type of the first device and / or the quality of service (QoS) level of the communication service corresponding to the training samples used when training the machine learning model.
[0016] In this implementation, since the value of at least one hyperparameter used in the training of the first machine learning model will affect the stability of the training process and the accuracy of the first machine learning model after training, and when the level and / or type of the first device is different, it means that the importance of the first device in the wireless communication network is different, and the QoS level of the communication service corresponding to the training sample used in the training of the first machine learning model is different, it means that the importance of using the aforementioned training samples to train the first machine learning model is different. Based on the level of the first device, the type of the first device and / or the QoS level of the communication service corresponding to the training sample used in the training of the first machine learning model, determining which set of parameter values to use for training the first machine learning model is beneficial to improving the adaptability between "a set of parameter values used when training the first machine learning model" and "the training process of the first machine learning model".
[0017] In one possible implementation, in one case, a set of parameter values used when training the first machine learning model can be updated periodically. Alternatively, a set of parameter values used when training the first machine learning model can be updated non-periodically. In another case, if the determining factors of the set of parameter values used when training the first machine learning model include QoS levels, since a training operation performed on the first machine learning model can include multiple trainings of the first machine learning model, the training samples used in different trainings can be different, and the QoS levels corresponding to different training samples can also be different, then within a training operation performed on the first machine learning model, a set of parameter values used when training the first machine learning model can also be determined based on the QoS level corresponding to the training samples used each time, that is, within a training operation performed on the first machine learning model, a set of parameter values used when training the first machine learning model can also be updated.
[0018] In one possible implementation, the higher the level of the first device, the smaller the value of the first parameter in a set of parameter values, and the larger the value of the second parameter in a set of parameter values; or, the higher the QoS level of the communication service corresponding to the training sample used in training the machine learning model, the smaller the value of the first parameter in a set of parameter values, and the larger the value of the second parameter in a set of parameter values.
[0019] Among them, the first parameter is the learning rate, and the second parameter includes any one or more of the following: a discount factor, a utilization probability, a penalty factor, a first threshold value or a second threshold value. The first threshold value is used to measure the pros and cons of the first action in the first state, and the second threshold value is used to measure the pros and cons of the second action other than the first action in the first state. The first training sample includes information about the first state and the first action. The first training sample is any training sample used when training the machine learning model.
[0020] In this implementation, for situations where the level of the first device is higher or the QoS level is higher, a smaller learning rate, a larger discount factor, utilization probability, penalty factor, first threshold value and / or second threshold value will be used. That is, for situations where the level of the first device is higher or the QoS level is higher, more attention will be paid to the stability of the training process and the accuracy of the trained first machine learning model, which is conducive to improving the communication quality when "performing communication tasks based on the information output by the trained first machine learning model".
[0021] In one possible implementation, the input to the machine learning model includes state information, and the output of the machine learning model includes action information, which includes information about the allocation of communication resources when executing a communication task under the state information. This implementation clarifies the information input to the first machine learning model and the information output by the first machine learning model, thereby clarifying the specific functionality of the first machine learning model and improving the integration of the first machine learning model with specific application scenarios.
[0022] In a second aspect, an embodiment of the present application provides a data processing method, which can apply artificial intelligence technology to the field of communications, and the method includes: a second device determines a first signaling; the second device sends the first signaling to the first device, wherein the first signaling is used for the first device to determine a set of parameter values to be used when training a first machine learning model, and the aforementioned set of parameter values is one of multiple sets of parameter values corresponding to the first machine learning model, and a set of parameter values includes the values of hyperparameters used when training the first machine learning model.
[0023] In one possible implementation, the second device determines the first signaling, including: the second device determines the first signaling based on the first information, the first information includes any one or more of the following: the level of the first device, the type of the first device and / or the QoS level of the communication service corresponding to the training samples used in the training of the machine learning model.
[0024] The meanings of the terms in each possible implementation of the second aspect of this application and the beneficial effects brought about can be referred to the first aspect and will not be repeated here.
[0025] In a third aspect, an embodiment of the present application provides a data processing device, which can apply artificial intelligence technology to the field of communications. The data processing device is applied to a first device, and the data processing device includes: an acquisition module for acquiring a set of parameter values corresponding to a first machine learning model; a training module for training the first machine learning model based on a set of parameter values, wherein a set of parameter values includes the value of at least one hyperparameter used when training the first machine learning model, and a set of parameter values is one of multiple sets of parameter values corresponding to the first machine learning model, and the multiple sets of parameter values corresponding to the first machine learning model are stored on the first device or other devices.
[0026] In one possible implementation, the data processing device also includes: a receiving module for receiving a first signaling; and an acquisition module for determining a set of parameter values corresponding to the first machine learning model based on the first signaling.
[0027] In one possible implementation, the first device stores multiple groups of parameter values corresponding to the first machine learning model, and the first signaling carries index information of a group of parameter values; or, the first signaling carries a group of parameter values corresponding to the first machine learning model.
[0028] In a possible implementation, the first signaling is NG interface signaling, Xn interface signaling or RRC signaling.
[0029] In one possible implementation, a set of parameter values used by the first device is changeable.
[0030] In one possible implementation, factors determining one set of parameter values in the multiple sets of parameter values include: the level of the first device, the type of the first device and / or the QoS level of the communication service corresponding to the training samples used when training the first machine learning model.
[0031] In one possible implementation, the higher the level of the first device, the smaller the value of the first parameter in the set of parameter values, and the larger the value of the second parameter in the set of parameter values; or, the higher the QoS level of the communication service corresponding to the training sample used when training the first machine learning model, the smaller the value of the first parameter in the set of parameter values, and the larger the value of the second parameter in the set of parameter values;
[0032] Among them, the first parameter is the learning rate, and the second parameter includes any one or more of the following: a discount factor, a utilization probability, a penalty factor, a first threshold value or a second threshold value. The first threshold value is used to measure the pros and cons of the first action in the first state, and the second threshold value is used to measure the pros and cons of the second action other than the first action in the first state. The first training sample includes information about the first state and the first action. The first training sample is any training sample used when training the first machine learning model.
[0033] In one possible implementation, the input of the first machine learning model includes state information, and the output of the first machine learning model includes action information, and the action information includes allocation information of communication resources when performing a communication task under the state information.
[0034] The specific implementation methods, meanings of terms and beneficial effects of the steps in each possible implementation method of the third aspect of this application can be referred to the first aspect and will not be repeated here.
[0035] In a fourth aspect, an embodiment of the present application provides a data processing device, which can apply artificial intelligence technology to the field of communications. The data processing device is applied to a second device, and the data processing device includes: a determination module, used to determine a first signaling; a sending module, used to send a first signaling to the first device, wherein the first signaling is used for the first device to determine a set of parameter values used when training a first machine learning model, a set of parameter values is one of multiple sets of parameter values corresponding to the first machine learning model, and a set of parameter values includes the values of hyperparameters used when training the first machine learning model.
[0036] In one possible implementation, the first signaling carries index information of a set of parameter values; or, the first signaling carries a set of parameter values corresponding to the first machine learning model.
[0037] In one possible implementation, the determination module is specifically used to determine the first signaling based on the first information, where the first information includes any one or more of the following: the level of the first device, the type of the first device and / or the quality of service QoS level of the communication service corresponding to the training samples used when training the first machine learning model.
[0038] The specific implementation methods, meanings of terms and beneficial effects of the steps in each possible implementation method of the fourth aspect of this application can be referred to the second aspect and will not be repeated here.
[0039] In the fifth aspect, an embodiment of the present application provides a device including a processor and a memory, wherein the processor is coupled to the memory, the memory is used to store programs; and the processor is used to execute the programs in the memory, so that the device executes the data processing method of the first aspect or the second aspect mentioned above.
[0040] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer executes the data processing method of the first or second aspect mentioned above.
[0041] In a seventh aspect, an embodiment of the present application provides a computer program product, which includes a program. When the program runs on a computer, it enables the computer to execute the data processing method of the first aspect or the second aspect mentioned above.
[0042] In an eighth aspect, the present application provides a chip system, which includes a processor for supporting the implementation of the functions involved in the above aspects, for example, sending or processing the data and / or information involved in the above methods. In one possible design, the chip system also includes a memory, which is used to store program instructions and data necessary for the terminal device or communication device. The chip system can be composed of a chip or can include a chip and other discrete devices. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] FIG1 is a schematic diagram of an architecture of a wireless communication system provided in an embodiment of the present application;
[0044] FIG2 is a schematic diagram of another architecture of a wireless communication system provided in an embodiment of the present application;
[0045] FIG3 is a flow chart of a data processing method provided in an embodiment of the present application;
[0046] FIG4 is another schematic diagram of a data processing method provided in an embodiment of the present application;
[0047] FIG5 is a schematic diagram illustrating the relationship between determining a target group parameter value used when training a first machine learning model, a training phase of the first machine learning model, and an inference phase of the first machine learning model, provided in an embodiment of the present application;
[0048] FIG6 is a schematic structural diagram of a data processing device provided in an embodiment of the present application;
[0049] FIG7 is another structural diagram of a data processing device provided in an embodiment of the present application;
[0050] FIG8 is a schematic diagram of a device provided in an embodiment of the present application;
[0051] FIG9 is another schematic diagram of the structure of the device provided in an embodiment of the present application;
[0052] FIG10 is a schematic diagram of the structure of a chip provided in an embodiment of the present application. DETAILED DESCRIPTION
[0053] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.
[0054] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, and this is merely a way of distinguishing the objects of the same attributes when describing them in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.
[0055] In the embodiments of the present application, "sending" and "receiving" indicate the direction of signal transmission. For example, "sending information to the XX device" can be understood as the destination of the information being the XX device, which can include direct transmission via the air interface or indirect transmission via the air interface from other units or modules. "Receiving information from the YY device" can be understood as the source of the information being the YY device, which can include direct reception from the YY device via the air interface or indirect reception from the YY device via the air interface from other units or modules. "Sending" can also be understood as the "output" of the chip interface, and "receiving" can also be understood as the "input" of the chip interface. In other words, sending and receiving can be performed between devices or within a device, for example, between components, modules, chips, software modules, or hardware modules within the device via a bus, trace, or interface. It is understandable that the information may undergo necessary processing, such as encoding, modulation, etc., between the source and destination of the information, but the destination can understand the valid information from the source. Similar expressions in this application can be understood similarly and will not be repeated here.
[0056] In the embodiments of the present application, "indication" may include direct indication and indirect indication, and may also include explicit indication and implicit indication. The information indicated by a certain information (such as the indication information described below) is called information to be indicated. In the specific implementation process, there are many ways to indicate the information to be indicated, such as but not limited to, the information to be indicated can be directly indicated, such as the information to be indicated itself or the index of the information to be indicated. The information to be indicated can also be indirectly indicated by indicating other information, wherein there is an association between the other information and the information to be indicated; it is also possible to indicate only a part of the information to be indicated, while the other parts of the information to be indicated are known or agreed in advance, for example, the indication of specific information can be achieved with the help of the arrangement order of each information agreed in advance (such as predefined by the protocol), thereby reducing the indication overhead to a certain extent. The present application does not limit the specific method of indication. It is understandable that, for the sender of the indication information, the indication information can be used to indicate the information to be indicated, and for the receiver of the indication information, the indication information can be used to determine the information to be indicated.
[0057] The present application can apply artificial intelligence technology to the field of communications. Optionally, the machine learning model in artificial intelligence technology can be applied to the application scenario of signal transmission. Before describing the method provided in the present application in detail, the scenario to which the method provided in the embodiment of the present application is applied is first introduced. Please refer to Figure 1, which is a schematic diagram of the architecture of a wireless communication system provided in an embodiment of the present application. The method provided in the present application can be applied to a wireless communication system. As shown in Figure 1, the wireless communication system includes a network device 101 and a terminal device (mobile station, MS) 102. Among them, a wireless connection can be established between the network device 101 and each terminal device 102, and a wireless connection can also be established between each terminal device 102.
[0058] Network device 101 may refer to a device that provides wireless access services in a wireless network. For example, network device 101 may be a device that connects terminal device 102 to a wireless network, and may also be referred to as a base station. The aforementioned base station may be various forms of macro base stations, micro base stations, relay stations, or access points. In wireless communication systems that utilize different wireless access technologies, the name of network device 101 having base station functionality may vary. For example, a base station may be referred to as an evolved Node B (eNB), a Node B (NB), the next generation Node B (gNB) in a fifth generation (5G) communication system, a home base station (e.g., a home evolved Node B or home Node B, HNB), a baseband unit (BBU), a wireless fidelity (Wi-Fi) access point (AP), a transmission reception point (TRP), or a radio network controller (RNC). In another possible scenario, multiple network nodes collaborate to facilitate wireless access, with different network nodes each implementing portions of the base station's functionality. For example, a network node may be a centralized unit (CU), a distributed unit (DU), a CU-control plane (CP), a CU-user plane (UP), or a radio unit (RU). The CU and DU may be separately configured or included in the same network element, such as a baseband unit (BBU). The RU may be included in a radio frequency device or radio frequency unit, such as a remote radio unit (RRU), an active antenna unit (AAU), or a remote radio head (RRH). In different systems, CU (or CU-CP and CU-UP), DU, or RU may have different names, but those skilled in the art will understand their meanings. For example, in the ORAN system, CU may also be called open CU (O-CU), DU may also be called open DU (O-DU), CU-CP may also be called open CU-CP (O-CU-CP), CU-UP may also be called open CU-UP (O-CU-UP), and RU may also be called open RU (O-RU).Among them, any unit among CU (or CU-CP, CU-UP), DU and RU can be implemented by software module, hardware module, or a combination of software module and hardware module. The embodiment of the present application does not limit the specific device form of network device 101.
[0059] Terminal device 102 refers to a wireless terminal device capable of receiving scheduling information and indication information sent by network device 101. Terminal device 102 may be a handheld device with wireless communication function, a vehicle-mounted device, a wearable device, a computing device or other processing device, etc., which are not exhaustive here.
[0060] The terminal device 102 can communicate with one or more core networks or the Internet via a wireless access network (RAN). For example, the terminal device 102 can be a portable, pocket-sized, handheld, computer-built-in, or vehicle-mounted mobile device that exchanges voice and / or data with the wireless access network. Exemplarily, the terminal device 102 can be a user agent, a cellular phone, a smart phone, a personal digital assistant (PDA), a tablet personal computer (Tablet PC), a wireless modem, a handset, a laptop computer, a personal communication service (PCS) phone, a remote station, an access point (AP), a remote terminal, an access terminal, customer premises equipment (CPE), a terminal, user equipment (UE), or a mobile terminal (MT), etc.
[0061] For another example, the terminal device 102 may also be a wearable device, which is a general term for wearable devices that are intelligently designed and developed using wearable technology for daily wear, such as glasses, gloves, watches, clothing, and shoes. A wearable device is a portable device that is worn directly on the body or integrated into the user's clothes or accessories. Wearable devices are not just hardware devices, but also achieve powerful functions through software support, data interaction, and cloud interaction. Broadly speaking, wearable smart devices include those that are fully functional, large in size, and can achieve complete or partial functions without relying on smartphones, such as smart watches or smart glasses, as well as those that only focus on a certain type of application function and need to be used in conjunction with other devices such as smartphones, such as various smart bracelets, smart helmets, and smart jewelry for vital sign monitoring.
[0062] For another example, the terminal device 102 may also be a drone, a robot, a terminal device in device-to-device (D2D) communication, a terminal device in vehicle to everything (V2X), a virtual reality (VR) device, an augmented reality (AR) device, a wireless terminal in industrial control, a terminal device in self-driving, a terminal device in remote medical, a terminal device in a smart grid, a wireless terminal in a smart city, a terminal device in a smart home, etc.
[0063] In addition, the terminal device 102 may also be a terminal device in a communication system after the 5G communication system (for example, a sixth generation (6G) communication system, etc.) or a terminal device in a future evolved public land mobile network (PLMN), etc. The embodiment of the present application does not limit the device form of the terminal device 102.
[0064] In some application scenarios, network device 101 may send downlink data to each terminal device 102, or each terminal device 102 may send uplink data to network device 101; network device 101 or terminal device 102 may utilize a machine learning model during the data transmission process. In other application scenarios, terminal devices 102 may also send data to each other, and machine learning models may be utilized during the data transmission process.
[0065] Please continue to refer to Figure 2, which is another architectural diagram of the wireless communication system provided in an embodiment of the present application. As shown in Figure 2, in a smart home scenario, various smart home products are connected through a wireless network to enable data to be transmitted between smart home products. In Figure 2, taking smart home products such as smart TVs, smart air purifiers, smart water dispensers, smart speakers, and sweeping robots as examples, these smart home products are all connected to the same wireless network through a wireless router, thereby enabling data interaction between various smart home products. In addition to the smart home products in the above examples, other types of smart home products may also be included in actual applications, such as smart refrigerators, smart range hoods, smart curtains, and other smart home products. This embodiment does not limit the types of smart home products.
[0066] In addition, different smart home products can also be directly connected wirelessly without the need to access the same wireless network through a wireless router. For example, the smart home products can be connected wirelessly via Bluetooth.
[0067] In addition to the scenarios described in Figures 1 and 2 above, the method provided in the embodiments of the present application can also be applied to other communication system scenarios. For example, in a smart factory scenario, different devices (such as intelligent robots, lathes, transport vehicles, and other equipment) are connected via a wireless network and transmit data to each other via the wireless network. For another example, the network elements and base stations in the core network can also be connected via a wireless network and transmit data to each other via the wireless network, etc. The embodiments of the present application do not limit the specific scenarios in which the data processing method is applied.
[0068] It should be noted that the wireless communication systems mentioned in the embodiments of the present application include but are not limited to: fifth generation mobile communication technology (5G) communication system, 6G communication system, satellite communication system, short-range communication system, narrowband Internet of Things system (NB-IoT), global system for mobile communications (GSM), enhanced data rate for GSM evolution system (EDGE), wideband code division multiple access system (WCDMA), code division multiple access 2000 system (CDMA2000), time division synchronous code division multiple access system (TD-SCDMA) and long term evolution system (LTE). The embodiments of the present application do not limit the specific architecture of the wireless communication system.
[0069] In the above-mentioned application scenarios, any device (hereinafter referred to as the "target device" for the convenience of description) may use a machine learning model (hereinafter referred to as the "first machine learning model" for the convenience of description) in the process of sending data. The output of the aforementioned first machine learning model may include allocation information of communication resources.
[0070] In one implementation, the steps of the training phase of the first machine learning model are performed on a device other than the target device (hereinafter referred to as the "first device" for ease of description), and only the steps of the inference phase of the first machine learning model are performed on the target device; in another implementation, the steps of the "training phase of the first machine learning model" and the "inference phase of the first machine learning model" can be performed on the first device. Among them, the "training phase of the first machine learning model" is the process of training the first machine learning model one or more times using training samples; the "inference phase of the first machine learning model" is the process of processing data using the first machine learning model. The process of iteratively training the first machine learning model is also the process of iteratively updating the parameters in the first machine learning model. After iteratively training the first machine learning model using training samples, the trained parameters corresponding to the first machine learning model can be obtained, and the aforementioned parameters obtained in the training phase will be used in the inference phase.
[0071] The data processing method provided in this application can be used to train a first machine learning model. For details, please refer to Figure 3, which is a flow chart of a data processing method provided in an embodiment of this application. As shown in Figure 3, 301, a first device obtains a set of parameter values corresponding to the first machine learning model.
[0072] 302. The first device trains a first machine learning model based on a set of parameter values, wherein the aforementioned set of parameter values includes the value of at least one hyperparameter used when training the first machine learning model, the aforementioned set of parameter values is one of multiple sets of parameter values corresponding to the first machine learning model, each set of parameter values in the multiple sets of parameter values corresponding to the first machine learning model may include the value of at least one hyperparameter used when training the first machine learning model, and the multiple sets of parameter values corresponding to the first machine learning model are stored on the first device or other devices.
[0073] Among them, the first device is an execution device for "training the first machine learning model". Optionally, if the steps of "the training phase of the first machine learning model" and "the inference phase of the first machine learning model" can be executed on the first device, then illustratively, the first device can be any device with wireless communication function in the above-mentioned multiple application scenarios; for example, the first device can be the network device 101 or the terminal device 102 in Figure 1; for another example, the first device can be the smart home or wireless router in Figure 2; or, the first device can also be other devices that need to have wireless communication functions, etc. The form of the first device is not limited in the embodiments of this application.
[0074] For example, in one case, the first machine learning model is obtained by deep reinforcement learning, that is, the input of the first machine learning model is state information The output of the first machine learning model indicates an action corresponding to the aforementioned state information During one or more trainings of the first machine learning model, the Execute under the indicated status The profit that can be generated by the indicated action By updating the parameters of the first machine learning model, the aforementioned benefits are maximized.
[0075] During the inference process of the first machine learning model, after a certain state information (hereinafter referred to as "target state information" for the convenience of description) is input into the first machine learning model, the optimal action information output by the first machine learning model under the target state information can be determined based on the aforementioned target state information (hereinafter referred to as "target action information" for the convenience of description). The action pointed to by the target action information will be executed with a certain probability. The aforementioned probability can also be called the utilization probability ε. The utilization probability is also a hyperparameter. An action will be randomly selected from other actions other than the action indicated by the target action information, and the aforementioned randomly selected action will be executed with a probability of 1-ε.
[0076] To facilitate understanding of this solution, the embodiments of this application use a method in which a first machine learning model is obtained using deep reinforcement learning as an example to describe the specific implementation method for training the first machine learning model once. Step 302 may include: during any training of the first machine learning model, the first device may obtain a first training sample from a training data set, and update parameters in the first machine learning model based on the first training sample; wherein the training data set may include one or more training samples, and the first training sample is any one of the training data set.
[0077] For example, a first machine learning model may be deployed on multiple devices in a wireless communication system, the first device is the kth device in the wireless communication system, and the first training sample may include in, represents the first training sample, Indicates the status information of the first device at time point t; Representatives in The final action information selected under the indicated state will be When input into the first machine learning model, the first machine learning model will output target action information. It can be determined based on target action information and utilization probability; For as well as The corresponding income information represents Execute under the indicated status The benefits of the indicated action, It can be calculated based on the performance information at the t+1th time point; Represents the status information at the t+1th time point. It should be noted that examples of "status information", "action information" and "performance information" can be described in subsequent steps and will not be repeated here.
[0078] For example, in a training run on the first machine learning model, the first device may update the parameters of the first machine learning model according to the first training sample to implement the state information. The Q value corresponding to each action under . For example, in one implementation, the formula used in the above update process can be as follows:
[0079] Among them, a k Represents the state information determined based on the first machine learning model Any one of the corresponding multiple action information, and All represent Any action information a k Q value, is the Q value before updating, is the updated Q value; α represents the learning rate, which is a hyperparameter.
[0080] Represents the income information in the first training sample, that is, Execute under the indicated status The indicated action is the benefit brought at the tth time point, γ represents the discount factor, which is a hyperparameter; Represents the state information of the first training sample at the t+1th time point.
[0081] Represents the status information before performing the update operation The maximum Q value among the multiple Q values corresponding to the multiple action information, a *k Representatives in Indicates the state a *k The Q value corresponding to the indicated action is the largest, is Execute a in the indicated state *k The maximum benefit that can be achieved when taking the indicated action. It can be understood as the first machine learning model based on the first training sample and the first update. Execute under the indicated status The maximum benefit that can be achieved when taking the indicated action. Represents the first machine learning model based on the update. In the indicated state The Q value corresponding to the indicated action can be understood as the Q value determined based on the first machine learning model before the update. Execute under the indicated status The maximum benefit that can be brought by the indicated action. It should be understood that the formula in formula (1) is only for the convenience of understanding this solution and is not used to limit this solution.
[0082] For example, in another implementation, the formula used in the aforementioned update process may be as follows:
[0083] Among them, formula (2) can be understood in conjunction with formula (1). The parameters that have been explained in formula (1) will not be explained here. k Represents the state information determined based on the first machine learning model Any one of the corresponding multiple action information, Represents the action information in the first training sample, when In the case of Right In the indicated state The Q value corresponding to the indicated action is updated.
[0084] exist If the Then To update, the update process is to Updated to That is to say, β represents the penalty factor, which is a hyperparameter; if it is not satisfied Then it is not correct to update.
[0085] in, Represents the first machine learning model determined based on the update The largest Q value among the corresponding multiple Q values, Representatives in In the indicated state The Q value corresponding to the indicated action is the largest; if Greater than t1, it means The value of is large enough, which means that Execute under the indicated status The benefit of the action indicated at time point t is large enough, and it can be inferred that Execute under the indicated status The action indicated by is a sufficiently good strategy; therefore, t1 represents the first threshold value, which is used to measure the In the first training sample under the indicated state The first threshold value is a hyperparameter that indicates the quality of the action.
[0086] Less than t2 means Execute a in the indicated state k The indicated action is not the optimal strategy, but Execute a in the indicated state k The benefits of the indicated actions are the same as those in Execute a in the indicated state *k The difference between the benefits of the indicated action (i.e., the optimal action) is less than t2; therefore, t2 represents the second threshold value, which is used to measure Indicates the state a k The action indicated (i.e. The second threshold value is a hyperparameter. It should be understood that the formula in formula (2) is only for the convenience of understanding this solution and is not used to limit this solution.
[0087] The first device may repeat the above steps multiple times to implement multiple training of the first machine learning model, that is, to implement multiple updates of the parameters of the first machine learning model. For example, if the first machine learning model is trained using formula (1), the hyperparameters used in each training of the first machine learning model may include utilization probability, learning rate, and discount factor; and the above set of parameter values may include the values of any one or more of the following parameters: utilization probability, learning rate, or discount factor.
[0088] For another example, if formula (2) is used to train the first machine learning model, the hyperparameters used in each training of the first machine learning model may include utilization probability, learning rate, discount factor, penalty factor, first threshold value, and second threshold value. The above set of parameter values may include parameter values of any one or more of the following parameters: utilization probability, learning rate, discount factor, penalty factor, first threshold value, or second threshold value.
[0089] In another case, if the first machine learning model is obtained using deep learning, a loss function may be used to train the first machine learning model each time. The hyperparameters used in each training of the first machine learning model may include the hyperparameters in the aforementioned loss function and the hyperparameters in other trainings.
[0090] It should be noted that the above examples are only for the convenience of understanding this solution. If other types of hyperparameters can be used in each training of the first machine learning model, the above set of parameter values can also include values of other types of hyperparameters. The specific values can be determined in combination with the actual application scenario, and are not limited in the embodiments of this application.
[0091] In an embodiment of the present application, multiple groups of parameter values corresponding to the machine learning model will be stored on the first device or other devices, and each group of parameter values in the multiple groups of parameter values includes the value of at least one hyperparameter used when training the machine learning model. The set of parameter values that the first device can obtain when training the machine learning model is one of the aforementioned multiple groups of parameter values, that is, the values of the hyperparameters used when training the machine learning model have more possibilities, thereby improving the flexibility when training the machine learning model.
[0092] In conjunction with the above description, the following describes the detailed implementation process of the training phase of the first machine learning model. Specifically, please refer to Figure 4, which is another schematic diagram of the data processing method provided in an embodiment of the present application. As shown in Figure 4, the data processing method provided in this application may include:
[0093] 401. The second device determines a first signaling, wherein the first signaling is used for the first device to determine a set of parameter values used when training the first machine learning model, the aforementioned set of parameter values is one of multiple sets of parameter values corresponding to the first machine learning model, and the aforementioned set of parameter values includes the values of hyperparameters used when training the first machine learning model.
[0094] In the embodiment of the present application, step 401 is an optional step. The first device is a device for training the first machine learning model, and the second device is a device for determining which set of parameter values, from among multiple sets of parameter values corresponding to the first machine learning model, the first device uses for training. That is, the first signaling generated by the second device is used by the first device to determine a set of parameter values (hereinafter referred to as "target set of parameter values" for convenience of description) to use when training the first machine learning model.
[0095] For example, referring to the network structure in Figures 1 and 2, the second device may be a base station, and the first device may be a terminal device; or, the second device and the first device may both be base stations; or, the second device may be a network element in the core network, and the first device may be a base station, etc. The specific product forms of the "first device" and the "second device" can be flexibly determined in combination with the actual application scenario, and are not limited in the embodiments of the present application.
[0096] For example, in one case, the second information may be deployed on the first device. The term "first information" will be used in the subsequent description. The concept of "first information" can refer to the subsequent description and will not be explained here. The second information may include multiple groups of parameter values corresponding to the first machine learning model and index information of each group of parameter values in the aforementioned multiple groups of parameter values. Each group of parameter values in the multiple groups of parameter values corresponding to the first machine learning model includes the value of at least one hyperparameter used when training the first machine learning model. The first signaling can carry the index information of the target group parameter value. After obtaining the first signaling, the first device can obtain the target group parameter value (that is, the above-mentioned "a set of parameter values used when training the first machine learning model") from the second information based on the index information of the target group parameter value carried in the first signaling.
[0097] For example, the index information of each group of parameter values in the multiple groups of parameter values corresponding to the first machine learning model may include only numbers, or may include numbers and letters, or may be expressed in other forms, etc. This is not limited in the embodiments of the present application. In order to understand this solution more intuitively, the following example uses the second information expressed as a table.
[0098] Table 1
[0099] Table 1 above shows four groups of parameter values corresponding to four index information as an example. Each group of parameter values includes the following parameter values: learning rate, discount factor, utilization probability, penalty factor, first threshold value and second threshold value. When the index information is 0, the corresponding group of parameter values may include α 1 , γ 1 , ε 1 , β 1 、 as well as For the case where the index information is 1, 2 or 3, you can refer to the case where the index information is 0 for understanding, and will not go into details here. It should be understood that the second information can also include more or less information. The examples in Table 1 are only for the convenience of understanding this solution and are not used to limit this solution.
[0100] In another case, the first signaling may carry the above-mentioned target group parameter value corresponding to the first machine learning model. After obtaining the first signaling, the first device may directly obtain the target group parameter value from the first signaling.
[0101] Exemplarily, in one case, the first machine learning model is obtained by reinforcement learning. In one implementation, the hyperparameters used in each training of the first machine learning model include utilization probability, learning rate and discount factor. The above set of parameter values may include the values of any one or more of the following parameters: utilization probability, learning rate or discount factor value.
[0102] In another implementation, the hyperparameters used in each training of the first machine learning model include a learning rate, a discount factor, a utilization probability, a penalty factor, a first threshold value, and a second threshold value. The first threshold value is used to measure the quality of the first action in the first state, and the second threshold value is used to measure the quality of the second action other than the first action in the first state, and the second action is different from the first action; wherein, the first training sample includes information about the first state and the first action, and the first training sample is any training sample used when training the first machine learning model; the above-mentioned set of parameters may include the values of any one or more of the following parameters: learning rate, discount factor, utilization probability, penalty factor, first threshold value, or second threshold value.
[0103] In another case, the first machine learning model is obtained by deep learning, and the hyperparameters used when training the first machine learning model may include hyperparameters in the loss function. Then the above set of parameter values may include the value of at least one hyperparameter in the loss function.
[0104] The concepts of "first machine learning model", "hyperparameters used when training the first machine learning model" and "the process of each training of the first machine learning model" can be understood in conjunction with the description in the embodiment corresponding to Figure 3 above, and will not be repeated here.
[0105] Optionally, the second device may obtain the first information before executing step 401. Step 401 may include: the second device determines the first signaling according to the first information.
[0106] In one case, if the first device not only performs the steps of the "training phase of the first machine learning model" but also performs the steps of the "inference phase of the first machine learning model", the first information includes any one or more of the following: the level of the first device, the type of the first device and / or the quality of service (QoS) level of the communication service corresponding to the training samples used in the training of the machine learning model. In another case, if the first device only performs the steps of the "training phase of the first machine learning model" and the target device is used to perform the steps of the "inference phase of the first machine learning model", the first information includes any one or more of the following: the level of the target device, the type of the target device and / or the QoS level of the communication service corresponding to the training samples used in the training of the first machine learning model.
[0107] For example, if the communication devices (which can be the first device or the target device) that perform the steps of the "inference phase of the first machine learning model" are all base stations, the levels of the communication devices can be divided into macro base stations, micro base stations, pico base stations, and home base stations, and the values of the hyperparameters used when training the first machine learning models on communication devices of different levels can be different. For another example, the levels of communication devices can be divided into primary user devices and secondary user devices. For another example, the levels of communication devices can be divided into cluster head node devices and non-cluster head node devices, etc.
[0108] For another example, if the communication devices (which can be the first device or the target device) that execute the steps of the "inference stage of the first machine learning model" are all terminal devices, the types of communication devices can be divided into vehicle-mounted devices, handheld devices and Internet of Things devices, and the values of the hyperparameters used when training the first machine learning models deployed on different types of communication devices are different, etc. It should be noted that the "level of communication equipment" and "type of communication equipment" can also be in other forms. The examples here are only used to prove the feasibility of this solution and are not used to limit this solution.
[0109] The second device can obtain the first information in a variety of ways. In one implementation, the first information can be sent by the first device to the second device. For example, the first device is a terminal device and the second device is a base station. When the terminal device accesses the cell where the base station is located, the level and / or type of the terminal device can be sent to the base station. For another example, the first device is a terminal device and the second device is a base station. When the terminal device needs to obtain the target group parameter value (that is, the above-mentioned "a set of parameter values used when training the first machine learning model"), the terminal device can send the level of the terminal device, the type of the terminal device and / or the QoS level of the communication service corresponding to the training sample used when training the first machine learning model to the base station. For another example, the first device is a base station, the second device is a network element in the core network, and the target device is a terminal device. When the base station needs to obtain the target group parameter value (that is, the above-mentioned "a set of parameter values used when training the first machine learning model"), the base station can send the first information target device level and / or type to the base station, etc. In other scenarios, the first device can also send the first information to the second device. The specific details can be flexibly determined in combination with the actual application scenario and are not limited here.
[0110] In another implementation method, the first information can also be sent by other devices to the second device. For example, the first device is a base station, and the second device is a network element in the core network. The level and / or type of the first device can be sent by other network elements in the core network to the second device, and so on. It should be noted that the examples of the second device obtaining the first information here are only to prove the feasibility of this solution and are not used to limit this solution.
[0111] Exemplarily, the second device may be deployed with third information. In one case, the first signaling carries index information of the target group parameter value (that is, the above-mentioned "a set of parameter values used when training the first machine learning model"). In one implementation, the third information may include multiple levels of communication devices and an index information corresponding to each level. Optionally, the third information may also include a set of parameter values pointed to by an index information corresponding to each level of communication devices. Step 401 may include: the second device obtains an index information corresponding to the level of the first device (or the level of the target device) from the third information, that is, the index information of the target group parameter value is determined.
[0112] In order to understand this solution more intuitively, the following example uses the third information represented as a table.
[0113] Table 2
[0114] Table 2 shows four index information and four groups of parameter values corresponding to four levels of communication equipment as an example. Each group of parameter values includes the following parameter values: learning rate, discount factor, utilization probability, penalty factor, first threshold value, and second threshold value. When the level of the communication equipment is a macro base station, the index information corresponding to the level of "macro base station" is 0. The group of parameter values pointed to by index information 0 may include α 1 , γ 1 , ε 1 , β 1 、 as well as For the case where the level of the communication equipment is a micro base station, a pico base station or a home base station, you can refer to the case where the level of the communication equipment is a macro base station for understanding, and will not go into details here. It should be understood that the third information can also include more or less information. The examples in Table 2 are only for the convenience of understanding this solution and are not used to limit this solution.
[0115] In another implementation, the third information may include multiple categories of communication devices and index information corresponding to each category. Optionally, the third information may also include a set of parameter values pointed to by the index information corresponding to each category of communication devices. Step 401 may include: the second device obtaining, from the third information, index information corresponding to the category of the first device (or the category of the target device), i.e., the index information that determines the target group parameter values.
[0116] In another implementation, the third information may include multiple QoS levels and an index information corresponding to each QoS level. Optionally, the third information may also include a group of parameter values pointed to by the index information corresponding to each QoS level. Step 401 may include: after the second device determines the QoS level of the communication service corresponding to the training sample used when training the first machine learning model one or more times (hereinafter referred to as the "target QoS level" for convenience of description), it may obtain an index information corresponding to the target QoS level from the third information, that is, determine the index information of the target group parameter value.
[0117] Exemplarily, the first training sample (that is, any training sample used to train the first machine learning model) is collected when a communication device (such as a first device or a target device) performs a communication service using the first machine learning model. The communication service corresponding to the first training sample can be understood as the communication service completed when the action indicated by the first training sample is executed under the state indicated by the first training sample.
[0118] In order to understand this solution more intuitively, the following example uses the third information represented as a table.
[0119] Table 3
[0120] Among them, Table 3 shows three index information and three groups of parameter values corresponding to three QoS levels as an example. Each group of parameter values includes the values of the following parameters: learning rate, discount factor, utilization probability, penalty factor, first threshold value and second threshold value. When the QoS level is 0, the index information corresponding to the level of "QoS level 0" is 0, and the group of parameter values pointed to by index information 0 may include α 1 , γ 1 , ε 1 , β 1 、 as well as For the cases of QoS level 1 or QoS level 2, you can refer to the case of QoS level 0 for understanding, and will not go into details here. It should be understood that the third information can also include more or less information. The examples in Table 3 are only for the convenience of understanding this solution and are not used to limit this solution.
[0121] In another case, the first signaling carries target group parameter values (i.e., the aforementioned "set of parameter values used when training the first machine learning model"). In one implementation, the third information may include multiple levels of communication devices and a set of parameter values corresponding to each level. Optionally, the third information may also include index information for each set of parameter values. Step 401 may include: the second device obtaining, from the third information, the target group parameter value corresponding to the level of the first device (or the level of the target device).
[0122] In one implementation, the third information may include multiple types of communication devices and a set of parameter values corresponding to each type. Step 401 may include: the second device obtains the target group parameter value corresponding to the type of the first device (or the type of the target device) from the third information.
[0123] In another implementation, the third information may include multiple QoS levels and a set of parameter values corresponding to each QoS level. Step 401 may include: the second device may obtain a target set of parameter values corresponding to a target QoS level from the third information. The concept of "target QoS level" can be referred to in the above description.
[0124] In an embodiment of the present application, since the value of at least one hyperparameter used in the training of the first machine learning model will affect the stability of the training process and the accuracy of the first machine learning model after training, and when the level and / or type of the first device is different, it means that the importance of the first device in the wireless communication network is different, and the QoS level of the communication service corresponding to the training sample used in the training of the first machine learning model is different, it means that the importance of using the aforementioned training samples to train the first machine learning model is different. Based on the level of the first device, the type of the first device and / or the QoS level of the communication service corresponding to the training sample used in the training of the first machine learning model, determining which set of parameter values to use for training the first machine learning model is beneficial to improving the adaptability between "a set of parameter values used when training the first machine learning model" and "the training process of the first machine learning model".
[0125] Optionally, the higher the level of the communication device (i.e., the first device or target device) used to execute the "inference stage of the first machine learning model", the smaller the value of the first parameter in a set of parameter values, and the larger the value of the second parameter in a set of parameter values; or, the higher the target QoS level, the smaller the value of the first parameter in a set of parameter values, and the larger the value of the second parameter in a set of parameter values, and the smaller the value of the first parameter in a set of parameter values, the larger the value of the second parameter in a set of parameter values.
[0126] The first parameter is a learning rate, and the second parameter includes any one or more of the following: a discount factor, a utilization probability, a penalty factor, a first threshold value, or a second threshold value.
[0127] Illustratively, a larger learning rate value results in a larger gradient descent during each training of the first machine learning model. Illustratively, a larger decrease in the Q value during each training of the first machine learning model results in a lower stability during the training phase of the first machine learning model. Optionally, a higher-level communication device used to execute the "inference phase of the first machine learning model" can reduce the learning rate value.
[0128] The larger the value of the discount factor, the greater the consideration of benefits at non-current time points during each training of the first machine learning model, the smoother the actions determined based on the trained first machine learning model, and, in other words, the smoother the benefits brought about by multiple actions determined based on the trained first machine learning model. Optionally, the higher the level of the communication device used to perform the "inference phase of the first machine learning model," the larger the value of the discount factor can be.
[0129] The larger the utilization probability, the more likely the first device or target device is to select the action indicated by the action information output by the first machine learning model when determining the final action to be executed, and the higher the stability of the final action to be executed. Optionally, the higher the level of the communication device used to execute the "inference phase of the first machine learning model," the larger the utilization probability can be.
[0130] A larger penalty factor, i.e., a larger penalty for one or more non-optimal actions under a given state, makes it easier for the first machine learning model to select the optimal action under the given state. Alternatively, a higher-level communication device used to execute the inference phase of the first machine learning model can increase the penalty factor.
[0131] The larger the first threshold value, the higher the quality of the training samples that meet the screening condition corresponding to the first threshold value, thereby making the parameter update process in the first machine learning model more cautious, that is, the training process of the first machine learning model is more cautious, and thus the training process of the first machine learning model is more stable. Optionally, the higher the level of the communication device used to perform the "inference phase of the first machine learning model", the larger the value of the first threshold value can be.
[0132] The larger the second threshold, the fewer Q values that meet the screening condition corresponding to the second threshold, and the fewer Q values that need to be penalized. This allows for more cautious updating of the parameters in the first machine learning model, thereby increasing the stability of the training process of the first machine learning model. Optionally, the higher the level of the communication device used to perform the "inference phase of the first machine learning model," the larger the second threshold can be.
[0133] For example, taking Table 2 above as an example, α 1 ≤α 2 ≤α 3 ≤α 4 , γ 1 ≥γ 2 ≥γ 3 ≥γ 4 , ε 1 ≥ε 2 ≥ε 3 ≥ε 4 , β 1 ≥β 2 ≥β 3 ≥β 4 , For another example, referring to Table 3 above, α 1 ≤α 2 ≤α 3 , γ 1 ≥γ 2 ≥γ3 , ε 1 ≥ε 2 ≥ε 3 , β 1 ≥β 2 ≥β 3 , It should be understood that the examples given here are only for the convenience of understanding this solution and are not intended to limit this solution.
[0134] In an embodiment of the present application, for situations where the level of the first device is higher or the QoS level is higher, a smaller learning rate will be used, and a larger discount factor, utilization probability, penalty factor, first threshold value and / or second threshold value will be used. That is, for situations where the level of the first device is higher or the QoS level is higher, more attention will be paid to the stability of the training process, and more attention will be paid to the accuracy of the trained first machine learning model, which is conducive to improving the communication quality when "performing communication tasks based on the information output by the trained first machine learning model".
[0135] 402. The second device sends a first signaling to the first device.
[0136] In the embodiment of the present application, step 402 is an optional step. After determining the information carried in the first signaling, the second device may send the first signaling to the first device, and correspondingly, the first device may receive the first signaling.
[0137] Among them, if the second device is a base station and the first device is a terminal device, the first signaling is the signaling used by the base station and the terminal device for wireless communication. For example, the first signaling can be radio resource control (RRC) signaling. Alternatively, if the second device and the first device are both base stations, the first signaling can be the signaling used for wireless communication between different base stations. For example, the first signaling can be Xn interface signaling. Alternatively, if the second device is a network element in the core network and the first device is a base station, the first signaling can be the signaling used when the network element of the core network and the base station perform wireless communication. For example, the first signaling can be NG interface signaling, etc. The first signaling can also use other types of signaling. The examples here are only to prove the feasibility of this solution and are not used to limit this solution.
[0138] To further understand this solution, the specific form of the first signaling when the target group parameter value is carried in the RRC signaling and the Xn interface signaling is exemplified below.
[0139] MARL-Config::=SEQUENCE{
[0140] Learning-Rate INTEGER (0..16)
[0141] Discount-Factor INTEGER (0..16)
[0142] Exploitation-Probability INTEGER (0..16)
[0143] Penalty-Factor INTEGER (0..16)
[0144] Sample-Conditional-Threshold INTEGER (0..16)
[0145] Policy-Conditional-Threshold INTEGER (0..16)
[0146] }
[0147] Disclosed above is a specific implementation method of carrying the target group parameter value in RRC signaling, where "INTEGER" represents an integer type. In the above example, the first threshold value is called the sample condition threshold value and the second threshold value is called the policy addition threshold value. It should be understood that the above example is only for the convenience of understanding this solution and is not used to limit this solution.
[0148] Table 4
[0149] Table 4 uses the example of target group parameter values carried in Xn interface signaling. IE / Group Name represents the names of multiple elements carried in the Xn interface signaling, IE type and reference represents the data type and related information of each element, "INTEGER" represents an integer data type, Semantics description represents the element's description, Criticality represents the element's critical value, and Assigned Criticality represents the element's assigned criticality. Table 4 uses the first threshold value as the sample conditional threshold and the second threshold value as the policy conditional threshold as an example. Table 4 also uses the example of the learning rate, discount factor, exploitation probability, sample conditional threshold, and policy conditional threshold carried in Xn interface signaling. It should be understood that the examples in Table 4 are intended only to facilitate understanding of this solution and are not intended to limit this solution.
[0150] The specific expression form of the first signaling when the index information of the target group parameter value is carried in the RRC signaling and the Xn interface signaling is exemplified below.
[0151] MARL-Config::=SEQUENCE{
[0152] Parameter-IndexINTEGER(0..15)
[0153] }
[0154] Among them, Parameter-Index represents the index information of the target group parameter value, and INTEGER represents that the data type is an integer. It should be understood that the above example is only for the convenience of understanding this solution and is not used to limit this solution.
[0155] Table 5
[0156] Among them, Table 5 can be understood in combination with the above description of Table 3. The header part of Table 5 is the same as the header part of Table 4 and will not be repeated here. Parameter-Index represents the index information of the target group parameter value. It should be understood that the examples in Table 5 are only for the convenience of understanding this solution and are not used to limit this solution.
[0157] In an embodiment of the present application, multiple forms of expression of the first signaling are provided, which is conducive to making the present solution compatible with various communication scenarios of "a set of parameter values corresponding to a machine learning model".
[0158] 403. The first device obtains a set of parameter values corresponding to the first machine learning model.
[0159] In an embodiment of the present application, before the first device trains the first machine learning model one or more times, it is necessary to first obtain a set of parameter values corresponding to the first machine learning model. The aforementioned set of parameter values includes the value of at least one hyperparameter used when training the first machine learning model one or more times. The aforementioned set of parameter values is a set determined from multiple sets of parameter values corresponding to the first machine learning model.
[0160] In one implementation, if steps 401 and 402 are not performed, step 403 may include: the first device may store multiple sets of parameter values corresponding to the first machine learning model, and the first device determines, based on the first information, from the multiple sets of parameter values a target set of parameter values to be used when training the first machine learning model one or more times. The specific implementation of the aforementioned steps and the meaning of "first information" can be found in the description of step 401, except that the second device in step 401 is replaced by the first device in step 403, and will not be further described here.
[0161] In another implementation, if steps 401 and 402 are performed, step 403 may include: the first device determines the target group parameter value (i.e., a set of parameters used when training the first machine learning model) based on the first signaling.
[0162] For example, in one case, second information may be stored on the first device, and the first device may obtain the target group parameter value from the second information based on the index information of the target group parameter value carried in the first signaling. The meaning of the "second information" can be found in the description in step 401.
[0163] In another case, the first signaling carries a set of parameter values corresponding to the first machine learning model, and step 403 may include: the first device may obtain the target group parameter values from the first signaling.
[0164] In an embodiment of the present application, directly instructing the first device through signaling which set of parameter values to use when training the first machine learning model is beneficial to improving the efficiency of the parameter value transmission process and also beneficial to reducing the communication resources consumed during the parameter value transmission process.
[0165] A set of parameter values used in training the machine learning model can be directly carried in the first signaling, or index information of the aforementioned set of parameter values can be carried in the first signaling, providing different implementation solutions and improving the implementation flexibility of this solution.
[0166] 404. The first device trains the first machine learning model based on a set of parameter values, wherein the set of parameter values includes the value of at least one hyperparameter used when training the first machine learning model, the set of parameter values is one of multiple sets of parameter values corresponding to the first machine learning model, and the multiple sets of parameter values corresponding to the first machine learning model are stored on the first device or other devices.
[0167] In an embodiment of the present application, after the first device obtains a set of parameters used when training the first machine learning model, it can use the training samples in the training data set to train the first machine learning model once or multiple times. The specific implementation process of the first device training the first machine learning model each time can refer to the description in the embodiment corresponding to Figure 3 above, which will not be repeated here. Among them, the training data set includes one or more training samples, and the first training sample in the training data set (that is, any training sample in the training data set) can include information on the state at the t-th time point (for the purpose of description, it can be referred to as the "first state" later), information on the first action, profit information, and state information at the t+1-th time point. The aforementioned profit information can be the profit brought about when the first action is performed in the first state.
[0168] Optionally, when the first machine learning model is a reinforcement learning algorithm, the input of the first machine learning model may be state information, and the output of the first machine learning model may be action information. If the first machine learning model is used in the communications field, the action information may include allocation information of communication resources when executing a communication task under the state information input to the first machine learning model.
[0169] For example, a first machine learning model is used to solve a power control problem in a wireless network. The state information may include channel state information from the data transmitter to the data receiver, interference caused by neighboring transmitters other than the data transmitter to the data transmitter and the data receiver, or other state information. Action information is used to indicate the transmit power of the data transmitter. Exemplarily, the action information may be one of multiple transmit power levels, where the aforementioned transmit power level can determine the final transmit power value. The action information may also be a transmit power value. The benefit information may be determined based on the data throughput rate, packet loss rate, latency, and / or other information collected after the action is executed.
[0170] For another example, the bandwidth resource allocation problem in a wireless network is solved by a first machine learning model. The state information may include the channel state information from the data transmitter (such as a base station) to the data receiver, the interference caused by neighbor transmitters other than the data transmitter to the data transmitter and the data receiver, and the amount of data to be sent by the data transmitter. The action information is used to indicate the bandwidth resources allocated to the data transmitter for data transmission. Exemplarily, the action information can be the number of resource blocks (RBs) or a specific bandwidth value in Hz. The benefit information can be determined based on the statistical data throughput rate, packet loss rate, delay and / or other information after performing the aforementioned action (i.e., sending data with the bandwidth resources corresponding to the aforementioned action).
[0171] For another example, taking the use of the first machine learning model to solve the user scheduling problem in a wireless network as an example, the state information may include the channel state information from the data transmitter to the target user, the amount of data to be sent by the data transmitter, the cache state information of the data to be transmitted by the data transmitter, etc. The action information is used to indicate the result of user scheduling. For example, the action information may include a set of 0 / 1 sequences with the same number of elements as the number of users. The users corresponding to the positions with a value of 1 in the sequence are scheduled to perform data transmission this time, and the users corresponding to the positions with a value of 0 in the sequence are not allowed to perform data transmission this time. The benefit information may include the statistical data throughput rate, packet loss rate, delay and / or other information after executing the aforementioned action (i.e., user scheduling based on the aforementioned action). It should be noted that the examples of "state information", "action information" and "benefit information" here are only for the convenience of understanding this solution and are not used to limit this solution.
[0172] In the embodiment of the present application, it is clarified what information is input into the first machine learning model and what information is output by the first machine learning model, that is, the specific function of the first machine learning model is clarified, and the degree of integration between the first machine learning model and the specific application scenario is improved.
[0173] In which, the first device can perform one or more training operations on the first machine learning model, and each training operation performed by the first device on the first machine learning model can include training the first machine learning model one or more times.
[0174] To further understand the present solution, please refer to Figure 5, which is a schematic diagram of the relationship between the target group parameter values used when training the first machine learning model, the training stage of the first machine learning model, and the inference stage of the first machine learning model, provided in an embodiment of the present application. In Figure 5, taking the example of the first device being able to perform both the steps of the "training stage of the first machine learning model" and the steps of the "inference stage of the first machine learning model", the first device can perform multiple training operations on the first machine learning model. As shown in Figure 5, after the first device performs one or more inferences (i.e., the inference stage in Figure 5) through the first machine learning model, it can train the first machine learning model again (i.e., the training stage in Figure 5). A training operation performed by the first device on the first machine learning model may include multiple trainings on the first machine learning model (i.e., the multiple training stages included in Figure 5).
[0175] Continuing with FIG5 , after the first device determines the target group parameter values to be used when training the first machine learning model, the target group parameter values may be used when performing multiple training operations on the first machine learning model. That is, this application does not limit the relationship between the execution frequency of "determining the target group parameter values to be used when training the first machine learning model" and the execution frequency of "training the first machine learning model." It should be understood that the example in FIG5 is merely for facilitating understanding of this solution and is not intended to limit this solution.
[0176] Optionally, the above-mentioned set of parameter values used by the first device when training the first machine learning model is changeable. In one case, in combination with the content shown in Figure 5, the set of parameter values used when training the first machine learning model can be periodically updated. Alternatively, the set of parameter values used when training the first machine learning model can be updated non-periodically. Exemplarily, if the communication device that executes the "inference phase of the first machine learning model" is a base station, and the base station is adjusted from a micro base station to a micro-micro base station, the set of parameter values used when training the first machine learning model on the aforementioned device can be updated, etc., and the update of the "set of parameter values used when training the first machine learning model" can also be triggered in other scenarios. The examples here are only for the convenience of understanding this solution and are not used to limit the solution.
[0177] In another case, if the determining factors of a set of parameter values used when training the first machine learning model include QoS levels, since one training operation performed on the first machine learning model may include multiple trainings of the first machine learning model, the training samples used in different trainings may be different, and the QoS levels corresponding to different training samples may also be different, then within one training operation performed on the first machine learning model, a set of parameter values used in the training of the first machine learning model can also be determined based on the QoS level corresponding to the training samples used each time, that is, the set of parameter values used in the training of the first machine learning model can also be updated within one training operation performed on the first machine learning model.
[0178] In an embodiment of the present application, since the communication task to be completed by the communication device that performs the reasoning operation of the first machine learning model may change during use, or the importance of the device may also change, that is, the communication device that performs the reasoning operation of the first machine learning model may undergo various changes, setting the set of parameter values used by the first device when training the first machine learning model to be changeable will help make the "training process of the first machine learning model" more flexible, thereby making the "set of parameter values used when training the first machine learning model" and the "training process of the first machine learning model" more adapted.
[0179] On the basis of the embodiments corresponding to Figures 1 to 5, in order to better implement the above-mentioned scheme of the embodiment of the present application, the following also provides related devices for implementing the above-mentioned scheme. Please refer to Figure 6 in detail. Figure 6 is a structural diagram of a data processing device provided in the embodiment of the present application. The data processing device 600 is applied to the first device. The data processing device 600 includes: an acquisition module 601 for acquiring a set of parameter values corresponding to the first machine learning model; a training module 602 for training the first machine learning model based on a set of parameter values, wherein a set of parameter values includes the value of at least one hyperparameter used when training the first machine learning model, and a set of parameter values is one of multiple sets of parameter values corresponding to the first machine learning model, and the multiple sets of parameter values corresponding to the first machine learning model are stored on the first device or other devices.
[0180] Optionally, the data processing device 600 also includes: a receiving module 603, used to receive a first signaling; and an acquisition module 601, specifically used to determine a set of parameter values corresponding to the first machine learning model based on the first signaling.
[0181] Optionally, the first device stores multiple groups of parameter values corresponding to the first machine learning model, and the first signaling carries index information of a group of parameter values; or, the first signaling carries a group of parameter values corresponding to the first machine learning model.
[0182] Optionally, the first signaling is NG interface signaling, Xn interface signaling or RRC signaling.
[0183] Optionally, the set of parameter values adopted by the first device is changeable.
[0184] Optionally, factors determining one set of parameter values in the multiple sets of parameter values include: the level of the first device, the type of the first device and / or the QoS level of the communication service corresponding to the training samples used when training the first machine learning model.
[0185] Optionally, the higher the level of the first device, the smaller the value of the first parameter in the set of parameter values, and the larger the value of the second parameter in the set of parameter values; or, the higher the QoS level of the communication service corresponding to the training sample used in training the first machine learning model, the smaller the value of the first parameter in the set of parameter values, and the larger the value of the second parameter in the set of parameter values;
[0186] Among them, the first parameter is the learning rate, and the second parameter includes any one or more of the following: a discount factor, a utilization probability, a penalty factor, a first threshold value or a second threshold value. The first threshold value is used to measure the pros and cons of the first action in the first state, and the second threshold value is used to measure the pros and cons of the second action other than the first action in the first state. The first training sample includes information about the first state and the first action. The first training sample is any training sample used when training the first machine learning model.
[0187] Optionally, the input of the first machine learning model includes state information, and the output of the first machine learning model includes action information, and the action information includes allocation information of communication resources when performing a communication task under the state information.
[0188] It should be noted that the information interaction, execution process, etc. between the modules / units in the data processing device 600 are based on the same concept as the various method embodiments corresponding to Figures 3 to 5 in this application. For specific contents, please refer to the description in the method embodiments shown above in this application, and will not be repeated here.
[0189] Please refer to Figure 7, which is another structural schematic diagram of the data processing device provided in an embodiment of the present application. The data processing device 700 is applied to the second device, and the data processing device 700 includes: a determination module 701, used to determine the first signaling; a sending module 702, used to send the first signaling to the first device, wherein the first signaling is used for the first device to determine a set of parameter values used when training the first machine learning model, the set of parameter values is one of multiple sets of parameter values corresponding to the first machine learning model, and the set of parameter values includes the values of hyperparameters used when training the first machine learning model.
[0190] Optionally, the first signaling carries index information of a set of parameter values; or, the first signaling carries a set of parameter values corresponding to the first machine learning model.
[0191] Optionally, the determination module 701 is specifically used to determine the first signaling based on the first information, where the first information includes any one or more of the following: the level of the first device, the type of the first device and / or the QoS level of the communication service corresponding to the training sample used when training the first machine learning model.
[0192] It should be noted that the information interaction, execution process, etc. between the modules / units in the data processing device 700 are based on the same concept as the various method embodiments corresponding to Figures 3 to 5 in this application. For specific contents, please refer to the description in the method embodiments shown above in this application, and will not be repeated here.
[0193] Please refer to Figure 8, which is a schematic diagram of a structure of a device provided in an embodiment of the present application. The device 800 can specifically be the device used as a terminal device in the above-mentioned embodiment. The example shown in Figure 8 is that the terminal device is implemented by the terminal device (or a component in the terminal device). In a possible logical structure diagram of the device 800, the device 800 may include, but is not limited to, at least one processor 801 and a communication port 802.
[0194] Optionally, the device may further include at least one of a memory 803 and a bus 804 . In an embodiment of the present application, the at least one processor 801 is used to control and process actions of the communication device 800 .
[0195] In addition, the processor 801 can be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field-programmable gate array or other programmable logic device, a transistor logic device, a hardware component, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a digital signal processor and a microprocessor, and so on. Those skilled in the art will clearly understand that for the sake of convenience and brevity of description, the specific operating processes of the systems, devices, and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0196] It should be noted that when the first device or the second device involved in the above embodiments is specifically manifested as a terminal device, the device 800 shown in Figure 8 can be used to implement the steps implemented by the terminal device in the aforementioned method embodiment. The specific implementation methods of the device 800 shown in Figure 8 to perform the aforementioned steps can refer to the description in the aforementioned method embodiment, and will not be repeated here one by one.
[0197] Please refer to Figure 9, which is another schematic diagram of the structure of a device provided in an embodiment of the present application. The device 900 can specifically be the device used as a network device in the above-mentioned embodiment. The example shown in Figure 9 is that the network device is implemented by the network device (or a component in the network device); that is, when the first device or the second device involved in the above-mentioned embodiment is specifically manifested as a network device, it can be implemented using the device 900 shown in Figure 9; illustratively, when the first device or the second device is a base station, it can be implemented using the device 900 shown in Figure 9.
[0198] The structure of the communication device can refer to the structure shown in Figure 9. The device 900 includes at least one processor 911 and at least one network interface 912. Further optionally, the communication device also includes at least one memory 914, at least one transceiver 913 and one or more antennas 915. The processor 911, the transceiver 913, the memory 914 and the network interface 912 are connected, for example, via a bus. In an embodiment of the present application, the connection may include various interfaces, transmission lines or buses, etc., which are not limited in this embodiment. The antenna 915 is connected to the transceiver 913. The network interface 912 is used to enable the communication device to communicate with other communication devices through a communication link. For example, the network interface 912 may include a network interface between the communication device and the core network device, such as an S1 interface, and the network interface may include a network interface between the communication device and other communication devices (such as other network devices or core network devices), such as an X2 or Xn interface.
[0199] Processor 911 is primarily used to process communication protocols and communication data, control the entire communication device, execute software programs, and process software program data, for example, to support the communication device in performing the actions described in the embodiments. A communication device may include a baseband processor and a central processing unit. The baseband processor is primarily used to process communication protocols and communication data, while the central processing unit is primarily used to control the entire terminal device, execute software programs, and process software program data. Processor 911 in Figure 9 may integrate the functions of both a baseband processor and a central processing unit. Those skilled in the art will appreciate that the baseband processor and the central processing unit may also be independent processors interconnected via a bus or other technology. Those skilled in the art will appreciate that a terminal device may include multiple baseband processors to accommodate different network standards, multiple central processing units to enhance its processing capabilities, and various components of the terminal device may be connected via various buses. The baseband processor may also be referred to as a baseband processing circuit or a baseband processing chip. The central processing unit may also be referred to as a central processing circuit or a central processing chip. The functionality for processing communication protocols and communication data may be built into the processor or stored in memory as a software program, which is executed by the processor to implement the baseband processing functionality.
[0200] The memory is primarily used to store software programs and data. Memory 914 can exist independently and be connected to processor 911. Alternatively, memory 914 and processor 911 can be integrated together, for example, within a single chip. Memory 914 can store program code for executing the technical solutions of the embodiments of the present application, and execution is controlled by processor 911. The various computer program codes executed can also be considered drivers for processor 911.
[0201] Figure 9 shows only one memory and one processor. In an actual terminal device, there may be multiple processors and multiple memories. The memory may also be referred to as a storage medium or a storage device. The memory may be a storage element on the same chip as the processor, i.e., an on-chip storage element, or an independent storage element, which is not limited in the present embodiment.
[0202] The transceiver 913 can be used to support the reception or transmission of radio frequency signals between the communication device and the terminal, and the transceiver 913 can be connected to the antenna 915. The transceiver 913 includes a transmitter Tx and a receiver Rx. Specifically, one or more antennas 915 can receive radio frequency signals. The receiver Rx of the transceiver 913 is used to receive the radio frequency signal from the antenna, convert the radio frequency signal into a digital baseband signal or a digital intermediate frequency signal, and provide the digital baseband signal or digital intermediate frequency signal to the processor 911 so that the processor 911 can further process the digital baseband signal or digital intermediate frequency signal, such as demodulation and decoding. In addition, the transmitter Tx in the transceiver 913 is also used to receive a modulated digital baseband signal or digital intermediate frequency signal from the processor 911, convert the modulated digital baseband signal or digital intermediate frequency signal into a radio frequency signal, and transmit the radio frequency signal through one or more antennas 915. Specifically, the receiver Rx can selectively perform one or more stages of down-mixing and analog-to-digital conversion on the RF signal to obtain a digital baseband signal or a digital intermediate frequency signal. The order of the down-mixing and analog-to-digital conversion processes is adjustable. The transmitter Tx can selectively perform one or more stages of up-mixing and digital-to-analog conversion on the modulated digital baseband signal or digital intermediate frequency signal to obtain a RF signal. The order of the up-mixing and digital-to-analog conversion processes is adjustable. The digital baseband signal and the digital intermediate frequency signal may be collectively referred to as digital signals.
[0203] The transceiver 913 may also be referred to as a transceiver unit, a transceiver, a transceiver device, etc. Optionally, a device in the transceiver unit that implements a receiving function may be referred to as a receiving unit, and a device in the transceiver unit that implements a transmitting function may be referred to as a transmitting unit. That is, the transceiver unit includes a receiving unit and a transmitting unit. The receiving unit may also be referred to as a receiver, an input port, a receiving circuit, etc., and the transmitting unit may be referred to as a transmitter, a transmitter, or a transmitting circuit, etc.
[0204] Exemplarily, when the first device or the second device involved in the above embodiments is specifically manifested as a base station, the device 900 shown in Figure 9 can be used to implement the steps implemented by the base station in the aforementioned method embodiment. The specific implementation methods of the device 900 shown in Figure 9 to perform the aforementioned steps can all be referred to the description in the aforementioned method embodiment, and will not be repeated here one by one.
[0205] A computer-readable storage medium is also provided in an embodiment of the present application. The computer-readable storage medium stores a program for signal processing. When the computer-readable storage medium is run on a computer, the computer executes the steps executed by the first device in the method described in the embodiments shown in Figures 3 to 5 above, or the computer executes the steps executed by the second device in the method described in the embodiments shown in Figures 3 to 5 above.
[0206] An embodiment of the present application also provides a computer program product, which, when running on a computer, enables the computer to execute the steps executed by the first device in the method described in the embodiments shown in Figures 3 to 5 above, or enables the computer to execute the steps executed by the second device in the method described in the embodiments shown in Figures 3 to 5 above.
[0207] The first device or the second device provided in the embodiments of the present application may specifically be a chip, and the chip includes: a processing unit and a communication unit. The processing unit may be, for example, a processor, and the communication unit may be, for example, an input / output interface, a pin, or a circuit. The processing unit may execute computer-executable instructions stored in the storage unit so that the chip performs the data processing method described in the embodiments shown in Figures 3 to 5 above. Optionally, the storage unit is a storage unit within the chip, such as a register, a cache, etc. The storage unit may also be a storage unit located outside the chip within the wireless access device, such as a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), etc.
[0208] Specifically, see Figure 10 , which is a schematic diagram of the structure of a chip provided in an embodiment of the present application. The chip can be represented as a neural network processor NPU 100. NPU 100 is mounted on a host CPU (host CPU) as a coprocessor, and the host CPU assigns tasks. The core of the NPU is arithmetic circuit 1003, which is controlled by controller 1004 to extract matrix data from memory and perform multiplication operations.
[0209] In some implementations, the arithmetic circuit 1003 includes multiple processing units (PEs). In some implementations, the arithmetic circuit 1003 is a two-dimensional systolic array. The arithmetic circuit 1003 can also be a one-dimensional systolic array or other electronic circuit capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuit 1003 is a general-purpose matrix processor.
[0210] For example, assume there are input matrix A, weight matrix B, and output matrix C. The arithmetic circuit retrieves the corresponding data of matrix B from weight memory 1002 and caches it on each PE in the arithmetic circuit. The arithmetic circuit retrieves the data of matrix A from input memory 1001 and performs a matrix operation on matrix B. The partial or final matrix result is stored in accumulator 1008.
[0211] Unified memory 1006 is used to store input and output data. Weight data is directly transferred to weight memory 1002 through the Direct Memory Access Controller (DMAC) 1005. Input data is also transferred to unified memory 1006 through the DMAC.
[0212] BIU stands for Bus Interface Unit, i.e., bus interface unit 1010 , which is used for interaction between the AXI bus, DMAC, and instruction fetch buffer (IFB) 1009 .
[0213] The bus interface unit 1010 (BIU) is used for the instruction fetch memory 1009 to obtain instructions from the external memory, and is also used for the storage unit access controller 1005 to obtain the original data of the input matrix A or the weight matrix B from the external memory.
[0214] DMAC is mainly used to transfer input data in the external memory DDR to the unified memory 1006 or transfer weight data to the weight memory 1002 or transfer input data to the input memory 1001.
[0215] The vector calculation unit 1007 includes multiple operation processing units. When necessary, it further processes the output of the operation circuit, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, etc. It is mainly used for non-convolutional / fully connected layer network calculations in neural networks, such as batch normalization, pixel-level summation, and upsampling of feature planes.
[0216] In some implementations, the vector calculation unit 1007 can store the processed output vector to the unified memory 1006. For example, the vector calculation unit 1007 can apply a linear function and / or a nonlinear function to the output of the operation circuit 1003, such as linear interpolation of the feature plane extracted by the convolution layer, or accumulate a vector of values to generate an activation value. In some implementations, the vector calculation unit 1007 generates a normalized value, a pixel-level summed value, or both. In some implementations, the processed output vector can be used as an activation input to the operation circuit 1003, for example, for use in a subsequent layer in a neural network.
[0217] An instruction fetch buffer 1009 connected to the controller 1004 is used to store instructions used by the controller 1004;
[0218] Unified memory 1006, input memory 1001, weight memory 1002, and instruction fetch memory 1009 are all on-chip memories. External memories are private to the NPU hardware architecture.
[0219] Among them, the operations of each layer in the first machine learning model can be performed by the operation circuit 1003 or the vector calculation unit 1007.
[0220] The processor mentioned in any of the above places can be a general-purpose central processing unit, a microprocessor, an ASIC, or one or more integrated circuits for controlling the execution of the program of the above-mentioned first aspect method.
[0221] It should also be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided in this application, the connection relationship between the modules indicates that there is a communication connection between them, which can be specifically implemented as one or more communication buses or signal lines.
[0222] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course can also be implemented by special hardware including application-specific integrated circuits, special CPUs, special memories, special components, etc. In general, all functions performed by computer programs can be easily implemented with corresponding hardware, and the specific hardware structures used to implement the same function can also be diverse, such as analog circuits, digital circuits or special circuits, etc. However, for the present application, software program implementation is a better implementation method in most cases. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer's floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk or optical disk, etc., and includes a number of instructions to enable a computer device (which can be a personal computer, training equipment, or network equipment, etc.) to execute the methods described in each embodiment of the present application.
[0223] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.
[0224] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, a computer, a training device or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website, a computer, a training device or a data center. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).
Claims
1. A data processing method, It is characterized in that The method comprises: The first device obtains a set of parameter values corresponding to the machine learning model; The first device trains the machine learning model based on the set of parameter values, wherein the set of parameter values includes the value of at least one hyperparameter used when training the machine learning model, and the set of parameter values is one of multiple sets of parameter values corresponding to the machine learning model, and the multiple sets of parameter values corresponding to the machine learning model are stored on the first device or other devices.
2. The method according to claim 1, It is characterized in that Before the first device acquires a set of parameter values corresponding to the machine learning model, the method further includes: the first device receiving a first signaling; The first device obtains a set of parameter values corresponding to the machine learning model, including: The first device determines the set of parameter values corresponding to the machine learning model based on the first signaling.
3. The method according to claim 2, It is characterized in that The first device stores the multiple groups of parameter values corresponding to the machine learning model, and the first signaling carries index information of the group of parameter values; or, The first signaling carries the set of parameter values corresponding to the machine learning model.
4. The method according to claim 2 or 3, It is characterized in that The first signaling is NG interface signaling, Xn interface signaling or radio resource control RRC signaling.
5. The method according to any one of claims 1 to 3, It is characterized in that The set of parameter values used by the first device is changeable.
6. The method according to any one of claims 1 to 3, It is characterized in that Factors determining one set of parameter values among the multiple sets of parameter values include: the level of the first device, the type of the first device and / or the quality of service QoS level of the communication service corresponding to the training samples used in training the machine learning model.
7. The method according to claim 6, It is characterized in that The higher the level of the first device, the smaller the value of the first parameter in the set of parameter values, and the larger the value of the second parameter in the set of parameter values; or, The higher the QoS level of the communication service corresponding to the training sample used in training the machine learning model, the smaller the value of the first parameter in the set of parameter values, and the larger the value of the second parameter in the set of parameter values; Among them, the first parameter is the learning rate, and the second parameter includes any one or more of the following: a discount factor, a utilization probability, a penalty factor, a first threshold value or a second threshold value, the first threshold value is used to measure the pros and cons of a first action in a first state, and the second threshold value is used to measure the pros and cons of a second action other than the first action in the first state, the first training sample includes information about the first state and the first action, and the first training sample is any training sample used when training the machine learning model.
8. The method according to any one of claims 1 to 3, It is characterized in that The input of the machine learning model includes state information, and the output of the machine learning model includes action information, and the action information includes allocation information of communication resources when performing a communication task under the state information.
9. A data processing method, It is characterized in that The method comprises: The second device determines the first signaling; The second device sends a first signaling to the first device, wherein the first signaling is used for the first device to determine a set of parameter values used when training a machine learning model, the set of parameter values being one of multiple sets of parameter values corresponding to the machine learning model, and the set of parameter values including values of hyperparameters used when training the machine learning model.
10. The method according to claim 9, It is characterized in that The first signaling carries index information of the set of parameter values; or, The first signaling carries the set of parameter values corresponding to the machine learning model.
11. The method according to claim 9 or 10, It is characterized in that The second device determines the first signaling, including: the second device determines the first signaling based on first information, and the first information includes any one or more of the following: the level of the first device, the type of the first device and / or the service quality QoS level of the communication service corresponding to the training samples used in training the machine learning model.
12. A data processing device, It is characterized in that The device is applied to a first device, and includes: An acquisition module, used to obtain a set of parameter values corresponding to the machine learning model; A training module, used to train the machine learning model based on the set of parameter values, wherein the set of parameter values includes the value of at least one hyperparameter used when training the machine learning model, and the set of parameter values is one of multiple sets of parameter values corresponding to the machine learning model, and the multiple sets of parameter values corresponding to the machine learning model are stored on the first device or other devices.
13. The device according to claim 12, It is characterized in that The device further includes: a receiving module, configured to receive a first signaling; The acquisition module is specifically used to determine the set of parameter values corresponding to the machine learning model based on the first signaling.
14. The device according to claim 13, It is characterized in that The first device stores the multiple groups of parameter values corresponding to the machine learning model, and the first signaling carries index information of the group of parameter values; or, The first signaling carries the set of parameter values corresponding to the machine learning model.
15. The device according to claim 12 or 13, It is characterized in that The first signaling is NG interface signaling, Xn interface signaling or radio resource control RRC signaling.
16. The device according to any one of claims 12 to 15, It is characterized in that The set of parameter values used by the first device is changeable.
17. The device according to any one of claims 12 to 15, It is characterized in that Factors determining one set of parameter values among the multiple sets of parameter values include: the level of the first device, the type of the first device and / or the quality of service QoS level of the communication service corresponding to the training samples used in training the machine learning model.
18. The device according to claim 17, It is characterized in that The higher the level of the first device, the smaller the value of the first parameter in the set of parameter values, and the larger the value of the second parameter in the set of parameter values; or, The higher the QoS level of the communication service corresponding to the training sample used in training the machine learning model, the smaller the value of the first parameter in the set of parameter values, and the larger the value of the second parameter in the set of parameter values; Among them, the first parameter is the learning rate, and the second parameter includes any one or more of the following: a discount factor, a utilization probability, a penalty factor, a first threshold value or a second threshold value, the first threshold value is used to measure the pros and cons of a first action in a first state, and the second threshold value is used to measure the pros and cons of a second action other than the first action in the first state, the first training sample includes information about the first state and the first action, and the first training sample is any training sample used when training the machine learning model.
19. The device according to any one of claims 12 to 15, It is characterized in that The input of the machine learning model includes state information, and the output of the machine learning model includes action information, and the action information includes allocation information of communication resources when performing a communication task under the state information.
20. A data processing device, It is characterized in that The device is applied to a second device, and includes: A determination module, configured to determine a first signaling; A sending module, used for sending a first signaling to a first device, wherein the first signaling is used for the first device to determine a set of parameter values used when training a machine learning model, the set of parameter values is one set of multiple sets of parameter values corresponding to the machine learning model, and the set of parameter values includes the values of hyperparameters used when training the machine learning model.
21. The device according to claim 20, It is characterized in that The first signaling carries index information of the set of parameter values; or, The first signaling carries the set of parameter values corresponding to the machine learning model.
22. The device according to claim 20 or 21, It is characterized in that The determination module is specifically used to determine the first signaling based on first information, where the first information includes any one or more of the following: the level of the first device, the type of the first device and / or the quality of service QoS level of the communication service corresponding to the training samples used in training the machine learning model.
23. A device, It is characterized in that comprising a processor and a memory, the processor being coupled to the memory, The memory is used to store programs; The processor is configured to execute the program in the memory so that the device performs the method according to any one of claims 1 to 11.
24. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a program, and when the program is executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 11.
25. A computer program product, It is characterized in that The computer program product comprises a program, and when the program is run on a computer, the computer is caused to execute the method according to any one of claims 1 to 11.
26. A chip, It is characterized in that The chip comprises a processor, and when the processor executes computer-implemented instructions, the chip is caused to perform the method according to any one of claims 1 to 11.
Citation Information
Patent Citations
System resource and model hyper-parameter collaborative optimization method in deep learning training
CN112836796A
Communication method and device
CN114143799A
Machine learning model training method and related equipment
CN114282678A
Reinforcement machine learning framework for dynamic demand forecasting
US20230054692A1
