Data processing method and related equipment
By storing multiple sets of parameter values and dynamically selecting, the problem of lack of flexibility in hyperparameter selection during machine learning model training in wireless communication is solved, and the flexibility and accuracy of model training is improved, and it is suitable for a variety of communication scenarios.
Patent Information
- Application Number
- CN202310652873.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-06-02
- Publication Date
- 2024-12-03
AI Technical Summary
In the field of wireless communication, the lack of flexibility in hyperparameter selection during training of machine learning models, resulting in limited training process and model accuracy.
Provide a data processing method, by storing multiple sets of parameter values and dynamically selecting during training, it improves the possibility of hyperparameters and adapts to the needs of different devices and communication services.
It improves the training flexibility and adaptability of machine learning models, improves the stability and model accuracy of the training process, and is suitable for different communication scenarios.
Smart Images

Figure CN119067230A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence, and in particular to a data processing method and related equipment. Background Art
[0002] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a branch of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.
[0003] Applying artificial intelligence technology to the field of wireless communications is one of the applications of artificial intelligence. A machine learning model can be configured in a device that performs wireless communications, and the device can use the machine learning model to allocate communication resources for the wireless communication system. In actual applications, the aforementioned machine learning model can be deployed on multiple devices with wireless communication functions, and multiple devices with wireless communication functions can also train the aforementioned machine learning model. However, the hyperparameters used by different devices when training the machine learning model are all the same set of parameter values, which lacks flexibility. Summary of the invention
[0004] An embodiment of the present application provides a data processing method and related devices. Multiple groups of parameter values corresponding to a machine learning model are stored on a first device or other devices. A group of parameter values that can be obtained by the first device when training the machine learning model is one of the aforementioned multiple groups of parameter values. That is, the values of the hyperparameters used when training the machine learning model have more possibilities, thereby improving the flexibility when training the machine learning model.
[0005] To solve the above technical problems, the present application provides the following technical solutions:
[0006] In a first aspect, an embodiment of the present application provides a data processing method, which can apply artificial intelligence technology to the field of communications, and the method includes: a first device obtains a set of parameter values corresponding to a first machine learning model; the first device trains the first machine learning model based on a set of parameter values, wherein the aforementioned set of parameter values includes the value of at least one hyperparameter used when training the first machine learning model, each set of parameter values in the multiple sets of parameter values corresponding to the first machine learning model includes the value of at least one hyperparameter used when training the first machine learning model, the aforementioned set of parameter values is one set of multiple sets of parameter values corresponding to the first machine learning model, and the multiple sets of parameter values corresponding to the machine learning model are stored on the first device or other devices.
[0007] In this implementation, multiple groups of parameter values corresponding to the machine learning model will be stored on the first device or other devices, and each group of parameter values includes at least one hyperparameter value used when training the machine learning model. The set of parameter values that can be obtained by the first device when training the machine learning model is one of the aforementioned multiple groups of parameter values, that is, the values of the hyperparameters used when training the machine learning model have more possibilities, thereby improving the flexibility when training the machine learning model.
[0008] In one possible implementation, if the first machine learning model is obtained by deep reinforcement learning, in one implementation, the hyperparameters used in each training of the first machine learning model include utilization probability, learning rate and discount factor, and the above-mentioned set of parameter values may include the values of any one or more of the following parameters: utilization probability, learning rate or discount factor value. In another implementation, the hyperparameters used in each training of the first machine learning model include learning rate, discount factor, utilization probability, penalty factor, first threshold value and second threshold value, the first threshold value is used to measure the quality of the first action in the first state, the second threshold value is used to measure the quality of the second action other than the first action in the first state, and the second action is different from the first action; wherein, the first training sample includes information of the first state and the first action, and the first training sample is any training sample used in training the first machine learning model; the above-mentioned set of parameters may include the values of any one or more of the following parameters: learning rate, discount factor, utilization probability, penalty factor, first threshold value or second threshold value.
[0009] In one possible implementation, before the first device obtains a set of parameter values corresponding to the machine learning model, the method further includes: the first device receives a first signaling. The first device obtains a set of parameter values corresponding to the machine learning model, including: the first device determines a set of parameter values corresponding to the machine learning model according to the first signaling. In this implementation, directly indicating which set of parameter values to use when training the first machine learning model through signaling is beneficial to improving the efficiency of the parameter value transmission process and also beneficial to reducing the communication resources consumed during the parameter value transmission process.
[0010] In one possible implementation, a plurality of groups of parameter values corresponding to the machine learning model are stored in the first device, and the first signaling carries index information of a group of parameter values; or, the first signaling carries a group of parameter values corresponding to the machine learning model. In this implementation, a group of parameter values used to train the machine learning model can be directly carried in the first signaling, or the index information of the aforementioned group of parameter values can be carried in the first signaling, which provides different implementation schemes and improves the implementation flexibility of this scheme.
[0011] In one possible implementation, the first signaling is NG interface signaling, Xn interface signaling, or radio resource control (RRC) signaling. Exemplarily, if the first device is a terminal device, and the first signaling is sent by a base station to the first device, the first signaling may use RRC signaling; or, if the first device is a base station, and the first signaling is sent by another base station to the first device, the first signaling may use Xn interface signaling; or, if the first device is a base station, and the first signaling is sent by a network element of the core network to the first device, the first signaling may use NG interface signaling, and so on, which are not exhaustive here. In this implementation, a variety of expressions of the first signaling are provided, which is conducive to making the present solution compatible with various communication scenarios of "a set of parameter values corresponding to a machine learning model".
[0012] In one possible implementation, a set of parameter values used by the first device is changeable. In this implementation, since the communication task to be completed by the communication device that performs the reasoning operation of the first machine learning model may change during use, or the importance of the device may also change, that is, the communication device that performs the reasoning operation of the first machine learning model may undergo various changes. Setting the set of parameter values used by the first device when training the first machine learning model to be changeable is conducive to making the "training process of the first machine learning model" more flexible, thereby making the "set of parameter values used when training the first machine learning model" and the "training process of the first machine learning model" more adaptable.
[0013] In one possible implementation, factors determining a set of parameter values among multiple sets of parameter values include: the level of the first device, the type of the first device and / or the quality of service (QoS) level of the communication service corresponding to the training samples used in training the machine learning model.
[0014] In this implementation, since the value of at least one hyperparameter used in the training of the first machine learning model will affect the stability of the training process and the accuracy of the first machine learning model after training, and when the level and / or type of the first device is different, it means that the importance of the first device in the wireless communication network is different, and the QoS level of the communication service corresponding to the training sample used in the training of the first machine learning model is different, it means that the importance of using the aforementioned training samples to train the first machine learning model is different. Based on the level of the first device, the type of the first device and / or the QoS level of the communication service corresponding to the training sample used in the training of the first machine learning model, determining which set of parameter values to use to train the first machine learning model is beneficial to improving the adaptability between "a set of parameter values used when training the first machine learning model" and "the training process of the first machine learning model".
[0015] In one possible implementation, in one case, a set of parameter values used when training the first machine learning model can be updated periodically. Alternatively, a set of parameter values used when training the first machine learning model can be updated non-periodically. In another case, if the determining factors of a set of parameter values used when training the first machine learning model include QoS level, since a training operation performed on the first machine learning model can include multiple trainings of the first machine learning model, the training samples used in different trainings can be different, and the QoS levels corresponding to different training samples can also be different. In a training operation performed on the first machine learning model, a set of parameter values used when training the first machine learning model can also be determined according to the QoS level corresponding to the training samples used each time, that is, a set of parameter values used when training the first machine learning model can also be updated in a training operation performed on the first machine learning model.
[0016] In one possible implementation, the higher the level of the first device, the smaller the value of the first parameter in a set of parameter values, and the larger the value of the second parameter in a set of parameter values; or, the higher the QoS level of the communication service corresponding to the training sample used in training the machine learning model, the smaller the value of the first parameter in a set of parameter values, and the larger the value of the second parameter in a set of parameter values.
[0017] Among them, the first parameter is the learning rate, and the second parameter includes any one or more of the following: a discount factor, a utilization probability, a penalty factor, a first threshold value or a second threshold value. The first threshold value is used to measure the pros and cons of a first action in a first state, and the second threshold value is used to measure the pros and cons of a second action other than the first action in the first state. The first training sample includes information about the first state and the first action. The first training sample is any training sample used when training a machine learning model.
[0018] In this implementation, for situations where the level of the first device is higher or the QoS level is higher, a smaller learning rate, a larger discount factor, utilization probability, penalty factor, first threshold value and / or second threshold value will be used. That is, for situations where the level of the first device is higher or the QoS level is higher, more emphasis will be placed on the stability of the training process, and more emphasis will be placed on the accuracy of the trained first machine learning model, which is conducive to improving the communication quality when "performing communication tasks based on the information output by the trained first machine learning model".
[0019] In one possible implementation, the input of the machine learning model includes state information, the output of the machine learning model includes action information, and the action information includes allocation information of communication resources when performing communication tasks under the state information. In this implementation, it is clarified what information is input into the first machine learning model and what information is output by the first machine learning model, that is, the specific function of the first machine learning model is clarified, and the degree of integration between the first machine learning model and the specific application scenario is improved.
[0020] In a second aspect, an embodiment of the present application provides a data processing method, which can apply artificial intelligence technology to the field of communications, and the method includes: a second device determines a first signaling; the second device sends a first signaling to the first device, wherein the first signaling is used for the first device to determine a set of parameter values used when training a first machine learning model, the aforementioned set of parameter values is one of multiple sets of parameter values corresponding to the first machine learning model, and a set of parameter values includes the values of hyperparameters used when training the first machine learning model.
[0021] In one possible implementation, the second device determines the first signaling, including: the second device determines the first signaling based on first information, the first information including any one or more of the following: the level of the first device, the type of the first device and / or the QoS level of the communication service corresponding to the training samples used in training the machine learning model.
[0022] The meanings of the terms in each possible implementation of the second aspect of the present application and the beneficial effects brought about can all be referred to the first aspect and will not be repeated here.
[0023] In a third aspect, an embodiment of the present application provides a data processing device, which can apply artificial intelligence technology to the field of communications. The data processing device is applied to a first device, and the data processing device includes: an acquisition module, used to acquire a set of parameter values corresponding to a first machine learning model; a training module, used to train the first machine learning model based on a set of parameter values, wherein a set of parameter values includes the value of at least one hyperparameter used when training the first machine learning model, and a set of parameter values is one of multiple sets of parameter values corresponding to the first machine learning model, and the multiple sets of parameter values corresponding to the first machine learning model are stored on the first device or other devices.
[0024] In one possible implementation, the data processing device also includes: a receiving module, used to receive a first signaling; and an acquisition module, specifically used to determine a set of parameter values corresponding to the first machine learning model based on the first signaling.
[0025] In one possible implementation, the first device stores multiple groups of parameter values corresponding to the first machine learning model, and the first signaling carries index information of a group of parameter values; or, the first signaling carries a group of parameter values corresponding to the first machine learning model.
[0026] In a possible implementation manner, the first signaling is NG interface signaling, Xn interface signaling or RRC signaling.
[0027] In a possible implementation, a set of parameter values used by the first device is changeable.
[0028] In one possible implementation, factors determining a set of parameter values among the multiple sets of parameter values include: the level of the first device, the type of the first device, and / or the QoS level of the communication service corresponding to the training samples used in training the first machine learning model.
[0029] In a possible implementation, the higher the level of the first device, the smaller the value of the first parameter in the set of parameter values, and the larger the value of the second parameter in the set of parameter values; or, the higher the QoS level of the communication service corresponding to the training sample used in training the first machine learning model, the smaller the value of the first parameter in the set of parameter values, and the larger the value of the second parameter in the set of parameter values;
[0030] Among them, the first parameter is the learning rate, and the second parameter includes any one or more of the following: a discount factor, a utilization probability, a penalty factor, a first threshold value or a second threshold value. The first threshold value is used to measure the pros and cons of a first action in a first state, and the second threshold value is used to measure the pros and cons of a second action other than the first action in the first state. The first training sample includes information about the first state and the first action, and the first training sample is any training sample used when training the first machine learning model.
[0031] In one possible implementation, the input of the first machine learning model includes state information, and the output of the first machine learning model includes action information, and the action information includes allocation information of communication resources when performing a communication task under the state information.
[0032] The specific implementation methods, meanings of terms and beneficial effects of the steps in each possible implementation method of the third aspect of the present application can all be referred to the first aspect and will not be repeated here.
[0033] In a fourth aspect, an embodiment of the present application provides a data processing device, which can apply artificial intelligence technology to the field of communications. The data processing device is applied to a second device, and the data processing device includes: a determination module, used to determine a first signaling; a sending module, used to send a first signaling to the first device, wherein the first signaling is used for the first device to determine a set of parameter values used when training a first machine learning model, a set of parameter values is one of multiple sets of parameter values corresponding to the first machine learning model, and a set of parameter values includes the values of hyperparameters used when training the first machine learning model.
[0034] In one possible implementation, the first signaling carries index information of a set of parameter values; or, the first signaling carries a set of parameter values corresponding to the first machine learning model.
[0035] In one possible implementation, the determination module is specifically used to determine the first signaling based on the first information, where the first information includes any one or more of the following: the level of the first device, the type of the first device and / or the quality of service QoS level of the communication service corresponding to the training samples used in training the first machine learning model.
[0036] The specific implementation methods, meanings of terms and beneficial effects of the steps in each possible implementation method of the fourth aspect of the present application can all be referred to the second aspect and will not be repeated here.
[0037] In a fifth aspect, an embodiment of the present application provides a device, including a processor and a memory, wherein the processor is coupled to the memory, the memory is used to store programs; the processor is used to execute the programs in the memory, so that the device executes the data processing method of the first aspect or the second aspect mentioned above.
[0038] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer executes the data processing method of the first aspect or the second aspect mentioned above.
[0039] In a seventh aspect, an embodiment of the present application provides a computer program product, which includes a program. When the program runs on a computer, the computer executes the data processing method of the first aspect or the second aspect mentioned above.
[0040] In an eighth aspect, the present application provides a chip system, which includes a processor for supporting the implementation of the functions involved in the above aspects, for example, sending or processing the data and / or information involved in the above methods. In one possible design, the chip system also includes a memory, which is used to store program instructions and data necessary for a terminal device or a communication device. The chip system can be composed of a chip, or it can include a chip and other discrete devices. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] Figure 1 A schematic diagram of an architecture of a wireless communication system provided in an embodiment of the present application;
[0042] Figure 2 Another schematic diagram of the architecture of a wireless communication system provided in an embodiment of the present application;
[0043] Figure 3 A flowchart of a data processing method provided in an embodiment of the present application;
[0044] Figure 4 Another schematic diagram of a data processing method provided in an embodiment of the present application;
[0045] Figure 5 A schematic diagram of the relationship between determining a target group parameter value used when training a first machine learning model, a training phase of the first machine learning model, and an inference phase of the first machine learning model provided in an embodiment of the present application;
[0046] Figure 6 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application;
[0047] Figure 7 Another structural schematic diagram of a data processing device provided in an embodiment of the present application;
[0048] Figure 8 A schematic diagram of a device provided in an embodiment of the present application;
[0049] Fig. 9 Another structural schematic diagram of the device provided in the embodiment of the present application;
[0050] Fig.10 A schematic diagram of the structure of a chip provided in an embodiment of the present application. DETAILED DESCRIPTION
[0051] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.
[0052] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.
[0053] "Send" and "receive" in the embodiments of the present application indicate the direction of signal transmission. For example, "send information to XX device" can be understood as the destination of the information is XX device, which can include direct transmission through the air interface, and also include indirect transmission through the air interface by other units or modules. "Receive information from YY device" can be understood as the source of the information is YY device, which can include direct reception from YY device through the air interface, and can also include indirect reception from YY device through other units or modules through the air interface. "Send" can also be understood as the "output" of the chip interface, and "receive" can also be understood as the "input" of the chip interface. In other words, sending and receiving can be carried out between devices or within the device, for example, sending or receiving between components, modules, chips, software modules or hardware modules within the device through a bus, a wiring or an interface. It is understandable that the information may be processed as necessary between the source and the destination of the information transmission, such as encoding, modulation, etc., but the destination can understand the valid information from the source. Similar expressions in this application can be understood in a similar way and will not be repeated.
[0054] In the embodiments of the present application, "indication" may include direct indication and indirect indication, and may also include explicit indication and implicit indication. The information indicated by a certain information (such as the indication information described below) is called information to be indicated. In the specific implementation process, there are many ways to indicate the information to be indicated, such as but not limited to, directly indicating the information to be indicated, such as the information to be indicated itself or the index of the information to be indicated. The information to be indicated may also be indirectly indicated by indicating other information, wherein there is an association between the other information and the information to be indicated; it may also be possible to indicate only a part of the information to be indicated, while the other part of the information to be indicated is known or agreed in advance, for example, the indication of specific information can be realized by means of the arrangement order of each information agreed in advance (such as predefined by the protocol), thereby reducing the indication overhead to a certain extent. The present application does not limit the specific method of indication. It is understandable that, for the sender of the indication information, the indication information can be used to indicate the information to be indicated, and for the receiver of the indication information, the indication information can be used to determine the information to be indicated.
[0055] This application can apply artificial intelligence technology to the field of communications. Optionally, the machine learning model in artificial intelligence technology can be applied to the application scenario of signal transmission. Before describing the method provided by this application in detail, the scenario in which the method provided by the embodiment of this application is applied is first introduced. Please refer to Figure 1 , Figure 1 The method provided in this application can be applied to a wireless communication system, such as Figure 1 As shown, the wireless communication system includes a network device 101 and a terminal device (mobile station, MS) 102. A wireless connection can be established between the network device 101 and each terminal device 102, and a wireless connection can also be established between each terminal device 102.
[0056] The network device 101 may refer to a device that provides wireless access services in a wireless network. Exemplarily, the network device 101 may be a device that connects the terminal device 102 to the wireless network, and may also be called a base station; the aforementioned base station may be various forms of macro base stations, micro base stations, relay stations or access points, etc. In wireless communication systems using different wireless access technologies, the name of the network device 101 having the base station function may be different. For example, the base station may be called an evolved Node B (eNB), a Node B (NB), the next generation Node B (gNB) in the fifth generation (5G) communication system, a home base station (e.g., home evolved NodeB, or home Node B, HNB), a base band unit (BBU), a wireless fidelity (Wi-Fi) access point (AP), a transmission reception point (TRP) or a radio network controller (RNC), etc. In another possible scenario, multiple network nodes collaborate to assist in achieving wireless access, and different network nodes respectively implement part of the functions of the base station. For example, a network node may be a centralized unit (CU), a distributed unit (DU), a CU-control plane (CP), a CU-user plane (UP), or a radio unit (RU), etc. CU and DU may be separately configured, or may be included in the same network element, such as a baseband unit (BBU). RU may be included in a radio frequency device or a radio frequency unit, such as a remote radio unit (RRU), an active antenna unit (AAU), or a remote radio head (RRH). In different systems, CU (or CU-CP and CU-UP), DU, or RU may also have different names, but those skilled in the art may understand their meanings. For example, in the ORAN system, CU may also be called open CU (O-CU), DU may also be called open DU (O-DU), CU-CP may also be called open CU-CP (O-CU-CP), CU-UP may also be called open CU-UP (O-CU-UP), and RU may also be called open RU (O-RU).Among them, any unit in CU (or CU-CP, CU-UP), DU and RU can be implemented by software module, hardware module, or a combination of software module and hardware module. The embodiment of the present application does not limit the specific device form of the network device 101.
[0057] The terminal device 102 refers to a wireless terminal device that can receive scheduling information and indication information sent by the network device 101. The terminal device 102 can be a handheld device with wireless communication function, a vehicle-mounted device, a wearable device, a computing device or other processing device, etc., which are not exhaustive here.
[0058] The terminal device 102 can communicate with one or more core networks or the Internet via a wireless access network (RAN). For example, the terminal device 102 can be a portable, pocket-sized, handheld, computer-built-in or vehicle-mounted mobile device that exchanges voice and / or data with the wireless access network. Exemplarily, the terminal device 102 can be a user agent, a cellular phone, a smart phone, a personal digital assistant (PDA), a tablet PC (Tablet Personal Computer, Tablet PC), a wireless modem, a handset, a laptop computer, a personal communication service (PCS) phone, a remote station (remotestation), an access point (AP), a remote terminal device (remote terminal), an access terminal device (access terminal), a customer premises equipment (CPE), a terminal, a user equipment (UE) or a mobile terminal (MT), etc.
[0059] For another example, the terminal device 102 may also be a wearable device, which is a general term for wearable devices that are intelligently designed and developed using wearable technology for daily wear, such as glasses, gloves, watches, clothing, and shoes. A wearable device is a portable device that is worn directly on the body or integrated into the user's clothes or accessories. Wearable devices are not just hardware devices, but also achieve powerful functions through software support, data interaction, and cloud interaction. Broadly speaking, wearable smart devices include those that are fully functional, large in size, and can achieve complete or partial functions without relying on smartphones, such as smart watches or smart glasses, as well as those that only focus on a certain type of application function and need to be used in conjunction with other devices such as smartphones, such as various types of smart bracelets, smart helmets, and smart jewelry for vital sign monitoring.
[0060] For another example, the terminal device 102 may also be a drone, a robot, a terminal device in device-to-device (D2D) communication, a terminal device in vehicle to everything (V2X), a virtual reality (VR) device, an augmented reality (AR) device, a wireless terminal in industrial control, a terminal device in self driving, a terminal device in remote medical, a terminal device in a smart grid, a wireless terminal in a smart city, a terminal device in a smart home, etc.
[0061] In addition, the terminal device 102 may also be a terminal device in a communication system after the 5G communication system (for example, a sixth generation (6G) communication system, etc.) or a terminal device in a future evolved public land mobile network (PLMN), etc. The embodiment of the present application does not limit the device form of the terminal device 102.
[0062] In some application scenarios, the network device 101 may send downlink data to each terminal device 102, or each terminal device 102 may send uplink data to the network device 101; the network device 101 or the terminal device 102 may use the machine learning model in the process of sending data. In other application scenarios, each terminal device 102 may also send data to each other, and the terminal device 102 may use the machine learning model in the process of sending data.
[0063] Please continue reading Figure 2 , Figure 2 Another schematic diagram of the architecture of the wireless communication system provided in the embodiment of the present application. Figure 2 As shown in FIG. 1 , in a smart home scenario, various smart home products are connected via a wireless network to enable data to be transmitted between the smart home products. Figure 2 In the embodiment, smart home products such as smart TV, smart air purifier, smart water dispenser, smart speaker and sweeping robot are taken as examples. These smart home products are connected to the same wireless network through a wireless router, so as to realize data interaction between the smart home products. In addition to the smart home products in the above examples, other types of smart home products may also be included in actual applications, such as smart refrigerators, smart range hoods, smart curtains and other smart home products. This embodiment does not limit the types of smart home products.
[0064] In addition, different smart home products can also be directly connected wirelessly without being connected to the same wireless network through a wireless router. For example, the smart home products can be connected wirelessly through Bluetooth.
[0065] In addition to the above Figure 1 and Figure 2 In addition to the scenarios described, the method provided in the embodiments of the present application can also be applied to other communication system scenarios. For example, in a smart factory scenario, different devices (such as intelligent robots, lathes, transport vehicles and other equipment) are connected through a wireless network and transmit data to each other through the wireless network. For another example, the network elements and base stations in the core network can also be connected through a wireless network, and data can be transmitted to each other through the wireless network, etc. The embodiments of the present application do not limit the specific scenarios in which the data processing method is applied.
[0066] It should be noted that the wireless communication systems mentioned in the embodiments of the present application include but are not limited to: fifth generation mobile communication technology (5th Generation Mobile Communication Technology, 5G) communication system, 6G communication system, satellite communication system, short-range communication system, narrowband Internet of Things system (Narrow Band-Internet of Things, NB-IoT), Global System for Mobile Communications (Global System for Mobile Communications, GSM), Enhanced Data rate for GSM Evolution (Enhanced Data rate for GSM Evolution, EDGE), Wideband Code Division Multiple Access (WCDMA), Code Division Multiple Access 2000 (Code Division Multiple Access, CDMA2000), Time Division-Synchronization Code Division Multiple Access (TD-SCDMA) and Long Term Evolution (LTE) and other communication systems. The embodiments of the present application do not limit the specific architecture of the wireless communication system.
[0067] In the above-mentioned application scenarios, any device (hereinafter referred to as the "target device" for the convenience of description) may use a machine learning model (hereinafter referred to as the "first machine learning model" for the convenience of description) in the process of sending data. The output of the aforementioned first machine learning model may include the allocation information of communication resources.
[0068] In one implementation, the steps of the training phase of the first machine learning model are executed on a device other than the target device (hereinafter referred to as the "first device" for ease of description), and only the steps of the inference phase of the first machine learning model are executed on the target device; in another implementation, the steps of the "training phase of the first machine learning model" and the "inference phase of the first machine learning model" can be executed on the first device. Among them, the "training phase of the first machine learning model" is the process of training the first machine learning model one or more times using training samples; the "inference phase of the first machine learning model" is the process of processing data using the first machine learning model. The process of iteratively training the first machine learning model is also the process of iteratively updating the parameters in the first machine learning model. After iteratively training the first machine learning model using training samples, the trained parameters corresponding to the first machine learning model can be obtained, and the aforementioned parameters obtained in the training phase will be used in the inference phase.
[0069] The data processing method provided in this application can be used to train the first machine learning model. For details, please refer to Figure 3 , Figure 3 A flow chart of a data processing method provided in an embodiment of the present application. Figure 3 As shown, 301, the first device obtains a set of parameter values corresponding to the first machine learning model.
[0070] 302. The first device trains a first machine learning model based on a set of parameter values, wherein the aforementioned set of parameter values includes the value of at least one hyperparameter used when training the first machine learning model, and the aforementioned set of parameter values is one of multiple sets of parameter values corresponding to the first machine learning model, and each set of parameter values in the multiple sets of parameter values corresponding to the first machine learning model may include the value of at least one hyperparameter used when training the first machine learning model, and the multiple sets of parameter values corresponding to the first machine learning model are stored on the first device or other devices.
[0071] The first device is an execution device for "training the first machine learning model". Optionally, if the steps of "training the first machine learning model" and "inference phase of the first machine learning model" can be performed on the first device, then illustratively, the first device can be any device with wireless communication function in the above multiple application scenarios; for example, the first device can be Figure 1 The network device 101 or the terminal device 102 in FIG. 101; for example, the first device may be Figure 2 or, the first device may also be other devices that need to have wireless communication functions, etc. The form of the first device is not limited in the embodiments of the present application.
[0072] Exemplarily, in one case, the first machine learning model is obtained by deep reinforcement learning, that is, the input of the first machine learning model is state information The output of the first machine learning model indicates an action corresponding to the aforementioned state information During one or more trainings of the first machine learning model, it is possible to calculate Execute under the indicated status The profit that can be generated by the indicated action By updating the parameters of the first machine learning model, the aforementioned benefits are maximized.
[0073] During the reasoning process of the first machine learning model, after a certain state information (hereinafter referred to as "target state information" for the convenience of description) is input into the first machine learning model, the optimal action information (hereinafter referred to as "target action information" for the convenience of description) output by the first machine learning model under the target state information can be determined based on the aforementioned target state information. The action pointed to by the target action information will be executed with a certain probability. The aforementioned probability can also be referred to as the utilization probability ε. The utilization probability is also a hyperparameter. An action will be randomly selected from other actions other than the action indicated by the target action information, and the aforementioned randomly selected action will be executed with a probability of 1-ε.
[0074] To facilitate understanding of the present solution, the present application embodiment takes the first machine learning model obtained by deep reinforcement learning as an example to introduce the specific implementation method of training the first machine learning model once. Step 302 may include: in any training of the first machine learning model, the first device may obtain a first training sample from the training data set, and update the parameters in the first machine learning model according to the first training sample; wherein the training data set may include one or more training samples, and the first training sample is any one of the training data set.
[0075] Exemplarily, a first machine learning model may be deployed on multiple devices in the wireless communication system, the first device is the kth device in the wireless communication system, and the first training sample may include in, represents the first training sample, Indicates the status information of the first device at the tth time point; Representatives in The action information finally selected under the indicated state will be When input into the first machine learning model, the first machine learning model will output the target action information. It can be determined based on target action information and utilization probability; For as well as The corresponding income information represents Execute under the indicated status The benefits of the indicated actions, It can be calculated based on the performance information at the t+1th time point; Represents the status information at the t+1th time point. It should be noted that examples of "status information", "action information" and "performance information" can be described in subsequent steps and will not be repeated here.
[0076] For example, in a training of the first machine learning model, the first device may update the parameters of the first machine learning model according to the first training sample to implement the state information. The Q value corresponding to each action under . For example, in one implementation, the formula used in the above updating process can be as follows:
[0077]
[0078] Among them, a k Represents the state information determined based on the first machine learning model Any one of the corresponding multiple action information, and Both represent Any action information a k The Q value, is the Q value before updating, is the updated Q value; α represents the learning rate, which is a hyperparameter.
[0079] Represents the income information in the first training sample, that is, Execute under the indicated status The indicated action is the benefit brought at the tth time point, γ represents the discount factor, which is a hyperparameter; Represents the state information of the first training sample at the t+1th time point.
[0080] Represents the status information before performing the update operation The largest Q value among the multiple Q values corresponding to the multiple action information, a *k Representatives in Indicated state a *k The Q value corresponding to the indicated action is the largest. is Execute a in the indicated state *k The maximum benefit that can be brought by the indicated action, It can be understood that based on the first training sample and the first machine learning model before updating Execute under the indicated status The action indicated is the one that brings the greatest benefit. Represents the first machine learning model based on the update. Indicated state The Q value corresponding to the indicated action can be understood as the Q value determined based on the first machine learning model before the update. Execute under the indicated status The maximum benefit that can be brought by the indicated action. It should be understood that the formula in formula (1) is only for the convenience of understanding this solution and is not used to limit this solution.
[0081] For another example, in another implementation, the formula used in the above updating process may be as follows:
[0082]
[0083] Among them, formula (2) can be understood in conjunction with formula (1). The parameters that have been explained in formula (1) will not be explained here. k Represents the state information determined based on the first machine learning model Any one of the corresponding multiple action information, represents the action information in the first training sample, when In the case of Yes Indicated state The Q value corresponding to the indicated action is updated.
[0084] exist If the then To update, the update process is to Updated to That is to say, β represents the penalty factor, which is a hyperparameter; if it is not satisfied Then it is not correct to update.
[0085] in, Represents the first machine learning model before updating. The largest Q value among the corresponding multiple Q values, Representatives in Indicated state The Q value corresponding to the indicated action is the largest; if Greater than t1, it means The value of is large enough, which means that Execute under the indicated status The indicated action brings a large enough benefit at the tth time point, and it can be inferred that Execute under the indicated status The action indicated is a sufficiently optimal strategy; therefore, t1 represents the first threshold value, which is used to measure the In the first training sample under the indicated state The first threshold value is a hyperparameter indicating the goodness of the action.
[0086] Less than t2 means Execute a in the indicated state k The indicated action is not the optimal strategy, but Execute a in the indicated state k The benefits of the indicated actions are the same as those of Execute a in the indicated state *k The difference between the benefits brought by the indicated action (i.e., the optimal action) is less than t2; therefore, t2 represents the second threshold value, which is used to measure Indicated state a k The action indicated (i.e. The second threshold value is a hyperparameter. It should be understood that the formula in formula (2) is only for the convenience of understanding this solution and is not used to limit this solution.
[0087] The first device may repeat the above steps multiple times to implement multiple training of the first machine learning model, that is, to implement multiple updates of the parameters of the first machine learning model. For example, if the first machine learning model is trained using formula (1), the hyperparameters used for each training of the first machine learning model may include utilization probability, learning rate, and discount factor; the above set of parameter values may include the values of any one or more of the following parameters: utilization probability, learning rate, or discount factor.
[0088] For another example, if the first machine learning model is trained using formula (2), the hyperparameters used for each training of the first machine learning model may include utilization probability, learning rate, discount factor, penalty factor, first threshold value, and second threshold value. The above set of parameter values may include parameter values of any one or more of the following parameters: utilization probability, learning rate, discount factor, penalty factor, first threshold value, or second threshold value.
[0089] In another case, the first machine learning model is obtained by deep learning, and the loss function can be used to train the first machine learning model each time. The hyperparameters used for each training of the first machine learning model may include the hyperparameters in the aforementioned loss function and the hyperparameters in other trainings.
[0090] It should be noted that the above examples are only for the convenience of understanding the present solution. If other types of hyperparameters can be used in each training of the first machine learning model, the above set of parameter values can also include values of other types of hyperparameters. The specific values can be determined in combination with the actual application scenario, and are not limited in the embodiments of the present application.
[0091] In an embodiment of the present application, multiple groups of parameter values corresponding to the machine learning model will be stored on the first device or other devices, and each group of parameter values in the multiple groups of parameter values includes the value of at least one hyperparameter used when training the machine learning model. The set of parameter values that can be obtained by the first device when training the machine learning model is one of the aforementioned multiple groups of parameter values, that is, the values of the hyperparameters used when training the machine learning model have more possibilities, thereby improving the flexibility when training the machine learning model.
[0092] Combined with the above description, the following is an introduction to the detailed implementation process of the training phase of the first machine learning model. Figure 4 , Figure 4 Another schematic diagram of the data processing method provided in the embodiment of the present application is as follows: Figure 4 As shown, the data processing method provided by the present application may include:
[0093] 401. The second device determines a first signaling, wherein the first signaling is used for the first device to determine a set of parameter values used when training a first machine learning model, the aforementioned set of parameter values is one set of multiple sets of parameter values corresponding to the first machine learning model, and the aforementioned set of parameter values includes the values of hyperparameters used when training the first machine learning model.
[0094] In the embodiment of the present application, step 401 is an optional step. The first device is a device for training the first machine learning model, and the second device is a device for determining which set of parameter values among multiple sets of parameter values corresponding to the first machine learning model the first device uses for training, that is, the first signaling generated by the second device is used for the first device to determine a set of parameter values (hereinafter referred to as "target group parameter values") to be used when training the first machine learning model.
[0095] For example, see Figure 1 and Figure 2In the network structure, the second device may be a base station, and the first device may be a terminal device; or, the second device and the first device may both be base stations; or, the second device may be a network element in the core network, and the first device may be a base station, and so on. The specific product forms of the "first device" and the "second device" may be flexibly determined in combination with the actual application scenarios, and are not limited in the embodiments of the present application.
[0096] Exemplarily, in one case, the second information may be deployed on the first device. The term "first information" will be used in the subsequent description. The concept of "first information" can refer to the subsequent description and will not be explained here. The second information may include multiple groups of parameter values corresponding to the first machine learning model and index information of each group of parameter values in the aforementioned multiple groups of parameter values. Each group of parameter values in the multiple groups of parameter values corresponding to the first machine learning model includes at least one hyperparameter value used when training the first machine learning model. The first signaling may carry index information of the target group parameter value. After obtaining the first signaling, the first device can obtain the target group parameter value (that is, the above-mentioned "a set of parameter values used when training the first machine learning model") from the second information according to the index information of the target group parameter value carried in the first signaling.
[0097] Exemplarily, the index information of each group of parameter values in the multiple groups of parameter values corresponding to the first machine learning model may include only numbers, or may include numbers and letters, or may be expressed in other forms, etc., which is not limited in the embodiments of the present application. In order to understand the present solution more intuitively, the following example is given in which the second information is expressed as a table.
[0098]
[0099] Table 1
[0100] In the above Table 1, 4 sets of parameter values corresponding to 4 index information are shown as an example. Each set of parameter values includes the following multiple parameter values: learning rate, discount factor, utilization probability, penalty factor, first threshold value and second threshold value. When the index information is 0, the corresponding set of parameter values may include α 1 , γ 1 , ε 1 , β 1 , as well as For the case where the index information is 1, 2 or 3, you can refer to the case where the index information is 0 for understanding, and will not go into details here. It should be understood that the second information can also include more or less information. The examples in Table 1 are only for the convenience of understanding this solution and are not used to limit this solution.
[0101] In another case, the first signaling may carry the above-mentioned target group parameter value corresponding to the first machine learning model. After obtaining the first signaling, the first device may directly obtain the target group parameter value from the first signaling.
[0102] Exemplarily, in one case, the first machine learning model is obtained by reinforcement learning. In one implementation, the hyperparameters used in each training of the first machine learning model include utilization probability, learning rate and discount factor. The above set of parameter values may include the values of any one or more of the following parameters: utilization probability, learning rate or discount factor value.
[0103] In another implementation, the hyperparameters used in each training of the first machine learning model include a learning rate, a discount factor, a utilization probability, a penalty factor, a first threshold value, and a second threshold value, the first threshold value is used to measure the quality of a first action in a first state, and the second threshold value is used to measure the quality of a second action other than the first action in the first state, and the second action is different from the first action; wherein, the first training sample includes information about the first state and the first action, and the first training sample is any training sample used when training the first machine learning model; the above set of parameters may include values of any one or more of the following parameters: learning rate, discount factor, utilization probability, penalty factor, first threshold value, or second threshold value.
[0104] In another case, the first machine learning model is obtained by deep learning, and the hyperparameters used when training the first machine learning model may include hyperparameters in the loss function. The above set of parameter values may include the value of at least one hyperparameter in the loss function.
[0105] The concepts of “first machine learning model”, “hyperparameters used when training the first machine learning model”, and “the process of each training of the first machine learning model” can all be combined with the above Figure 3 The description in the corresponding embodiment can be understood and will not be repeated here.
[0106] Optionally, the second device may acquire the first information before executing step 401, and step 401 may include: the second device determines the first signaling according to the first information.
[0107] Among them, in one case, if the first device not only executes the steps of the "training phase of the first machine learning model", but also executes the steps of the "inference phase of the first machine learning model", the first information includes any one or more of the following: the level of the first device, the type of the first device and / or the quality of service (QoS) level of the communication service corresponding to the training samples used in the training of the machine learning model. In another case, if the first device only executes the steps of the "training phase of the first machine learning model" and the target device is used to execute the steps of the "inference phase of the first machine learning model", the first information includes any one or more of the following: the level of the target device, the type of the target device and / or the QoS level of the communication service corresponding to the training samples used in the training of the first machine learning model.
[0108] For example, if the communication devices (which can be the first device or the target device) that perform the step of "the reasoning stage of the first machine learning model" are all base stations, the levels of the communication devices can be divided into macro base stations, micro base stations, micro-micro base stations, and home base stations, and the values of the hyperparameters used when training the first machine learning models on communication devices of different levels can be different. For another example, the levels of communication devices can be divided into primary user devices and secondary user devices. For another example, the levels of communication devices can be divided into cluster head node devices and non-cluster head node devices, etc.
[0109] For another example, if the communication devices (which can be the first device or the target device) that execute the steps of "the inference stage of the first machine learning model" are all terminal devices, the types of communication devices can be divided into vehicle-mounted devices, handheld devices and Internet of Things devices, and the values of the hyperparameters used when training the first machine learning models deployed on different types of communication devices are different. It should be noted that the "level of communication equipment" and "type of communication equipment" can also be in other forms. The examples given here are only to prove the feasibility of this solution and are not used to limit this solution.
[0110] The second device can obtain the first information in a variety of ways. In one implementation, the first information can be sent by the first device to the second device. For example, the first device is a terminal device, and the second device is a base station. When the terminal device accesses the cell where the base station is located, the level and / or type of the terminal device can be sent to the base station. For another example, the first device is a terminal device, and the second device is a base station. When the terminal device needs to obtain the target group parameter value (that is, the above-mentioned "a set of parameter values used when training the first machine learning model"), the terminal device can send the level of the terminal device, the type of the terminal device and / or the QoS level of the communication service corresponding to the training sample used when training the first machine learning model to the base station. For another example, the first device is a base station, the second device is a network element in the core network, and the target device is a terminal device. When the base station needs to obtain the target group parameter value (that is, the above-mentioned "a set of parameter values used when training the first machine learning model"), the base station sends the first information to the base station. The level and / or type of the target device, etc., can also be sent by the first device to the second device in other scenarios. The specific details can be flexibly determined in combination with the actual application scenario and are not limited here.
[0111] In another implementation method, the first information can also be sent by other devices to the second device. For example, the first device is a base station, and the second device is a network element in the core network. The level and / or type of the first device can be sent by other network elements in the core network to the second device, and so on. It should be noted that the examples of the second device obtaining the first information here are only to prove the feasibility of this solution and are not used to limit this solution.
[0112] Exemplarily, the second device may be deployed with third information. In one case, the first signaling carries index information of the target group parameter value (that is, the above-mentioned "a set of parameter values used when training the first machine learning model"). In one implementation, the third information may include multiple levels of communication devices and an index information corresponding to each level. Optionally, the third information may also include a set of parameter values pointed to by an index information corresponding to each level of communication devices. Step 401 may include: the second device obtains an index information corresponding to the level of the first device (or the level of the target device) from the third information, that is, the index information that determines the target group parameter value.
[0113] In order to understand the present solution more intuitively, an example is given below in which the third information is expressed in a table.
[0114]
[0115] Table 2
[0116] Table 2 shows four index information and four groups of parameter values corresponding to four levels of communication equipment as an example. Each group of parameter values includes the following multiple parameter values: learning rate, discount factor, utilization probability, penalty factor, first threshold value and second threshold value. When the level of the communication equipment is a macro base station, the index information corresponding to the level of "macro base station" is 0, and the group of parameter values pointed to by index information 0 may include α 1 , γ 1 , ε 1 , β 1 , as well as For the case where the level of the communication equipment is a micro base station, a pico base station or a home base station, you can refer to the case where the level of the communication equipment is a macro base station for understanding, and will not go into details here. It should be understood that the third information may also include more or less information. The examples in Table 2 are only for the convenience of understanding this solution and are not used to limit this solution.
[0117] In another implementation, the third information may include multiple categories of communication devices and an index information corresponding to each category. Optionally, the third information may also include a group of parameter values pointed to by an index information corresponding to each category of communication devices. Step 401 may include: the second device obtains an index information corresponding to the category of the first device (or the category of the target device) from the third information, that is, the index information that determines the target group parameter value.
[0118] In another implementation, the third information may include multiple QoS levels and an index information corresponding to each QoS level. Optionally, the third information may also include a group of parameter values pointed to by an index information corresponding to each QoS level. Step 401 may include: after the second device determines the QoS level of the communication service corresponding to the training sample used when training the first machine learning model one or more times (hereinafter referred to as the "target QoS level" for convenience of description), it can obtain an index information corresponding to the target QoS level from the third information, that is, the index information of the target group parameter value is determined.
[0119] Exemplarily, the first training sample (that is, any training sample used to train the first machine learning model) is collected when a communication device (such as a first device or a target device) performs a communication service using the first machine learning model. The communication service corresponding to the first training sample can be understood as the communication service completed when the action indicated by the first training sample is executed under the state indicated by the first training sample.
[0120] In order to understand the present solution more intuitively, an example is given below in which the third information is expressed in a table.
[0121]
[0122] Table 3
[0123] Among them, Table 3 shows three index information and three groups of parameter values corresponding to three QoS levels as an example. Each group of parameter values includes the values of the following multiple parameters: learning rate, discount factor, utilization probability, penalty factor, first threshold value and second threshold value. When the QoS level is 0, the index information corresponding to the level of "QoS level 0" is 0, and the group of parameter values pointed to by index information 0 may include α 1 , γ 1 , ε 1 , β 1 , as well as For the case of QoS level 1 or QoS level 2, you can refer to the case of QoS level 0 for understanding, and will not go into details here. It should be understood that the third information can also include more or less information. The examples in Table 3 are only for the convenience of understanding this solution and are not used to limit this solution.
[0124] In another case, the first signaling carries a target group parameter value (i.e., the above-mentioned "a set of parameter values used when training the first machine learning model"). In one implementation, the third information may include multiple levels of communication devices and a set of parameter values corresponding to each level. Optionally, the third information may also include index information of each set of parameter values. Step 401 may include: the second device obtains the target group parameter value corresponding to the level of the first device (or the level of the target device) from the third information.
[0125] In one implementation, the third information may include multiple types of communication devices and a set of parameter values corresponding to each type. Step 401 may include: the second device obtains a target group parameter value corresponding to the type of the first device (or the type of the target device) from the third information.
[0126] In another implementation, the third information may include multiple QoS levels and a set of parameter values corresponding to each QoS level. Step 401 may include: the second device may obtain a target group parameter value corresponding to a target QoS level from the third information. The concept of "target QoS level" may refer to the above description.
[0127] In an embodiment of the present application, since the value of at least one hyperparameter used in the training of the first machine learning model will affect the stability of the training process and the accuracy of the first machine learning model after training, and when the level and / or type of the first device is different, it means that the importance of the first device in the wireless communication network is different, and the QoS level of the communication service corresponding to the training sample used in the training of the first machine learning model is different, it means that the importance of using the aforementioned training samples to train the first machine learning model is different. Based on the level of the first device, the type of the first device and / or the QoS level of the communication service corresponding to the training sample used in the training of the first machine learning model, determining which set of parameter values to use to train the first machine learning model is beneficial to improving the adaptability between "a set of parameter values used when training the first machine learning model" and "the training process of the first machine learning model".
[0128] Optionally, the higher the level of the communication device (i.e., the first device or the target device) used to execute the "inference stage of the first machine learning model", the smaller the value of the first parameter in a set of parameter values, and the larger the value of the second parameter in a set of parameter values; or, the higher the target QoS level, the smaller the value of the first parameter in a set of parameter values, and the larger the value of the second parameter in a set of parameter values; the smaller the value of the first parameter in a set of parameter values, the larger the value of the second parameter in a set of parameter values.
[0129] The first parameter is a learning rate, and the second parameter includes any one or more of the following: a discount factor, a utilization probability, a penalty factor, a first threshold value, or a second threshold value.
[0130] Exemplarily, the larger the value of the learning rate, the greater the magnitude of the gradient descent during each training of the first machine learning model. Exemplarily, the greater the magnitude of the decrease in the Q value during each training of the first machine learning model, the lower the stability of the training phase of the first machine learning model. Optionally, the higher the level of the communication device used to perform the "inference phase of the first machine learning model", the smaller the value of the learning rate can be.
[0131] The larger the value of the discount factor, the greater the consideration of the benefits of non-current time points during each training of the first machine learning model, the higher the smoothness of the actions determined based on the trained first machine learning model, that is, the higher the smoothness of the benefits brought by multiple actions determined based on the trained first machine learning model. Optionally, the higher the level of the communication device used to perform the "inference phase of the first machine learning model", the larger the value of the discount factor can be.
[0132] The larger the value of the utilization probability, the more the first device or the target device tends to select the action indicated by the action information output by the first machine learning model when determining the final action to be performed, and the higher the stability of the final action to be performed. Optionally, the higher the level of the communication device used to perform the "reasoning phase of the first machine learning model", the larger the value of the utilization probability can be.
[0133] The larger the value of the penalty factor, that is, the larger the penalty value for one or more non-optimal action information under a certain state information, the easier it is for the first machine learning model to select the optimal action information under the aforementioned determined state information. Optionally, the higher the level of the communication device used to execute the "inference phase of the first machine learning model", the larger the value of the penalty factor can be.
[0134] The larger the value of the first threshold value, the higher the quality of the training samples that meet the screening condition corresponding to the first threshold value, so that the updating process of the parameters in the first machine learning model is more cautious, that is, the training process of the first machine learning model is more cautious, and thus the stability of the training process of the first machine learning model is higher. Optionally, the higher the level of the communication device used to perform the "inference phase of the first machine learning model", the larger the value of the first threshold value can be.
[0135] The larger the value of the second threshold value is, the fewer Q values that meet the screening condition corresponding to the second threshold value are, and the fewer Q values that need to be punished are, so that the updating process of the parameters in the first machine learning model is more cautious, and the training process of the first machine learning model is more stable. Optionally, the higher the level of the communication device used to perform the "inference phase of the first machine learning model", the larger the value of the second threshold value can be.
[0136] For example, in combination with the above Table 2, α 1 ≤α 2 ≤α 3 ≤α 4 , γ 1 ≥γ 2 ≥γ 3 ≥γ 4 , ε 1 ≥ε 2 ≥ε 3 ≥ε 4 , β 1 ≥β 2 ≥β 3 ≥β 4 , For another example, in combination with the above Table 3, α 1 ≤α 2 ≤α 3 , γ 1 ≥γ 2 ≥γ3 , ε 1 ≥ε 2 ≥ε 3 , β 1 ≥β 2 ≥β 3 , It should be understood that the examples given here are only for the convenience of understanding the present solution and are not intended to limit the present solution.
[0137] In an embodiment of the present application, for a higher level of the first device or a higher QoS level, a smaller learning rate, a larger discount factor, utilization probability, penalty factor, first threshold value and / or second threshold value will be used. That is, for a higher level of the first device or a higher QoS level, more emphasis will be placed on the stability of the training process, and more emphasis will be placed on the accuracy of the trained first machine learning model, which is conducive to improving the communication quality when "performing communication tasks based on the information output by the trained first machine learning model".
[0138] 402. The second device sends a first signaling to the first device.
[0139] In the embodiment of the present application, step 402 is an optional step. After determining the information carried in the first signaling, the second device may send the first signaling to the first device, and correspondingly, the first device may receive the first signaling.
[0140] Among them, if the second device is a base station and the first device is a terminal device, the first signaling is the signaling used by the base station and the terminal device for wireless communication. Exemplarily, the first signaling can be a radio resource control (RRC) signaling. Alternatively, if the second device and the first device are both base stations, the first signaling can be the signaling used for wireless communication between different base stations. Exemplarily, the first signaling can be Xn interface signaling. Alternatively, if the second device is a network element in the core network and the first device is a base station, the first signaling can be the signaling used when the network element of the core network and the base station perform wireless communication. Exemplarily, the first signaling can be NG interface signaling, etc. The first signaling can also use other types of signaling. The examples here are only to prove the feasibility of this solution and are not used to limit this solution.
[0141] To further understand the present solution, the specific expression form of the first signaling when the target group parameter value is carried in the RRC signaling and the Xn interface signaling is exemplified below.
[0142]
[0143] Disclosed above is a specific implementation method of carrying the target group parameter value in RRC signaling, where "INTEGER" represents an integer type. In the above example, the first threshold value is called the sample condition threshold value, and the second threshold value is called the policy addition threshold value. It should be understood that the above example is only for the convenience of understanding this solution and is not used to limit this solution.
[0144]
[0145] Table 4
[0146] Table 4 takes the example of carrying the target group parameter value in the Xn interface signaling, where IE / Group Name represents the names of multiple elements carried in the Xn interface signaling, IE type and reference represents the data type and related information of each element, "INTEGER" represents that the data type of the element is integer, Semantics description represents the description information of the element, Criticality represents the critical value of the element, and Assigned Criticality represents the designated criticality of the element. In Table 4, the first threshold value is called the sample conditional threshold value (Sample-Conditional-Threshold), and the second threshold value is called the policy addition threshold value (Policy-Conditional-Threshold) as an example. In Table 4, the learning rate (Learning Rate), discount factor (Discount-Factor), exploitation probability (Exploitation-Probability), sample conditional threshold value and policy addition threshold value carried in the Xn interface signaling are taken as examples. It should be understood that the examples in Table 4 are only for the convenience of understanding this solution and are not used to limit this solution.
[0147] The specific expression form of the first signaling when the index information of the target group parameter value is carried in the RRC signaling and the Xn interface signaling is exemplified as follows.
[0148] MARL-Config::=SEQUENCE{
[0149] Parameter-IndexINTEGER(0..15)
[0150] }
[0151] Among them, Parameter-Index represents the index information of the target group parameter value, and INTEGER represents that the data type is an integer. It should be understood that the above example is only for the convenience of understanding this solution and is not used to limit this solution.
[0152]
[0153] Table 5
[0154] Among them, Table 5 can be understood in combination with the above description of Table 3. The header part of Table 5 is the same as the header part of Table 4 and will not be repeated here. Parameter-Index represents the index information of the target group parameter value. It should be understood that the examples in Table 5 are only for the convenience of understanding this solution and are not used to limit this solution.
[0155] In an embodiment of the present application, multiple forms of expression of the first signaling are provided, which is conducive to making the present solution compatible with various communication scenarios of "a set of parameter values corresponding to a machine learning model".
[0156] 403. The first device obtains a set of parameter values corresponding to the first machine learning model.
[0157] In an embodiment of the present application, before the first device trains the first machine learning model one or more times, it is necessary to first obtain a set of parameter values corresponding to the first machine learning model, the aforementioned set of parameter values including the value of at least one hyperparameter used when training the first machine learning model one or more times, and the aforementioned set of parameter values is a set determined from multiple sets of parameter values corresponding to the first machine learning model.
[0158] In one implementation, if steps 401 and 402 are not performed, step 403 may include: the first device may store multiple groups of parameter values corresponding to the first machine learning model, and the first device determines the target group parameter values used when training the first machine learning model one or more times from the multiple groups of parameter values based on the first information. The specific implementation of the aforementioned steps and the meaning of "first information" can be found in the description of step 401, except that the second device in step 401 is replaced by the first device in step 403, which will not be repeated here.
[0159] In another implementation, if steps 401 and 402 are performed, step 403 may include: the first device determines a target group parameter value (i.e., a set of parameters used when training the first machine learning model) according to the first signaling.
[0160] Exemplarily, in one case, second information may be stored on the first device, and the first device may obtain the target group parameter value from the second information based on the index information of the target group parameter value carried in the first signaling. The meaning of the "second information" may refer to the description in step 401.
[0161] In another case, the first signaling carries a set of parameter values corresponding to the first machine learning model, and step 403 may include: the first device may obtain the target group parameter values from the first signaling.
[0162] In an embodiment of the present application, directly instructing the first device through signaling which set of parameter values to use when training the first machine learning model is beneficial to improving the efficiency of the parameter value transmission process and also beneficial to reducing the communication resources consumed during the parameter value transmission process.
[0163] A set of parameter values used in training the machine learning model can be directly carried in the first signaling, or index information of the aforementioned set of parameter values can be carried in the first signaling, providing different implementation schemes and improving the implementation flexibility of the present scheme.
[0164] 404. The first device trains the first machine learning model based on a set of parameter values, wherein the set of parameter values includes the value of at least one hyperparameter used when training the first machine learning model, the set of parameter values is one of multiple sets of parameter values corresponding to the first machine learning model, and the multiple sets of parameter values corresponding to the first machine learning model are stored on the first device or other devices.
[0165] In the embodiment of the present application, after the first device obtains a set of parameters used when training the first machine learning model, it can use the training samples in the training data set to train the first machine learning model once or multiple times. The specific implementation process of the first device training the first machine learning model each time can refer to the above Figure 3 The description in the corresponding embodiment is not repeated here. Among them, the training data set includes one or more training samples, and the first training sample in the training data set (that is, any training sample in the training data set) may include information on the state at the tth time point (for the sake of description, it may be referred to as the "first state" later), information on the first action, income information, and state information at the t+1th time point, and the aforementioned income information may be the income brought by executing the first action in the first state.
[0166] Optionally, when the first machine learning model is a reinforcement deep learning algorithm, the input of the first machine learning model may be state information, and the output of the first machine learning model may be action information. If the first machine learning model is used in the communication field, the action information may include allocation information of communication resources when performing a communication task under the state information input to the first machine learning model.
[0167] For example, the power control problem in the wireless network is solved by the first machine learning model, and the state information may include the channel state information from the data transmitter to the data receiver, the interference caused by the neighbor transmitter other than the data transmitter to the data transmitter and the data receiver, or other state information. The action information is used to indicate the transmission power of the data transmitter. Exemplarily, the action information may be one of multiple transmission power levels, and the aforementioned transmission power level can determine the final transmission power value adopted. The action information may also be a transmission power value. The benefit information can be determined based on the data throughput rate, packet loss rate, delay and / or other information statistically calculated after the action is executed.
[0168] For another example, the first machine learning model is used to solve the problem of bandwidth resource allocation in a wireless network. The state information may include channel state information from a data transmitter (e.g., a base station) to a data receiver, interference caused by neighbor transmitters other than the data transmitter to the data transmitter and the data receiver, and the amount of data to be sent by the data transmitter. The action information is used to indicate the bandwidth resources allocated to the data transmitter for data transmission. Exemplarily, the action information may be the number of resource blocks (RBs) or a specific bandwidth value in Hz. The revenue information may be determined based on statistical data throughput, packet loss rate, latency, and / or other information after performing the aforementioned action (i.e., sending data with the bandwidth resources corresponding to the aforementioned action).
[0169] For another example, taking the user scheduling problem in a wireless network as an example, the state information may include the channel state information from the data transmitter to the target user, the amount of data to be sent by the data transmitter, the cache state information of the data to be transmitted by the data transmitter, etc. The action information is used to indicate the result of user scheduling. For example, the action information may include a set of 0 / 1 sequences with the same number of elements as the number of users. The user corresponding to the position with a value of 1 in the sequence is scheduled to send data this time, and the user corresponding to the position with a value of 0 in the sequence cannot send data this time. The benefit information may include the data throughput rate, packet loss rate, delay and / or other information determined based on statistics after executing the aforementioned action (i.e., user scheduling based on the aforementioned action). It should be noted that the examples of "state information", "action information" and "benefit information" here are only for the convenience of understanding this solution and are not used to limit this solution.
[0170] In the embodiments of the present application, it is clarified what information is input into the first machine learning model and what information is output by the first machine learning model, that is, the specific functions of the first machine learning model are clarified, thereby improving the degree of integration between the first machine learning model and the specific application scenarios.
[0171] In which, the first device can perform one or more training operations on the first machine learning model, and each training operation performed by the first device on the first machine learning model may include training the first machine learning model one or more times.
[0172] For further understanding of this solution, please refer to Figure 5 , Figure 5 A schematic diagram provided in an embodiment of the present application for determining the relationship between a target group parameter value used when training a first machine learning model, a training phase of the first machine learning model, and an inference phase of the first machine learning model. Figure 5 In the example, the first device can perform both the steps of the "training phase of the first machine learning model" and the steps of the "inference phase of the first machine learning model", and the first device can perform multiple training operations on the first machine learning model, such as Figure 5 As shown, the first device performs one or more inferences (i.e. Figure 5 After the inference phase in the first machine learning model, the first machine learning model can be trained again (i.e. Figure 5 A training operation performed by the first device on the first machine learning model may include training the first machine learning model multiple times (i.e. Figure 5 multiple training stages included in ).
[0173] Continue reading Figure 5 After the first device determines the target group parameter value used when training the first machine learning model, the target group parameter value can be used when performing multiple training operations on the first machine learning model. That is, the relationship between the execution frequency of "determining the target group parameter value used when training the first machine learning model" and the execution frequency of "training operation of the first machine learning model" is not limited in this application. It should be understood that Figure 5 The examples are only for facilitating the understanding of this solution and are not intended to limit this solution.
[0174] Optionally, the set of parameter values used by the first device when training the first machine learning model is changeable. Figure 5As shown in the content, a set of parameter values used when training the first machine learning model can be periodically updated. Alternatively, a set of parameter values used when training the first machine learning model can be updated non-periodically. Exemplarily, if the communication device that executes the "inference phase of the first machine learning model" is a base station, and the base station is adjusted from a micro base station to a micro micro base station, the set of parameter values used when training the first machine learning model on the aforementioned device can be updated, etc., and the update of the "set of parameter values used when training the first machine learning model" can also be triggered in other scenarios. The examples here are only for the convenience of understanding this solution and are not used to limit the solution.
[0175] In another case, if the determining factors of a set of parameter values used in training the first machine learning model include QoS level, since one training operation performed on the first machine learning model may include multiple trainings of the first machine learning model, the training samples used in different trainings may be different, and the QoS levels corresponding to different training samples may also be different. Therefore, within one training operation performed on the first machine learning model, a set of parameter values used in the training of the first machine learning model can also be determined according to the QoS level corresponding to the training samples used each time, that is, within one training operation performed on the first machine learning model, a set of parameter values used in the training of the first machine learning model can also be updated.
[0176] In an embodiment of the present application, since the communication task to be completed by the communication device that performs the reasoning operation of the first machine learning model may change during use, or the importance of the device may also change, that is, the communication device that performs the reasoning operation of the first machine learning model may undergo various changes. Setting the set of parameter values used by the first device when training the first machine learning model to be changeable is conducive to making the "training process of the first machine learning model" more flexible, thereby making the "set of parameter values used when training the first machine learning model" and the "training process of the first machine learning model" more adapted.
[0177] exist Figures 1 to 5 On the basis of the corresponding embodiments, in order to better implement the above solutions of the embodiments of the present application, the following also provides related devices for implementing the above solutions. Figure 6 , Figure 6A structural schematic diagram of a data processing device provided in an embodiment of the present application, the data processing device 600 is applied to a first device, and the data processing device 600 includes: an acquisition module 601, used to obtain a set of parameter values corresponding to a first machine learning model; a training module 602, used to train the first machine learning model based on a set of parameter values, wherein a set of parameter values includes the value of at least one hyperparameter used when training the first machine learning model, a set of parameter values is one of multiple sets of parameter values corresponding to the first machine learning model, and the multiple sets of parameter values corresponding to the first machine learning model are stored on the first device or other devices.
[0178] Optionally, the data processing device 600 also includes: a receiving module 603, used to receive a first signaling; an acquisition module 601, specifically used to determine a set of parameter values corresponding to the first machine learning model based on the first signaling.
[0179] Optionally, the first device stores multiple groups of parameter values corresponding to the first machine learning model, and the first signaling carries index information of a group of parameter values; or, the first signaling carries a group of parameter values corresponding to the first machine learning model.
[0180] Optionally, the first signaling is NG interface signaling, Xn interface signaling or RRC signaling.
[0181] Optionally, the set of parameter values adopted by the first device is changeable.
[0182] Optionally, factors determining a set of parameter values among the multiple sets of parameter values include: the level of the first device, the type of the first device and / or the QoS level of the communication service corresponding to the training samples used in training the first machine learning model.
[0183] Optionally, the higher the level of the first device, the smaller the value of the first parameter in the set of parameter values, and the larger the value of the second parameter in the set of parameter values; or, the higher the QoS level of the communication service corresponding to the training sample used in training the first machine learning model, the smaller the value of the first parameter in the set of parameter values, and the larger the value of the second parameter in the set of parameter values;
[0184] Among them, the first parameter is the learning rate, and the second parameter includes any one or more of the following: a discount factor, a utilization probability, a penalty factor, a first threshold value or a second threshold value. The first threshold value is used to measure the pros and cons of a first action in a first state, and the second threshold value is used to measure the pros and cons of a second action other than the first action in the first state. The first training sample includes information about the first state and the first action, and the first training sample is any training sample used when training the first machine learning model.
[0185] Optionally, the input of the first machine learning model includes state information, and the output of the first machine learning model includes action information, and the action information includes allocation information of communication resources when performing a communication task under the state information.
[0186] It should be noted that the information interaction, execution process, etc. between the modules / units in the data processing device 600 are the same as those in the present application. Figures 3 to 5 The corresponding method embodiments are based on the same concept. For specific contents, please refer to the description in the method embodiments shown above in this application, which will not be repeated here.
[0187] See also Figure 7 , Figure 7 Another structural schematic diagram of a data processing device provided in an embodiment of the present application, the data processing device 700 is applied to a second device, and the data processing device 700 includes: a determination module 701, used to determine a first signaling; a sending module 702, used to send a first signaling to the first device, wherein the first signaling is used for the first device to determine a set of parameter values used when training a first machine learning model, the set of parameter values is one of multiple sets of parameter values corresponding to the first machine learning model, and the set of parameter values includes the value of a hyperparameter used when training the first machine learning model.
[0188] Optionally, the first signaling carries index information of a set of parameter values; or, the first signaling carries a set of parameter values corresponding to the first machine learning model.
[0189] Optionally, the determination module 701 is specifically used to determine the first signaling based on the first information, where the first information includes any one or more of the following: the level of the first device, the type of the first device and / or the QoS level of the communication service corresponding to the training samples used in training the first machine learning model.
[0190] It should be noted that the information interaction, execution process, etc. between the modules / units in the data processing device 700 are the same as those in the present application. Figures 3 to 5 The corresponding method embodiments are based on the same concept. For specific contents, please refer to the description in the method embodiments shown above in this application, which will not be repeated here.
[0191] See also Figure 8 , Figure 8 A schematic diagram of a structure of a device provided in an embodiment of the present application. The device 800 may specifically be a device used as a terminal device in the above embodiment. Figure 8 The example shown is that the terminal device is implemented by the terminal device (or a component in the terminal device). In a possible logical structure diagram of the device 800 , the device 800 may include but is not limited to at least one processor 801 and a communication port 802 .
[0192] Optionally, the device may further include at least one of a memory 803 and a bus 804 . In an embodiment of the present application, the at least one processor 801 is used to control and process the actions of the communication device 800 .
[0193] In addition, the processor 801 can be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a transistor logic device, a hardware component or any combination thereof. It can implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements a computing function, such as a combination of one or more microprocessors, a combination of a digital signal processor and a microprocessor, and the like. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described system, device and unit can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.
[0194] It should be noted that when the first device or the second device involved in the above embodiment is specifically a terminal device, Figure 8 The device 800 shown in the figure implements the steps implemented by the terminal device in the aforementioned method embodiment, Figure 8 The specific implementation methods of the device 800 shown in the figure to perform the aforementioned steps can all be referred to the description in the aforementioned method embodiment, and will not be described in detail here.
[0195] See also Fig. 9 , Fig. 9 Another structural diagram of a device provided in an embodiment of the present application. The device 900 may specifically be a device used as a network device in the above embodiment. Fig. 9 The example shown is that the network device is implemented by the network device (or a component in the network device); that is, when the first device or the second device involved in the above embodiment is specifically manifested as a network device, it can be adopted Fig. 9 Implementation of the device 900 shown in FIG. 1; illustratively, when the first device or the second device is a base station, it can be implemented by Fig. 9 The device 900 shown in FIG.
[0196] The structure of the communication device can refer to Fig. 9The structure shown. The device 900 includes at least one processor 911 and at least one network interface 912. Further optionally, the communication device also includes at least one memory 914, at least one transceiver 913 and one or more antennas 915. The processor 911, the transceiver 913, the memory 914 and the network interface 912 are connected, for example, through a bus. In an embodiment of the present application, the connection may include various interfaces, transmission lines or buses, etc., which are not limited in this embodiment. The antenna 915 is connected to the transceiver 913. The network interface 912 is used to enable the communication device to communicate with other communication devices through a communication link. For example, the network interface 912 may include a network interface between the communication device and the core network device, such as an S1 interface, and the network interface may include a network interface between the communication device and other communication devices (such as other network devices or core network devices), such as an X2 or Xn interface.
[0197] The processor 911 is mainly used to process the communication protocol and communication data, and to control the entire communication device, execute the software program, and process the data of the software program, for example, to support the communication device to perform the actions described in the embodiment. The communication device may include a baseband processor and a central processing unit. The baseband processor is mainly used to process the communication protocol and communication data, and the central processing unit is mainly used to control the entire terminal device, execute the software program, and process the data of the software program. Fig. 9 The processor 911 in the embodiment can integrate the functions of the baseband processor and the central processor. It can be understood by those skilled in the art that the baseband processor and the central processor can also be independent processors, which are interconnected through technologies such as buses. It can be understood by those skilled in the art that the terminal device can include multiple baseband processors to adapt to different network formats, and the terminal device can include multiple central processors to enhance its processing capabilities. The various components of the terminal device can be connected through various buses. The baseband processor can also be described as a baseband processing circuit or a baseband processing chip. The central processor can also be described as a central processing circuit or a central processing chip. The function of processing the communication protocol and the communication data can be built into the processor, or it can be stored in the memory in the form of a software program, and the processor executes the software program to realize the baseband processing function.
[0198] The memory is mainly used to store software programs and data. The memory 914 can exist independently and be connected to the processor 911. Optionally, the memory 914 can be integrated with the processor 911, for example, integrated into a chip. Among them, the memory 914 can store program codes for executing the technical solutions of the embodiments of the present application, and the execution is controlled by the processor 911. The various types of computer program codes executed can also be regarded as drivers of the processor 911.
[0199] Fig. 9Only one memory and one processor are shown. In an actual terminal device, there may be multiple processors and multiple memories. The memory may also be referred to as a storage medium or a storage device, etc. The memory may be a storage element on the same chip as the processor, i.e., an on-chip storage element, or an independent storage element, which is not limited in the embodiments of the present application.
[0200] The transceiver 913 can be used to support the reception or transmission of radio frequency signals between the communication device and the terminal, and the transceiver 913 can be connected to the antenna 915. The transceiver 913 includes a transmitter Tx and a receiver Rx. Specifically, one or more antennas 915 can receive radio frequency signals, and the receiver Rx of the transceiver 913 is used to receive the radio frequency signal from the antenna, convert the radio frequency signal into a digital baseband signal or a digital intermediate frequency signal, and provide the digital baseband signal or the digital intermediate frequency signal to the processor 911, so that the processor 911 further processes the digital baseband signal or the digital intermediate frequency signal, such as demodulation processing and decoding processing. In addition, the transmitter Tx in the transceiver 913 is also used to receive a modulated digital baseband signal or a digital intermediate frequency signal from the processor 911, and convert the modulated digital baseband signal or the digital intermediate frequency signal into a radio frequency signal, and send the radio frequency signal through one or more antennas 915. Specifically, the receiver Rx can selectively perform one or more stages of down-mixing and analog-to-digital conversion processing on the RF signal to obtain a digital baseband signal or a digital intermediate frequency signal, and the order of the down-mixing and analog-to-digital conversion processing is adjustable. The transmitter Tx can selectively perform one or more stages of up-mixing and digital-to-analog conversion processing on the modulated digital baseband signal or digital intermediate frequency signal to obtain a RF signal, and the order of the up-mixing and digital-to-analog conversion processing is adjustable. The digital baseband signal and the digital intermediate frequency signal can be collectively referred to as a digital signal.
[0201] The transceiver 913 may also be referred to as a transceiver unit, a transceiver, a transceiver device, etc. Optionally, a device in the transceiver unit for implementing a receiving function may be regarded as a receiving unit, and a device in the transceiver unit for implementing a sending function may be regarded as a sending unit, that is, the transceiver unit includes a receiving unit and a sending unit, the receiving unit may also be referred to as a receiver, an input port, a receiving circuit, etc., and the sending unit may be referred to as a transmitter, a transmitter, or a transmitting circuit, etc.
[0202] For example, when the first device or the second device involved in the above embodiment is specifically a base station, it can be adopted Fig. 9 The device 900 shown in the figure implements the steps implemented by the base station in the aforementioned method embodiment. Fig. 9 The specific implementation methods of the device 900 shown in the figure to perform the aforementioned steps can all be referred to the description in the aforementioned method embodiment, and will not be described in detail here.
[0203] The present application also provides a computer-readable storage medium in which a program for signal processing is stored. When the program is run on a computer, the computer executes the above-mentioned Figures 3 to 5 The steps performed by the first device in the method described in the embodiment shown, or causing the computer to perform the above Figures 3 to 5 The illustrated embodiment describes the steps performed by the second device in the method.
[0204] The present application also provides a computer program product which, when executed on a computer, enables the computer to execute the above Figures 3 to 5 The steps performed by the first device in the method described in the embodiment shown, or causing the computer to perform the above Figures 3 to 5 The illustrated embodiment describes the steps performed by the second device in the method.
[0205] The first device or the second device provided in the embodiment of the present application may be a chip, and the chip includes: a processing unit and a communication unit, the processing unit may be a processor, and the communication unit may be an input / output interface, a pin or a circuit, etc. The processing unit may execute the computer execution instructions stored in the storage unit to enable the chip to execute the above Figures 3 to 5 The data processing method described in the illustrated embodiment. Optionally, the storage unit is a storage unit in the chip, such as a register, a cache, etc. The storage unit may also be a storage unit located outside the chip in the wireless access device, such as a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), etc.
[0206] For details, please refer to Fig.10 , Fig.10 A schematic diagram of the structure of a chip provided in an embodiment of the present application, the chip can be expressed as a neural network processor NPU 100, NPU 100 is mounted on the host CPU (Host CPU) as a coprocessor, and the host CPU assigns tasks. The core part of the NPU is the operation circuit 1003, which is controlled by the controller 1004 to extract matrix data from the memory and perform multiplication operations.
[0207] In some implementations, the operation circuit 1003 includes multiple processing units (Process Engine, PE) inside. In some implementations, the operation circuit 1003 is a two-dimensional systolic array. The operation circuit 1003 can also be a one-dimensional systolic array or other electronic circuits capable of performing mathematical operations such as multiplication and addition. In some implementations, the operation circuit 1003 is a general-purpose matrix processor.
[0208] For example, assume there is an input matrix A, a weight matrix B, and an output matrix C. The operation circuit takes the corresponding data of matrix B from the weight memory 1002 and caches it on each PE in the operation circuit. The operation circuit takes the matrix A data from the input memory 1001 and performs matrix operation with matrix B, and the partial result or final result of the matrix is stored in the accumulator 1008.
[0209] The unified memory 1006 is used to store input data and output data. The weight data is directly transferred to the weight memory 1002 through the direct memory access controller (DMAC) 1005. The input data is also transferred to the unified memory 1006 through the DMAC.
[0210] BIU stands for Bus Interface Unit, i.e., bus interface unit 1010 , which is used for interaction between AXI bus, DMAC and instruction fetch buffer (IFB) 1009 .
[0211] The bus interface unit 1010 (Bus Interface Unit, BIU for short) is used for the instruction fetch memory 1009 to obtain instructions from the external memory, and is also used for the storage unit access controller 1005 to obtain the original data of the input matrix A or the weight matrix B from the external memory.
[0212] DMAC is mainly used to transfer input data in the external memory DDR to the unified memory 1006 or to transfer weight data to the weight memory 1002 or to transfer input data to the input memory 1001.
[0213] The vector calculation unit 1007 includes multiple operation processing units, and further processes the output of the operation circuit when necessary, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, etc. It is mainly used for non-convolutional / fully connected layer network calculations in neural networks, such as Batch Normalization, pixel-level summation, upsampling of feature planes, etc.
[0214] In some implementations, the vector calculation unit 1007 can store the processed output vector to the unified memory 1006. For example, the vector calculation unit 1007 can apply a linear function and / or a nonlinear function to the output of the operation circuit 1003, such as linear interpolation of the feature plane extracted by the convolution layer, and then, for example, a vector of accumulated values to generate an activation value. In some implementations, the vector calculation unit 1007 generates a normalized value, a pixel-level summed value, or both. In some implementations, the processed output vector can be used as an activation input to the operation circuit 1003, for example, for use in a subsequent layer in a neural network.
[0215] An instruction fetch buffer 1009 connected to the controller 1004 is used to store instructions used by the controller 1004;
[0216] Unified memory 1006, input memory 1001, weight memory 1002 and instruction fetch memory 1009 are all on-chip memories. External memories are private to the NPU hardware architecture.
[0217] Among them, the operations of each layer in the first machine learning model can be performed by the operation circuit 1003 or the vector calculation unit 1007.
[0218] The processor mentioned in any of the above places may be a general-purpose central processing unit, a microprocessor, an ASIC, or one or more integrated circuits for controlling the execution of the program of the above-mentioned first aspect method.
[0219] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.
[0220] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.
[0221] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.
[0222] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.
Claims
1. A data processing method, characterized in that: The method comprises: The first device obtains a set of parameter values corresponding to the machine learning model; The first device trains the machine learning model based on the set of parameter values, wherein the set of parameter values includes the value of at least one hyperparameter used when training the machine learning model, and the set of parameter values is one of multiple sets of parameter values corresponding to the machine learning model, and the multiple sets of parameter values corresponding to the machine learning model are stored on the first device or other devices.
2. The method according to claim 1, characterized in that Before the first device acquires a set of parameter values corresponding to the machine learning model, the method further includes: the first device receiving a first signaling; The first device obtains a set of parameter values corresponding to the machine learning model, including: The first device determines the set of parameter values corresponding to the machine learning model based on the first signaling.
3. The method according to claim 2, characterized in that The first device stores the multiple groups of parameter values corresponding to the machine learning model, and the first signaling carries index information of the group of parameter values; or, The first signaling carries the set of parameter values corresponding to the machine learning model.
4. The method according to claim 2 or 3, characterized in that: The first signaling is NG interface signaling, Xn interface signaling or radio resource control RRC signaling.
5. The method according to any one of claims 1 to 3, characterized in that: The set of parameter values used by the first device is changeable.
6. The method according to any one of claims 1 to 3, characterized in that: Factors determining one set of parameter values among the multiple sets of parameter values include: the level of the first device, the type of the first device and / or the quality of service QoS level of the communication service corresponding to the training samples used in training the machine learning model.
7. The method according to claim 6, characterized in that The higher the level of the first device, the smaller the value of the first parameter in the set of parameter values, and the larger the value of the second parameter in the set of parameter values; or, The higher the QoS level of the communication service corresponding to the training sample used in training the machine learning model, the smaller the value of the first parameter in the set of parameter values, and the larger the value of the second parameter in the set of parameter values; Among them, the first parameter is the learning rate, and the second parameter includes any one or more of the following: a discount factor, a utilization probability, a penalty factor, a first threshold value or a second threshold value, the first threshold value is used to measure the pros and cons of a first action in a first state, and the second threshold value is used to measure the pros and cons of a second action other than the first action in the first state, the first training sample includes information about the first state and the first action, and the first training sample is any training sample used when training the machine learning model.
8. The method according to any one of claims 1 to 3, characterized in that: The input of the machine learning model includes state information, and the output of the machine learning model includes action information, and the action information includes allocation information of communication resources when performing a communication task under the state information.
9. A data processing method, characterized in that: The method comprises: The second device determines the first signaling; The second device sends a first signaling to the first device, wherein the first signaling is used for the first device to determine a set of parameter values used when training a machine learning model, the set of parameter values being one of multiple sets of parameter values corresponding to the machine learning model, and the set of parameter values including values of hyperparameters used when training the machine learning model.
10. The method according to claim 9, characterized in that The first signaling carries index information of the set of parameter values; or, The first signaling carries the set of parameter values corresponding to the machine learning model.
11. The method according to claim 9 or 10, characterized in that: The second device determines the first signaling, including: the second device determines the first signaling based on first information, and the first information includes any one or more of the following: the level of the first device, the type of the first device and / or the service quality QoS level of the communication service corresponding to the training samples used in training the machine learning model.
12. A data processing device, characterized in that: The device is applied to a first device, and includes: An acquisition module, used to obtain a set of parameter values corresponding to the machine learning model; A training module, used to train the machine learning model based on the set of parameter values, wherein the set of parameter values includes the value of at least one hyperparameter used when training the machine learning model, and the set of parameter values is one of multiple sets of parameter values corresponding to the machine learning model, and the multiple sets of parameter values corresponding to the machine learning model are stored on the first device or other devices.
13. A data processing device, characterized in that: The device is applied to a second device, and includes: A determination module, configured to determine a first signaling; A sending module, used for sending a first signaling to a first device, wherein the first signaling is used for the first device to determine a set of parameter values used when training a machine learning model, the set of parameter values is one set of multiple sets of parameter values corresponding to the machine learning model, and the set of parameter values includes the values of hyperparameters used when training the machine learning model.
14. A device, characterized in that comprising a processor and a memory, the processor being coupled to the memory, The memory is used to store programs; The processor is configured to execute the program in the memory so that the device performs the method according to any one of claims 1 to 11.
15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program, and when the program is executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 11.
16. A computer program product, characterized in that The computer program product comprises a program, and when the program is run on a computer, the computer is caused to execute the method according to any one of claims 1 to 11.
