Model training method and device

Abstract

The embodiment of the invention provides a model training method and device. A source coding and decoding model and a channel coding and decoding model can be jointly trained. The method comprises the steps that a model training device obtains channel coding and decoding algorithm information and information source coding and decoding algorithm information, the channel coding and decoding algorithm information is used for indicating a model structure of a first coding and decoding model, and the information source coding and decoding algorithm information is used for indicating a model structure of a second coding and decoding model; the model training device carries out training based on the channel coding and decoding algorithm information and the information source coding and decoding algorithm information to obtain at least one of the weight of the first coding and decoding model and the weight of the second coding and decoding model.

Description

Technical Field

[0001] This application relates to the field of encoding and decoding technology, and more specifically, to model training methods and apparatus. Background Technology

[0002] In traditional codec systems, source coding and channel coding are designed as two separate modules. For example, the transmitter performs source coding and channel coding on the media data to obtain coded data, and then transmits the coded data through the channel; correspondingly, the receiver receives the coded data through the channel, and performs channel decoding and source decoding on the coded data to obtain the media data.

[0003] However, since source coding and decoding and channel coding and decoding are independent modules in the existing technology, and the mutual influence between source coding and decoding is not taken into account, there is still room for improvement in the performance of the coding and decoding system. Summary of the Invention

[0004] This application provides a model training method and apparatus that can jointly train a source coding / decoding model and a channel coding / decoding model, thereby improving the transmission performance of the system.

[0005] In a first aspect, embodiments of this application provide an encoding / decoding method, which may include: a model training device acquiring channel encoding / decoding algorithm information and source encoding / decoding algorithm information, wherein the channel encoding / decoding algorithm information is used to indicate the model structure of a first encoding / decoding model, and the source encoding / decoding algorithm information is used to indicate the model structure of a second encoding / decoding model, wherein the first encoding / decoding model includes at least one of a first encoding model or a first decoding model, the first encoding model is used by the channel encoding device to encode data output by the source encoding device, and the first decoding model is used by the channel decoding device to decode data output by the channel encoding device and transmitted through the channel, the second encoding / decoding model includes at least one of a second encoding model or a second decoding model, the second encoding model is used by the source encoding device to encode data input by the source encoding device, and the second decoding model is used by the source decoding device to decode data output by the channel decoding device; the model training device trains at least one of the weights of the first encoding / decoding model or the weights of the second encoding / decoding model based on the channel encoding / decoding algorithm information and the source encoding / decoding algorithm information.

[0006] The model training method and apparatus provided in this application can be used to jointly train the source coding and decoding model and the channel coding and decoding model, thereby improving the system performance.

[0007] Optionally, the specific form of the model training device is not limited in the embodiments of this application.

[0008] In one possible implementation, the model training device can be a source coding device or a source decoding device.

[0009] In another possible implementation, the model training device can be equipment associated with the source encoding and / or source decoding devices. For example, server equipment owned by the designer or manufacturer of the source encoding and / or source decoding devices.

[0010] In another possible implementation, the model training device can be a channel coding device or a channel decoding device.

[0011] In another possible implementation, the model training device can be equipment associated with the channel coding and / or channel decoding devices. For example, server equipment owned by the designer or manufacturer of the channel coding and / or channel decoding devices.

[0012] In another possible implementation, the model training device can be a third-party device. For example, a computing device or server device with model training capabilities.

[0013] Optionally, the model training device can obtain channel coding and decoding algorithm information in various ways, and the embodiments of this application do not limit this.

[0014] In one possible implementation, the model training device can receive channel coding and decoding algorithm information from channel-related devices (such as training devices for channel coding and decoding models).

[0015] In another possible implementation, if the channel coding / decoding algorithm information is used to indicate the model structure of the first coding model, the model training device can receive the channel coding / decoding algorithm information from the channel coding device.

[0016] In another possible implementation, if the channel coding / decoding algorithm information is used to indicate the model structure of the first decoding model, the model training device can receive the channel coding / decoding algorithm information from the channel decoding device.

[0017] In another possible implementation, if the channel coding / decoding algorithm information is used to indicate the structure of the first coding model and the model structure of the first decoding model, the model training device can receive the channel coding / decoding algorithm information from the channel coding device and the channel decoding device.

[0018] Optionally, the channel coding and decoding algorithm information can also be pre-agreed or standardized, and this application embodiment does not limit this.

[0019] Optionally, the channel coding and decoding algorithm information can indicate the model structure of the first coding and decoding model in various ways, and the embodiments of this application do not limit this.

[0020] In one possible implementation, the channel coding / decoding algorithm information may include a first model structure identifier, which indicates the model structure of the first coding / decoding model.

[0021] In another possible implementation, the channel coding / decoding algorithm information may include the model structure of the first coding / decoding model.

[0022] Optionally, the model structure described in the embodiments of this application (such as the model structure of the first encoding / decoding model) may include multiple layers.

[0023] Optionally, the layers described in the embodiments of this application may include, but are not limited to, the following layers: fully connected layers, convolutional layers, pooling layers, or deconvolutional layers.

[0024] Optionally, the model structure of the encoding / decoding model described in the embodiments of this application may include at least one of the model structure of the encoding model or the model structure of the decoding model.

[0025] Optionally, the model training device can obtain source encoding and decoding algorithm information in various ways, and the embodiments of this application do not limit this.

[0026] In one possible implementation, the model training device can receive source encoding / decoding algorithm information from source-related devices (such as training devices for source encoding / decoding models).

[0027] In another possible implementation, if the source coding / decoding algorithm information is used to indicate the model structure of the second coding model, the model training device can receive the source coding / decoding algorithm information from the source coding device.

[0028] In another possible implementation, if the source codec algorithm information is used to indicate the model structure of the second decoding model, the model training device can receive the source codec algorithm information from the source decoding device.

[0029] In another possible implementation, if the source encoding / decoding algorithm information is used to indicate the structure of the second encoding model and the model structure of the second decoding model, the model training device can receive the source encoding / decoding algorithm information from the source encoding device and the source decoding device.

[0030] Optionally, the source encoding and decoding algorithm information can also be pre-agreed or standardized, and this application embodiment does not limit this.

[0031] Optionally, the source encoding / decoding algorithm information can indicate the model structure of the second encoding / decoding model in various ways, and the embodiments of this application do not limit this.

[0032] In one possible implementation, the source codec algorithm information may include a second model structure identifier, which indicates the model structure of the second codec model.

[0033] In another possible implementation, the source encoding / decoding algorithm information may include the model structure of the second encoding / decoding model.

[0034] It should be noted that the model described in the embodiments of this application may include a model structure and weights corresponding to the model structure.

[0035] It should also be noted that the weights of the model described in this application embodiment may include the weights corresponding to the model structure or the weights corresponding to the sub-model structure. The sub-model structure may include any part of the layers in the model structure.

[0036] Optionally, this application embodiment does not limit the number of layers included in the sub-model structure, or the position and order of each layer in the model structure.

[0037] The aforementioned model training device trains at least one of the weights of the first codec model or the weights of the second codec model based on the channel codec algorithm information and the source codec algorithm information, and may include, but is not limited to, the following seven possible implementation methods.

[0038] In a first possible implementation, the channel coding / decoding algorithm information is further used to indicate the weights corresponding to the model structure of the first coding / decoding model. The model training device trains at least one of the weights of the first coding / decoding model or the weights of the second coding / decoding model based on the channel coding / decoding algorithm information and the source coding / decoding algorithm information. This may include: the model training device trains at the weights corresponding to the model structure of the second coding / decoding model based on the channel coding / decoding algorithm information and the source coding / decoding algorithm information.

[0039] In a second possible implementation, the source codec algorithm information is further used to indicate the weights corresponding to the model structure of the second codec model. The model training device trains at least one of the weights of the first codec model or the weights of the second codec model based on the channel codec algorithm information and the source codec algorithm information. This may include: the model training device trains the weights corresponding to the model structure of the first codec model based on the channel codec algorithm information and the source codec algorithm information.

[0040] In a third possible implementation, the model structure of the first codec model includes a first sub-model structure and a second sub-model structure. The channel codec algorithm information is also used to indicate the weights corresponding to the first sub-model structure of the first codec model. The model training device trains at least one of the weights of the first codec model or the weights of the second codec model based on the channel codec algorithm information and the source codec algorithm information. This may include: the model training device trains the weights corresponding to the second sub-model structure of the first codec model and the weights corresponding to the model structure of the second codec model based on the channel codec algorithm information and the source codec algorithm information.

[0041] In a fourth possible implementation, the model structure of the second codec model includes a first sub-model structure and a second sub-model structure. The source codec algorithm information is also used to indicate the weights corresponding to the first sub-model structure of the second codec model. The model training device trains at least one of the weights of the first codec model or the weights of the second codec model based on the channel codec algorithm information and the source codec algorithm information. This may include: the model training device trains the weights corresponding to the second sub-model structure of the second codec model and the weights corresponding to the model structure of the first codec model based on the channel codec algorithm information and the source codec algorithm information.

[0042] In a fifth possible implementation, the model structure of the first codec model includes a first substructure and a second substructure. The channel codec algorithm information is further used to indicate the weights corresponding to the first substructure of the first codec model. The model structure of the second codec model includes a first substructure and a second substructure. The source codec algorithm information is further used to indicate the weights corresponding to the first substructure of the second codec model. The model training device trains at least one of the weights of the first codec model or the weights of the second codec model based on the channel codec algorithm information and the source codec algorithm information. This may include: the model training device trains at least one of the weights corresponding to the second substructure of the second codec model and the weights corresponding to the second substructure of the first codec model based on the channel codec algorithm information and the source codec algorithm information.

[0043] In a sixth possible implementation, the model structure of the first codec model includes a first sub-model structure and a second sub-model structure. The channel codec algorithm information is further used to indicate the weights corresponding to the first sub-model structure of the first codec model, and the source codec algorithm information is further used to indicate the weights corresponding to the model structure of the second codec model. The model training device trains at least one of the weights of the first codec model or the weights of the second codec model based on the channel codec algorithm information and the source codec algorithm information. This may include: the model training device trains the weights corresponding to the second sub-model structure of the first codec model based on the channel codec algorithm information and the source codec algorithm information.

[0044] In the seventh possible implementation, the model structure of the second codec model includes a first sub-model structure and a second sub-model structure. The source codec algorithm information is further used to indicate the weights corresponding to the first sub-model structure of the second codec model, and the channel codec algorithm information is further used to indicate the weights corresponding to the model structure of the first codec model. The model training device trains at least one of the weights of the first codec model or the weights of the second codec model based on the channel codec algorithm information and the source codec algorithm information. This may include: the model training device trains the weights corresponding to the second sub-model structure of the second codec model based on the channel codec algorithm information and the source codec algorithm information.

[0045] Optionally, the method of receiving the auxiliary information is not limited in the embodiments of this application.

[0046] In one possible implementation, the model training device can receive the auxiliary information all at once.

[0047] In another possible implementation, the model training device can receive the auxiliary information in multiple stages. That is, the model training device receives only a portion of the auxiliary information each time until all the auxiliary information has been received.

[0048] Optionally, the sources of this auxiliary information are not limited in the embodiments of this application.

[0049] In one possible implementation, the model training device can receive the auxiliary information from the same device (or apparatus).

[0050] For example, the model training device can receive auxiliary information from the channel-related equipment.

[0051] In another possible implementation, the model training device can receive the auxiliary information from multiple devices (or apparatuses).

[0052] For example, the model training device can receive operational information from the channel coding device and the channel decoding device.

[0053] For example, the model training device can receive source data information from the source coding device.

[0054] For example, the model training device can receive channel information from the channel decoding device.

[0055] By employing the model training method provided in this application, since auxiliary information can more closely simulate the actual usage scenario of the encoding and decoding model, further combining auxiliary information to jointly train the source encoding and decoding model and the channel encoding and decoding model can improve the transmission performance of the system.

[0056] Optionally, the method may further include: the model training device sending the weights of the first codec model and / or the weights of the second codec model obtained through training.

[0057] Optionally, the model training device may directly or indirectly send the weights of the first codec model trained to the user of the first codec model through other devices; and / or directly or indirectly send the weights of the second codec model trained to the user of the second codec model through other devices.

[0058] Optionally, the method may further include: the model training device sending indication information, the indication information including at least one of the following: training completion indication information, used to indicate that the training of the first codec model and / or the second codec model has been completed; or, joint codec indication information, used to indicate that the channel codec device and the source codec device can perform joint codec; or, joint codec initiation indication information, used to initiate joint codec between the channel codec device and the source codec device; or, time indication information, used to indicate the initiation time of joint codec between the channel codec device and the source codec device.

[0059] Secondly, embodiments of this application provide a codec apparatus, which may include: an acquisition unit and a training unit. The acquisition unit is used to acquire channel codec algorithm information and source codec algorithm information. The channel codec algorithm information is used to indicate the model structure of a first codec model, and the source codec algorithm information is used to indicate the model structure of a second codec model. The first codec model includes at least one of a first encoding model or a first decoding model. The first encoding model is used by the channel encoding device to encode data output by the source encoding device, and the first decoding model is used by the channel decoding device to decode data output by the channel encoding device and transmitted through the channel. The second codec model includes at least one of a second encoding model or a second decoding model. The second encoding model is used by the source encoding device to encode data input by the source encoding device, and the second decoding model is used by the source decoding device to decode data output by the channel decoding device. The training unit is used to train at least one of the weights of the first codec model or the weights of the second codec model based on the channel codec algorithm information and the source codec algorithm information.

[0060] In one possible implementation, the channel coding / decoding algorithm information is also used to indicate the weights corresponding to the model structure of the first coding / decoding model. Specifically, the training unit is used to train the weights corresponding to the model structure of the second coding / decoding model based on the channel coding / decoding algorithm information and the source coding / decoding algorithm information.

[0061] In one possible implementation, the source codec algorithm information is also used to indicate the weights corresponding to the model structure of the second codec model. The training unit is specifically used to: train the weights corresponding to the model structure of the first codec model based on the channel codec algorithm information and the source codec algorithm information.

[0062] In one possible implementation, the model structure of the first codec model includes a first sub-model structure and a second sub-model structure. The channel codec algorithm information is also used to indicate the weights corresponding to the first sub-model structure of the first codec model. The training unit is specifically used to: train the weights corresponding to the second sub-model structure of the first codec model and the weights corresponding to the model structure of the second codec model based on the channel codec algorithm information and the source codec algorithm information.

[0063] In one possible implementation, the model structure of the second codec model includes a first sub-model structure and a second sub-model structure. The source codec algorithm information is also used to indicate the weights corresponding to the first sub-model structure of the second codec model. The training unit is specifically used to: train the weights corresponding to the second sub-model structure of the second codec model and the weights corresponding to the model structure of the first codec model based on the channel codec algorithm information and the source codec algorithm information.

[0064] In one possible implementation, the model structure of the first codec model includes a first sub-model structure and a second sub-model structure. The channel codec algorithm information is also used to indicate the weights corresponding to the first sub-model structure of the first codec model. The model structure of the second codec model includes a first sub-model structure and a second sub-model structure. The source codec algorithm information is also used to indicate the weights corresponding to the first sub-model structure of the second codec model. The training unit is specifically used to: train and obtain the weights corresponding to the second sub-model structure of the second codec model and the weights corresponding to the second sub-model structure of the first codec model based on the channel codec algorithm information and the source codec algorithm information.

[0065] In one possible implementation, the model structure of the first codec model includes a first sub-model structure and a second sub-model structure. The channel codec algorithm information is also used to indicate the weights corresponding to the first sub-model structure of the first codec model, and the source codec algorithm information is also used to indicate the weights corresponding to the model structure of the second codec model. The training unit is specifically used to: train and obtain the weights corresponding to the second sub-model structure of the first codec model based on the channel codec algorithm information and the source codec algorithm information.

[0066] In one possible implementation, the model structure of the second codec model includes a first sub-model structure and a second sub-model structure. The source codec algorithm information is also used to indicate the weights corresponding to the first sub-model structure of the second codec model, and the channel codec algorithm information is also used to indicate the weights corresponding to the model structure of the first codec model. The training unit is specifically used to: train and obtain the weights corresponding to the second sub-model structure of the second codec model based on the channel codec algorithm information and the source codec algorithm information.

[0067] In one possible implementation, the training unit is specifically used to: train at least one of the weights of the first encoding / decoding model or the weights of the second encoding / decoding model based on the channel encoding / decoding algorithm information, the source encoding / decoding algorithm information, and auxiliary information; wherein the auxiliary information includes at least one of the following: channel information, used to indicate the influence of the channel on data transmission; or, operation information, used to indicate at least one of the following operations: a first operation performed by the channel coding device before encoding the data input to the channel coding device, a second operation performed by the channel coding device after encoding the data input to the channel coding device, a third operation performed by the channel decoding device before decoding the data input to the channel decoding device, or a third operation performed by the channel decoding device before decoding the data input to the channel decoding device. The fourth operation performed after decoding; or, auxiliary data information, used to indicate at least one of the following auxiliary data: first auxiliary data generated between the source coding device and the channel coding device, second auxiliary data generated between the channel coding device and the channel, third auxiliary data generated between the channel and the channel decoding device, or fourth auxiliary data generated between the channel decoding device and the source decoding device, wherein the first auxiliary data is used to assist the channel coding device in performing the first operation, the second auxiliary data is used to assist the channel coding device in performing the second operation, the third auxiliary data is used to assist the channel decoding device in performing the third operation, and the fourth auxiliary data is used to assist the channel decoding device in performing the fourth operation; or, source data information, used to indicate the input data of the source coding device.

[0068] Optionally, the apparatus may further include a receiving unit, which is configured to receive the auxiliary information before the training unit trains at least one of the weights of the first codec model or the weights of the second codec model based on the channel codec algorithm information and the source codec algorithm information.

[0069] Optionally, the device 300 may further include: a transmitting unit, which is used to: transmit at least one of the weights of the first codec model or the weights of the second codec model obtained through training.

[0070] In one possible implementation, the transmitting unit is further configured to: transmit model training capability information before the acquiring unit acquires the channel coding and decoding algorithm information and the source coding and decoding algorithm information, wherein the model training capability information is used to indicate that the model training device has the ability to train a coding and decoding model.

[0071] Thirdly, this application also provides an encoding / decoding apparatus, comprising: a processor and a communication interface, the processor and the communication interface being coupled, the communication interface being used to provide information and / or data to the processor, and the processor being used to execute computer program instructions to perform the encoding / decoding methods provided in the above aspects or their various possible implementations.

[0072] Fourthly, this application also provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the methods provided by the foregoing aspects or any possible implementation thereof.

[0073] Fifthly, this application also provides a computer program product containing instructions that, when run on a computer, cause the computer to perform the methods provided by the foregoing aspects or any possible implementation thereof.

[0074] It is understood that any of the encoding / decoding devices, computer storage media, or computer program products provided above are used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here. Attached Figure Description

[0075] Figure 1 This is a schematic block diagram of the encoding / decoding system 100 provided in the embodiments of this application;

[0076] Figure 2 This is a schematic block diagram of the model training method 200 provided in the embodiments of this application;

[0077] Figure 3 This is a schematic block diagram of the model training device 300 provided in the embodiments of this application;

[0078] Figure 4 This is a schematic block diagram of the model training device 400 provided in the embodiments of this application. Detailed Implementation

[0079] The technical solutions in this application will now be described with reference to the accompanying drawings.

[0080] First, let me introduce the encoding and decoding system provided in the embodiments of this application.

[0081] Figure 1 A schematic diagram of an encoding / decoding system 100 provided in an embodiment of this application is shown. For example... Figure 1 As shown, the system 100 may include an encoding end 110 and a decoding end 120. The encoding end 110 may include a source coding device 111 and a channel coding device 112, and the decoding end 120 may include a channel decoding device 121 and a source decoding device 122.

[0082] The source coding device 111 is used to encode the input raw data (such as media data) based on the source coding model to obtain coded data 1; and to transmit the coded data 1 to the channel coding device 112.

[0083] The channel coding device 112 is used to encode the coded data 1 based on the channel coding model to obtain coded data 2; the coded data 2 is transmitted to the channel decoding device 121 through the channel to become coded data 2a.

[0084] The channel decoding device 121 is used to decode the encoded data 2a based on the channel decoding model to obtain the decoded data 1a; and to transmit the decoded data 1a to the source decoding device 122.

[0085] The source decoding device 122 is used to decode the decoded data 1a based on the source decoding model to obtain decoded data 3, which includes the original data.

[0086] Optionally, other devices may also exist between the source coding device 111 and the channel coding device 112, between the channel coding device 112 and the channel decoding device 121, and between the channel decoding device 121 and the source decoding device 122.

[0087] Understandably, encoded data 1a and encoded data 1 may be the same, or they may be different due to channel transmission and / or other processing therein; encoded data 2a and encoded data 2 may be the same, or they may be different due to channel transmission and / or other processing therein; decoded data 3 includes the original data and the original data may be the same, or they may be different due to channel transmission and / or other processing therein.

[0088] Optionally, each of the above-mentioned encoding devices may preprocess the input data before encoding it; and / or, each of the above-mentioned encoding devices may post-process the encoded data after encoding it. Optionally, the specific operations of the above-mentioned preprocessing can refer to the prior art, and will not be elaborated here.

[0089] Optionally, each of the above-mentioned decoding devices may preprocess the input data before decoding it; and / or, each of the above-mentioned decoding devices may post-process the decoded data after decoding it. Optionally, the specific operations of the above-mentioned post-processing can refer to the prior art, and will not be elaborated here.

[0090] Optionally, one or more of the above-mentioned source coding model (i.e., the second coding model), channel coding model (i.e., the first coding model), channel decoding model (i.e., the first decoding model), or source decoding model (i.e., the second decoding model) can be trained by the model training method provided in the embodiments of this application, which will be described in detail below.

[0091] Optionally, the specific forms of the source coding device 111, the channel coding device 112, the channel decoding device 121, and the source decoding device 122 are not limited in the embodiments of this application.

[0092] In one possible implementation, the source coding device 111 can be a standalone device A, the channel coding device 112 can be a standalone device B, and the channel decoding device 121 and the source decoding device 122 can be integrated into the device C as functional modules or chip devices.

[0093] For example, device A can be a video server, device B can be a base station, and device C can be a mobile phone. That is, the video server generates the media stream, which is then transmitted to the mobile phone via the base station for decoding and playback.

[0094] In another possible implementation, the source coding device 111 can be a standalone device D, the channel coding device 112 can be a standalone device E, the channel decoding device 121 can be a standalone device F, and the source decoding device 122 can be a standalone device G.

[0095] For example, device D can be a video server, device E can be a base station, device F can be a mobile phone, and device G can be a television. That is, the video server generates a media stream, the media stream is transmitted to the mobile phone via the base station, and the mobile phone, acting as a hotspot, then forwards the media stream to the television for decoding and playback.

[0096] In another possible implementation, the source coding device 111 and the channel coding device 112 can be integrated into device H as functional modules or chip devices, and the channel decoding device 121 and the source decoding device 122 can be integrated into device I as functional modules or chip devices.

[0097] For example, device H can be walkie-talkie A, and device I can be walkie-talkie B. That is, device A and device B perform D2D (device-to-device communication).

[0098] In one possible implementation, the encoding end 110 and the decoding end 120 can communicate via a channel.

[0099] Optionally, the technical solutions of the embodiments of this application can be applied to various communication systems. For example, Global System for Mobile Communications (GSM), Long Term Evolution (LTE) system, Universal Mobile Telecommunications System (UMTS), 4th generation (4G) mobile communication system, 4.5th generation (4.5G) mobile communication system, Worldwide Interoperability for Microwave Access (WiMAX) communication system, 5th generation (5G) mobile communication system, or new radio access technology (NR). As communication technology continues to develop, the technical solutions of the embodiments of this application can also be used in subsequent evolved communication systems, such as 6th generation (6G) mobile communication system, 7th generation (7G) mobile communication system, and so on. The technical solutions of this application embodiment can also be applied to Universal Mobile Telecommunications System (UMTS), Code Division Multiple Access (CDMA) system, Wireless Local Area Network (WLAN), Open RAN (O-RAN or ORAN), Cloud Radio Access Network (CRAN), etc.The technical solutions of this application embodiment can also be applied to vehicle-to-X (V2X), where V2X can include vehicle-to-network (V2N), vehicle-to-vehicle (V2V), vehicle-to-infrastructure (V2I), vehicle-to-pedestrian (V2P), Long Term Evolution-Vehicle (LTE-V), vehicle-to-everything (V2X), machine-type communication (MTC), Internet of Things (IoT), Long Term Evolution-Machine (LTE-M), machine-to-machine (M2M), etc.

[0100] In traditional codec systems, source coding and channel coding are designed as two separate modules. For example, the transmitter performs source coding and channel coding on the media data to obtain coded data, which is then transmitted through the channel. Similarly, source decoding and channel decoding are also designed as two separate modules. For example, the receiver receives the coded data through the channel and performs channel decoding and source decoding on the coded data to obtain the media data.

[0101] However, since source coding and decoding and channel coding and decoding are independent modules in the existing technology, and the mutual influence between source coding and decoding is not taken into account, there is still room for improvement in the transmission performance of the coding and decoding system.

[0102] This application provides a model training method and apparatus. The method may include: acquiring channel coding / decoding algorithm information and source coding / decoding algorithm information, wherein the channel coding / decoding algorithm information is used to indicate the model structure of a first coding / decoding model, and the source coding / decoding algorithm information is used to indicate the model structure of a second coding / decoding model. The first coding / decoding model includes at least one of a first encoding model or a first decoding model. The first encoding model is used by a channel coding device to encode data output by a source coding device, and the first decoding model is used by a channel decoding device to decode data output by the channel coding device and transmitted through a channel. The second coding / decoding model includes at least one of a second encoding model or a second decoding model, wherein the second encoding model is used by the source coding device to encode data input by the source coding device, and the second decoding model is used by the source decoding device to decode data output by the channel decoding device. Based on the channel coding / decoding algorithm information and the source coding / decoding algorithm information, at least one of the weights of the first coding / decoding model or the weights of the second coding / decoding model is trained. Using the model training method and apparatus provided in this application, a source coding / decoding model and a channel coding / decoding model can be trained jointly, thereby improving system performance.

[0103] The model training method provided in the embodiments of this application will be further described below with reference to the accompanying drawings.

[0104] Figure 2 A schematic flowchart of a model training method 200 provided in an embodiment of this application is shown. This method 200 can be executed by a model training device.

[0105] Optionally, the specific form of the model training device is not limited in the embodiments of this application.

[0106] In one possible implementation, the model training device can be the source coding device 111 or the source decoding device 122 in the system 100 described above.

[0107] In another possible implementation, the model training device can be a device associated with the source encoding device 111 and / or the source decoding device 122 in the system 100 described above. For example, the model training device can be a server device owned by the designer or manufacturer of the source encoding device 111 and / or the source decoding device 122 in the system 100 described above.

[0108] In another possible implementation, the model training device can be the channel coding device 112 or the channel decoding device 121 in system 100.

[0109] In another possible implementation, the model training device can be equipment related to the channel coding device 112 and / or the channel decoding device 121 in system 100. For example, the model training device can be server equipment owned by the designer or manufacturer of the channel coding device 112 and / or the channel decoding device 121 in system 100.

[0110] In another possible implementation, the model training device can be a third-party device other than the devices in system 100. For example, a computing device or server device with model training capabilities.

[0111] like Figure 2 As shown, method 200 may include steps S201 and S202. The steps of method 200 are described in detail below.

[0112] S201. The model training device acquires channel coding / decoding algorithm information and source coding / decoding algorithm information. The channel coding / decoding algorithm information is used to indicate the model structure of the first coding / decoding model, and the source coding / decoding algorithm information is used to indicate the model structure of the second coding / decoding model. The first coding / decoding model includes at least one of a first encoding model or a first decoding model. The first encoding model is used by the channel coding device to encode the data output by the source coding device, and the first decoding model is used by the channel decoding device to decode the data output by the channel coding device and transmitted through the channel. The second coding / decoding model includes at least one of a second encoding model or a second decoding model. The second encoding model is used by the source coding device to encode the data input by the source coding device, and the second decoding model is used by the source decoding device to decode the data output by the channel decoding device.

[0113] It should be noted that "A and / or B" or "at least one of A or B" mentioned in the embodiments of this application can include any of the following cases: (1) A; (2) B; (3) A and B.

[0114] It should also be noted that the encoding and decoding model (such as source encoding and decoding model or channel encoding and decoding model) described in the embodiments of this application may include any of the following: (1) encoding model; (2) decoding model; (3) encoding model and decoding model.

[0115] It should also be noted that the first encoding / decoding model described in the embodiments of this application can also be called a channel encoding / decoding model, wherein the first encoding model can also be called a channel coding model, and the first decoding model can also be called a channel decoding model; the second encoding / decoding model described in the embodiments of this application can also be called a source encoding / decoding model, wherein the second encoding model can also be called a source coding model, and the second decoding model can also be called a source decoding model.

[0116] Optionally, the model training device can obtain channel coding and decoding algorithm information in various ways, and the embodiments of this application do not limit this.

[0117] In one possible implementation, the model training device can receive channel coding and decoding algorithm information from channel-related devices (such as training devices for channel coding and decoding models).

[0118] In another possible implementation, if the channel coding / decoding algorithm information is used to indicate the model structure of the first coding model, the model training device can receive the channel coding / decoding algorithm information from the channel coding device.

[0119] In another possible implementation, if the channel coding / decoding algorithm information is used to indicate the model structure of the first decoding model, the model training device can receive the channel coding / decoding algorithm information from the channel decoding device.

[0120] In another possible implementation, if the channel coding / decoding algorithm information is used to indicate the structure of the first coding model and the model structure of the first decoding model, the model training device can receive the channel coding / decoding algorithm information from the channel coding device and the channel decoding device.

[0121] Optionally, the channel coding and decoding algorithm information can also be pre-agreed or standardized, and this application embodiment does not limit this.

[0122] Optionally, the channel coding and decoding algorithm information can indicate the model structure of the first coding and decoding model in various ways, and the embodiments of this application do not limit this.

[0123] In one possible implementation, the channel coding / decoding algorithm information may include a first model structure identifier, which indicates the model structure of the first coding / decoding model.

[0124] In another possible implementation, the channel coding / decoding algorithm information may include the model structure of the first coding / decoding model.

[0125] Optionally, the model structure described in the embodiments of this application (such as the model structure of the first encoding / decoding model) may include multiple layers.

[0126] Optionally, the layers described in the embodiments of this application may include, but are not limited to, the following layers: fully connected layers, convolutional layers, pooling layers, or deconvolutional layers.

[0127] Optionally, the model structure of the encoding / decoding model described in the embodiments of this application may include at least one of the model structure of the encoding model or the model structure of the decoding model.

[0128] Optionally, the model training device can obtain source encoding and decoding algorithm information in various ways, and the embodiments of this application do not limit this.

[0129] In one possible implementation, the model training device can receive source encoding / decoding algorithm information from source-related devices (such as training devices for source encoding / decoding models).

[0130] In another possible implementation, if the source coding / decoding algorithm information is used to indicate the model structure of the second coding model, the model training device can receive the source coding / decoding algorithm information from the source coding device.

[0131] In another possible implementation, if the source codec algorithm information is used to indicate the model structure of the second decoding model, the model training device can receive the source codec algorithm information from the source decoding device.

[0132] In another possible implementation, if the source encoding / decoding algorithm information is used to indicate the structure of the second encoding model and the model structure of the second decoding model, the model training device can receive the source encoding / decoding algorithm information from the source encoding device and the source decoding device.

[0133] Optionally, the source encoding and decoding algorithm information can also be pre-agreed or standardized, and this application embodiment does not limit this.

[0134] Optionally, the source encoding / decoding algorithm information can indicate the model structure of the second encoding / decoding model in various ways, and the embodiments of this application do not limit this.

[0135] In one possible implementation, the source codec algorithm information may include a second model structure identifier, which indicates the model structure of the second codec model.

[0136] In another possible implementation, the source encoding / decoding algorithm information may include the model structure of the second encoding / decoding model.

[0137] Optionally, if the model training device is a source coding device, a source decoding device, or a source-related device, before S201, the method 200 may further include: the model training device sending a channel coding and decoding algorithm information request message to the object providing the channel coding and decoding algorithm information (such as a channel-related device), the request message being used to request the object providing the channel coding and decoding algorithm information to send the channel coding and decoding algorithm information.

[0138] Optionally, if the model training device is a source coding device, a source decoding device, or a source-related device, before S201, the method 200 may further include: the model training device sending model training capability information to the provider of channel coding and decoding algorithm information, the model training capability information being used to indicate that the provider of channel coding and decoding algorithm information has the ability to train a coding and decoding model.

[0139] Optionally, if the model training device is a source coding device, a source decoding device, or a source-related device, before S201, the method 200 may further include: the model training device receiving model provision capability information from the provider of channel coding and decoding algorithm information, the model provision capability information being used to indicate that the provider of channel coding and decoding algorithm information has the ability to provide the model structure of the first coding and decoding model.

[0140] Optionally, if the model training device is a channel coding device, a channel decoding device, or a channel-related device, before S201, the method 200 may further include: the model training device sending a source coding and decoding algorithm information request message to the source coding and decoding algorithm information provider (such as a source-related device), the request message being used to request the source coding and decoding algorithm information provider to send the source coding and decoding algorithm information.

[0141] Optionally, if the model training device is a channel coding device, a channel decoding device, or a channel-related device, before S201, the method 200 may further include: the model training device sending model training capability information to the provider of the source coding and decoding algorithm information, the model training capability information being used to indicate that the provider of the source coding and decoding algorithm information has the ability to train a coding and decoding model.

[0142] Optionally, if the model training device is a channel coding device, a channel decoding device, or a channel-related device, before S201, the method 200 may further include: the model training device receiving model provision capability information from the source coding and decoding algorithm information provider, the model provision capability information being used to indicate that the source coding and decoding algorithm information provider has the capability to provide the model structure of the second coding and decoding model.

[0143] S202. The model training device trains at least one of the weights of the first codec model or the weights of the second codec model based on the channel codec algorithm information and the source codec algorithm information.

[0144] It should be noted that the model described in the embodiments of this application may include a model structure and weights corresponding to the model structure.

[0145] It should also be noted that the weights of the model described in this application embodiment may include the weights corresponding to the model structure or the weights corresponding to the sub-model structure. The sub-model structure may include any part of the layers in the model structure.

[0146] Optionally, this application embodiment does not limit the number of layers included in the sub-model structure, or the position and order of each layer in the model structure.

[0147] For example, the model structure of the first codec model may include layers 1, 2, 3 through 10, wherein the first sub-model structure may include layers 1, 2, 3, 5, and 8, and the second sub-model structure may include layers 4, 6, and 7. Correspondingly, the weights of the model structure may include the weights corresponding to layers 1 through 10, the weights corresponding to the first sub-model structure may include the weights corresponding to layers 1, 2, 3, 5, and 8, and the weights of the second sub-model structure may include the weights corresponding to layers 4, 6, and 7.

[0148] Alternatively, S202 may include, but is not limited to, the following seven possible implementations:

[0149] In a first possible implementation, the channel coding and decoding algorithm information is further used to indicate the weights corresponding to the model structure of the first coding and decoding model. S202 may include: the model training device trains the weights corresponding to the model structure of the second coding and decoding model based on the channel coding and decoding algorithm information and the source coding and decoding algorithm information.

[0150] In a second possible implementation, the source codec algorithm information is further used to indicate the weights corresponding to the model structure of the second codec model. S202 may include: the model training device trains the weights corresponding to the model structure of the first codec model based on the channel codec algorithm information and the source codec algorithm information.

[0151] In a third possible implementation, the model structure of the first codec model includes a first sub-model structure and a second sub-model structure. The channel codec algorithm information is also used to indicate the weights corresponding to the first sub-model structure of the first codec model. S202 may include: the model training device trains the weights corresponding to the second sub-model structure of the first codec model and the weights corresponding to the model structure of the second codec model based on the channel codec algorithm information and the source codec algorithm information.

[0152] In a fourth possible implementation, the model structure of the second codec model includes a first sub-model structure and a second sub-model structure. The source codec algorithm information is also used to indicate the weights corresponding to the first sub-model structure of the second codec model. S202 may include: the model training device trains the weights corresponding to the second sub-model structure of the second codec model and the weights corresponding to the model structure of the first codec model based on the channel codec algorithm information and the source codec algorithm information.

[0153] In a fifth possible implementation, the model structure of the first codec model includes a first substructure and a second substructure. The channel codec algorithm information is further used to indicate the weights corresponding to the first substructure of the first codec model. The model structure of the second codec model includes a first substructure and a second substructure. The source codec algorithm information is further used to indicate the weights corresponding to the first substructure of the second codec model. S202 may include: the model training device trains the weights corresponding to the second substructure of the second codec model and the weights corresponding to the second substructure of the first codec model based on the channel codec algorithm information and the source codec algorithm information.

[0154] In a sixth possible implementation, the model structure of the first codec model includes a first sub-model structure and a second sub-model structure. The channel codec algorithm information is further used to indicate the weights corresponding to the first sub-model structure of the first codec model, and the source codec algorithm information is further used to indicate the weights corresponding to the model structure of the second codec model. S202 may include: the model training device trains the weights corresponding to the second sub-model structure of the first codec model based on the channel codec algorithm information and the source codec algorithm information.

[0155] In the seventh possible implementation, the model structure of the second codec model includes a first sub-model structure and a second sub-model structure. The source codec algorithm information is also used to indicate the weights corresponding to the first sub-model structure of the second codec model, and the channel codec algorithm information is also used to indicate the weights corresponding to the model structure of the first codec model. S202 may include: the model training device trains the weights corresponding to the second sub-model structure of the second codec model based on the channel codec algorithm information and the source codec algorithm information.

[0156] Optionally, S202 may include: the model training device training at least one of the weights of the first codec model or the weights of the second codec model based on the channel codec algorithm information, the source codec algorithm information and auxiliary information.

[0157] Optionally, the auxiliary information may include at least one of the following:

[0158] Channel information, used to indicate the impact of the channel on data transmission; or,

[0159] Operation information is provided to indicate at least one of the following operations: a first operation performed by the channel coding device before encoding the data input to the channel coding device; a second operation performed by the channel coding device after encoding the data input to the channel coding device; a third operation performed by the channel decoding device before decoding the data input to the channel decoding device; or a fourth operation performed by the channel decoding device after decoding the data input to the channel decoding device; or,

[0160] Auxiliary data information is used to indicate at least one of the following auxiliary data: first auxiliary data generated between the source coding device and the channel coding device, second auxiliary data generated between the channel coding device and the channel, third auxiliary data generated between the channel and the channel decoding device, or fourth auxiliary data generated between the channel decoding device and the source decoding device, wherein the first auxiliary data is used to assist the channel coding device in performing the first operation, the second auxiliary data is used to assist the channel coding device in performing the second operation, the third auxiliary data is used to assist the channel decoding device in performing the third operation, and the fourth auxiliary data is used to assist the channel decoding device in performing the fourth operation; or...

[0161] Source data information is used to indicate the input data of the source encoding device.

[0162] It can be understood that the data input to the channel coding device refers to the data input to the channel coding device.

[0163] For example, the channel information may be used to indicate at least one of the following: the type of channel, such as an additive white Gaussian noise (AWGN) channel, a Rayleigh fading channel, etc.; a channel simulator, such as a channel simulator based on generative adversarial networks (GANs); and channel signal-to-noise ratio (SNR) information, such as the SNR range.

[0164] For example, the first operation or the second operation may include at least one of the following operations: segmentation, merging, encapsulation, unpacking, interleaving, modulation, cyclic redundancy check (CRC), etc.

[0165] For example, the auxiliary data information may include at least one of the following: first auxiliary data, data distribution characteristics of the first auxiliary data, generation method of the first auxiliary data, generator of the first auxiliary data, or header information required by the channel coding device when encapsulating packets.

[0166] For example, the source data information may include at least one of the following: the source data, the data distribution characteristics of the source data, the method of generating the source data, or the generator of the source data.

[0167] In one possible implementation, some or all of the auxiliary information can be combined with the channel coding and decoding algorithm information and presented together in the form of a software or application programming interface (API) for training the weights of the second coding and decoding model.

[0168] In another possible implementation, some or all of the information in the auxiliary information can be combined with the source encoding / decoding algorithm information and presented together in the form of software or API for training to obtain the weights of the first encoding / decoding model.

[0169] Optionally, the method of receiving the auxiliary information is not limited in the embodiments of this application.

[0170] In one possible implementation, the model training device can receive the auxiliary information all at once.

[0171] In another possible implementation, the model training device can receive the auxiliary information in multiple stages. That is, the model training device receives only a portion of the auxiliary information each time until all the auxiliary information has been received.

[0172] Optionally, the sources of this auxiliary information are not limited in the embodiments of this application.

[0173] In one possible implementation, the model training device can receive the auxiliary information from the same device (or apparatus).

[0174] For example, the model training device can receive auxiliary information from the channel-related equipment.

[0175] In another possible implementation, the model training device can receive the auxiliary information from multiple devices (or apparatuses).

[0176] For example, the model training device can receive operational information from the channel coding device and the channel decoding device.

[0177] For example, the model training device can receive source data information from the source coding device.

[0178] For example, the model training device can receive channel information from the channel decoding device.

[0179] Optionally, after S202, the method 200 may further include: the model training device sending the weights of the first codec model and / or the weights of the second codec model obtained from training.

[0180] Optionally, the model training device may directly or indirectly send the weights of the first codec model trained to the user of the first codec model through other devices; and / or directly or indirectly send the weights of the second codec model trained to the user of the second codec model through other devices.

[0181] For example, if the model training device is a third-party device, the first encoding model can be used by a channel coding device, the first decoding model can be used by a channel decoding device, the second encoding model can be used by a source coding device, and the second decoding model can be used by a source decoding device.

[0182] Optionally, if the model training device is a source coding device, after S202, the method 200 may further include: the source coding device encoding the input data based on the second coding model to obtain first coded data; and sending the first coded data to the channel coding device.

[0183] Optionally, if the model training device is a channel coding device, after S202, the method 200 may further include: the channel coding device encoding the input data based on the first coding model to obtain second coded data; and sending the second coded data to the channel decoding device through the channel.

[0184] Optionally, if the model training device is a channel decoding device, after S202, the method 200 may further include: the channel decoding device decoding the input data based on the first decoding model to obtain first decoded data; and sending the first decoded data to the source decoding device.

[0185] Optionally, if the model training device is a source decoding device, after S202, the method 200 may further include: the source decoding device decoding the input data based on the second decoding model to obtain second decoded data.

[0186] Optionally, after S202, the method 200 may further include: the model training device sending indication information, the indication information including at least one of the following: training completion indication information, used to indicate that the training of the first codec model and / or the second codec model has been completed; or, joint codec indication information, used to indicate that the channel codec device and the source codec device can perform joint codec; or, joint codec start indication information, used to start joint codec between the channel codec device and the source codec device; or, time indication information, used to indicate the start time of joint codec between the channel codec device and the source codec device.

[0187] In one possible implementation, the joint encoding and decoding between the channel encoding / decoding device and the source encoding / decoding device described in this application embodiment refers to the source encoding device encoding the input data using a trained second encoding model; and / or, the channel encoding device encoding the data output by the source encoding device using a trained first encoding model and transmitting it to the channel decoding device via the channel; and / or, the channel decoding device decoding the received data using a trained first decoding model; and / or, the source decoding device decoding the data output by the channel decoding device using a trained second decoding model.

[0188] In one possible implementation, taking the model training device as an example of a source-related device, the model training method provided in this application embodiment may include the following steps:

[0189] (1) The model training device sends model training capability information to the channel-related equipment. The model training capability information is used to indicate that the model training device has the ability to train the encoding and decoding model.

[0190] (2) The channel-related device sends model provision capability information to the model training device. The model provision capability information is used to indicate that the channel-related device has the ability to provide the model structure of the first encoding and decoding model. The structure of the first encoding and decoding model includes a first encoding model structure and a first decoding model structure. The first encoding model is used by the channel coding device to encode the data output by the source coding device, and the first decoding model is used by the channel decoding device to decode the data output by the channel coding device and transmitted through the channel.

[0191] (3) The model training device receives channel coding and decoding algorithm information from the channel-related device. The channel coding and decoding algorithm information is used to indicate the model structure of the first coding model and the model structure of the first decoding model.

[0192] (4) The model training device receives source coding and decoding algorithm information from the source coding device and the source decoding device. The source coding and decoding algorithm information is used to indicate the model structure of the second coding model and the model structure of the second decoding model. The second coding model is used by the source coding device to encode the data input to the source coding device, and the second decoding model is used by the source decoding device to decode the data output by the channel decoding device.

[0193] (5) The model training device trains the weights corresponding to the model structure of the first encoding model, the model structure of the first decoding model, the model structure of the second encoding model, the model structure of the second decoding model, and the auxiliary information to obtain the weights corresponding to the model structure of the first encoding model, the model structure of the first decoding model, the model structure of the second encoding model, and the model structure of the second decoding model.

[0194] (6) The model training device sends the second coding model to the source coding device and the second decoding model to the source decoding device;

[0195] (7) The model training device sends the first coding model to the channel coding device and the first decoding model to the channel decoding device;

[0196] (8) The model training device sends indication information to the source coding device, the source decoding device, the channel coding device, and the channel decoding device. The indication information includes at least one of the following: training completion indication information, used to indicate that the training of the first coding-decoding model and / or the second coding-decoding model has been completed; or, joint coding-decoding indication information, used to indicate that the channel coding-decoding device and the source coding-decoding device can perform joint coding-decoding; or, joint coding-decoding start indication information, used to start joint coding-decoding between the channel coding-decoding device and the source coding-decoding device; or, time indication information, used to indicate the start time of joint coding-decoding between the channel coding-decoding device and the source coding-decoding device.

[0197] Accordingly, the source coding device, the channel coding device, the channel decoding device, and the source decoding device perform joint coding and decoding based on the indication information, starting from the start time.

[0198] For example, the above joint encoding and decoding may include: the source coding device encoding the input data based on the second coding model to obtain coded data 1; sending the coded data 1 to the channel coding device; the channel coding device encoding the coded data 1 based on the first coding model to obtain coded data 2, and sending the coded data 2 to the channel decoding device through the channel; the channel decoding device decoding the coded data 2 based on the first decoding model to obtain decoded data 1, and sending the decoded data 1 to the source decoding device; the source decoding device decoding the decoded data 1 based on the second decoding model to obtain decoded data 2.

[0199] The above combination Figure 2 The model training method provided in the embodiments of this application has been introduced. The model training device provided in the embodiments of this application will be introduced below.

[0200] Figure 3 A schematic block diagram of the model training apparatus 300 provided in an embodiment of this application is shown. Figure 3 As shown, the device 300 may include an acquisition unit 301 and a training unit 302.

[0201] Optionally, the device 300 can be used in the model training device in the above method 200 embodiment. Further, the device 300 can be a virtual device formed by software executed by the processor or controller of the model training device in the above method 200 embodiment.

[0202] The acquisition unit 301 is used to acquire channel coding and decoding algorithm information and source coding and decoding algorithm information. The channel coding and decoding algorithm information is used to indicate the model structure of the first coding and decoding model, and the source coding and decoding algorithm information is used to indicate the model structure of the second coding and decoding model. The first coding and decoding model includes at least one of a first encoding model or a first decoding model. The first encoding model is used by the channel coding device to encode the data output by the source coding device, and the first decoding model is used by the channel decoding device to decode the data output by the channel coding device and transmitted through the channel. The second coding and decoding model includes at least one of a second encoding model or a second decoding model. The second encoding model is used by the source coding device to encode the data input by the source coding device, and the second decoding model is used by the source decoding device to decode the data output by the channel decoding device.

[0203] The training unit 302 is used to train at least one of the weights of the first codec model or the weights of the second codec model based on the channel codec algorithm information and the source codec algorithm information.

[0204] In one possible implementation, the channel coding / decoding algorithm information is also used to indicate the weights corresponding to the model structure of the first coding / decoding model. The training unit 302 is specifically used to: train the weights corresponding to the model structure of the second coding / decoding model based on the channel coding / decoding algorithm information and the source coding / decoding algorithm information.

[0205] In one possible implementation, the source codec algorithm information is also used to indicate the weights corresponding to the model structure of the second codec model. The training unit 302 is specifically used to: train the weights corresponding to the model structure of the first codec model based on the channel codec algorithm information and the source codec algorithm information.

[0206] In one possible implementation, the model structure of the first codec model includes a first sub-model structure and a second sub-model structure. The channel codec algorithm information is also used to indicate the weights corresponding to the first sub-model structure of the first codec model. The training unit 302 is specifically used to: train the weights corresponding to the second sub-model structure of the first codec model and the weights corresponding to the model structure of the second codec model based on the channel codec algorithm information and the source codec algorithm information.

[0207] In one possible implementation, the model structure of the second codec model includes a first sub-model structure and a second sub-model structure. The source codec algorithm information is also used to indicate the weights corresponding to the first sub-model structure of the second codec model. The training unit 302 is specifically used to: train the weights corresponding to the second sub-model structure of the second codec model and the weights corresponding to the model structure of the first codec model based on the channel codec algorithm information and the source codec algorithm information.

[0208] In one possible implementation, the model structure of the first codec model includes a first sub-model structure and a second sub-model structure. The channel codec algorithm information is also used to indicate the weights corresponding to the first sub-model structure of the first codec model. The model structure of the second codec model includes a first sub-model structure and a second sub-model structure. The source codec algorithm information is also used to indicate the weights corresponding to the first sub-model structure of the second codec model. The training unit 302 is specifically used to: train the weights corresponding to the second sub-model structure of the second codec model and the weights corresponding to the second sub-model structure of the first codec model based on the channel codec algorithm information and the source codec algorithm information.

[0209] In one possible implementation, the model structure of the first codec model includes a first sub-model structure and a second sub-model structure. The channel codec algorithm information is also used to indicate the weights corresponding to the first sub-model structure of the first codec model, and the source codec algorithm information is also used to indicate the weights corresponding to the model structure of the second codec model. The training unit 302 is specifically used to: train the weights corresponding to the second sub-model structure of the first codec model based on the channel codec algorithm information and the source codec algorithm information.

[0210] In one possible implementation, the model structure of the second codec model includes a first sub-model structure and a second sub-model structure. The source codec algorithm information is also used to indicate the weights corresponding to the first sub-model structure of the second codec model, and the channel codec algorithm information is also used to indicate the weights corresponding to the model structure of the first codec model. The training unit 302 is specifically used to: train the weights corresponding to the second sub-model structure of the second codec model based on the channel codec algorithm information and the source codec algorithm information.

[0211] In one possible implementation, the training unit 302 is specifically used to: train at least one of the weights of the first encoding / decoding model or the weights of the second encoding / decoding model based on the channel encoding / decoding algorithm information, the source encoding / decoding algorithm information, and auxiliary information; wherein the auxiliary information includes at least one of the following: channel information, used to indicate the influence of the channel on data transmission; or, operation information, used to indicate at least one of the following operations: a first operation performed by the channel coding device before encoding the data input to the channel coding device, a second operation performed by the channel coding device after encoding the data input to the channel coding device, a third operation performed by the channel decoding device before decoding the data input to the channel decoding device, or a third operation performed by the channel decoding device before decoding the data input to the channel decoding device. The fourth operation performed after decoding; or, auxiliary data information, used to indicate at least one of the following auxiliary data: first auxiliary data generated between the source coding device and the channel coding device, second auxiliary data generated between the channel coding device and the channel, third auxiliary data generated between the channel and the channel decoding device, or fourth auxiliary data generated between the channel decoding device and the source decoding device, wherein the first auxiliary data is used to assist the channel coding device in performing the first operation, the second auxiliary data is used to assist the channel coding device in performing the second operation, the third auxiliary data is used to assist the channel decoding device in performing the third operation, and the fourth auxiliary data is used to assist the channel decoding device in performing the fourth operation; or, source data information, used to indicate the input data of the source coding device.

[0212] Optionally, the device 300 may further include a receiving unit 303.

[0213] In one possible implementation, the receiving unit 303 is configured to receive the auxiliary information before the training unit 302 trains at least one of the weights of the first codec model or the weights of the second codec model based on the channel codec algorithm information and the source codec algorithm information.

[0214] Optionally, the device 300 may further include a transmitting unit 304.

[0215] In one possible implementation, the sending unit 304 is used to send at least one of the weights of the first codec model or the weights of the second codec model obtained through training.

[0216] In one possible implementation, the sending unit 304 is further configured to: send model training capability information before the acquisition unit 301 acquires the channel coding and decoding algorithm information and the source coding and decoding algorithm information, wherein the model training capability information is used to indicate that the model training device has the ability to train the coding and decoding model.

[0217] It should be noted that the information interaction and execution process between the above-mentioned devices are based on the same concept as the method 200 embodiment of this application. Their specific functions and technical effects can be found in the method embodiment section, and will not be repeated here. In an optional example, the device 300 can specifically be the model training device in the above-mentioned method 200 embodiment. The device 300 can be used to execute the various processes and / or steps corresponding to the model training device in the above-mentioned method 200 embodiment. To avoid repetition, these will not be repeated here.

[0218] Figure 3 One or more of the modules in the illustrated embodiments can be implemented by software, hardware, firmware, or a combination thereof. The software or firmware includes, but is not limited to, computer program instructions or code, and can be executed by a hardware processor. The hardware includes, but is not limited to, various integrated circuits such as a central processing unit (CPU), a digital signal processor (DSP), a field-programmable gate array (FPGA), or an application-specific integrated circuit (ASIC).

[0219] Figure 4A schematic block diagram of a model training apparatus 400 provided in an embodiment of this application is shown. The apparatus 400 may include a processor 401 and a communication interface 402, which are coupled together.

[0220] In an optional example, those skilled in the art will understand that the device 400 may specifically be the model training device in the above-described method 200 embodiments, and the device 400 may be the physical hardware structure of the model training device. The device 400 may be used to execute the various processes and / or steps corresponding to the model training device in the above-described method 200 embodiments, and will not be described again here to avoid repetition.

[0221] The processor 401 in this embodiment may include one or more processing units. Optionally, the processing unit may include, but is not limited to, a CPU, a general-purpose processor, a DSP, an ASIC, an FPGA, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor may be a microprocessor, a microcontroller, or any conventional processor.

[0222] For example, the processor 401 is used to acquire channel coding / decoding algorithm information and source coding / decoding algorithm information. The channel coding / decoding algorithm information is used to indicate the model structure of a first coding / decoding model, and the source coding / decoding algorithm information is used to indicate the model structure of a second coding / decoding model. The first coding / decoding model includes at least one of a first encoding model or a first decoding model. The first encoding model is used by the channel coding device to encode the data output by the source coding device, and the first decoding model is used by the channel decoding device to decode the data output by the channel coding device and transmitted through the channel. The second coding / decoding model includes at least one of a second encoding model or a second decoding model. The second encoding model is used by the source coding device to encode the data input by the source coding device, and the second decoding model is used by the source decoding device to decode the data output by the channel decoding device. Based on the channel coding / decoding algorithm information and the source coding / decoding algorithm information, at least one of the weights of the first coding / decoding model or the weights of the second coding / decoding model is trained.

[0223] Optionally, the device 400 may also include a memory 403.

[0224] Memory 403 can be volatile memory or non-volatile memory, or may include both. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DRRAM).

[0225] Specifically, memory 403 is used to store program code and instructions of device 400. Optionally, memory 403 is also used to store data obtained by processor 401 during the execution of the above method 200, such as channel encoding / decoding algorithm information or source encoding / decoding algorithm information.

[0226] Alternatively, the memory 403 may be a separate device or integrated into the processor 401.

[0227] It should be noted that, Figure 4 Only a simplified design of the device 400 is shown. In practical applications, the device 400 may also include other necessary components, including but not limited to any number of communication interfaces, processors, selectors, memories, etc., and all devices 400 that can implement this application are within the protection scope of this application.

[0228] In one possible design, the device 400 can be a chip. Optionally, the chip may further include one or more memories for storing computer-executable instructions. When the chip device is running, the processor can execute the computer-executable instructions stored in the memories to cause the chip to perform the steps performed by the model training device described in method 200 above.

[0229] Optionally, the chip device can be a field-programmable gate array, a dedicated integrated circuit, a system-on-a-chip, a central processing unit, a network processor, a digital signal processing circuit, a microcontroller, or a programmable controller or other integrated chip to implement the relevant functions.

[0230] This application also provides a computer-readable storage medium storing computer instructions that, when executed on a computer, implement the model training method described in the above method embodiments.

[0231] This application also provides a computer program product that, when run on a processor, implements the model training method described in the above method embodiments.

[0232] The model training device, computer-readable storage medium, computer program product or chip provided in the embodiments of this application are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects described in the corresponding methods provided above, and will not be repeated here.

[0233] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0234] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0235] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0236] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0237] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0238] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0239] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0240] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A model training method, characterized in that, include: The system acquires channel coding / decoding algorithm information and source coding / decoding algorithm information. The channel coding / decoding algorithm information is used to indicate the model structure of a first coding / decoding model, and the source coding / decoding algorithm information is used to indicate the model structure of a second coding / decoding model. The first coding / decoding model includes at least one of a first encoding model or a first decoding model. The first encoding model is used by the channel coding device to encode data output by the source coding device, and the first decoding model is used by the channel decoding device to decode data output by the channel coding device and transmitted through the channel. The second coding / decoding model includes at least one of a second encoding model or a second decoding model. The second encoding model is used by the source coding device to encode data input by the source coding device, and the second decoding model is used by the source decoding device to decode data output by the channel decoding device. Based on the channel coding and decoding algorithm information and the source coding and decoding algorithm information, at least one of the weights of the first coding and decoding model or the weights of the second coding and decoding model is trained.

2. The method according to claim 1, characterized in that, The channel coding / decoding algorithm information is further used to indicate the weights corresponding to the model structure of the first coding / decoding model. The step of training to obtain at least one of the weights of the first coding / decoding model or the weights of the second coding / decoding model based on the channel coding / decoding algorithm information and the source coding / decoding algorithm information includes: Based on the channel coding and decoding algorithm information and the source coding and decoding algorithm information, the weights corresponding to the model structure of the second coding and decoding model are trained.

3. The method according to claim 1, characterized in that, The source coding / decoding algorithm information is also used to indicate the weights corresponding to the model structure of the second coding / decoding model. The step of training to obtain at least one of the weights of the first coding / decoding model or the weights of the second coding / decoding model based on the channel coding / decoding algorithm information and the source coding / decoding algorithm information includes: Based on the channel coding and decoding algorithm information and the source coding and decoding algorithm information, the weights corresponding to the model structure of the first coding and decoding model are trained.

4. The method according to claim 1, characterized in that, The model structure of the first codec model includes a first sub-model structure and a second sub-model structure. The channel codec algorithm information is further used to indicate the weights corresponding to the first sub-model structure of the first codec model. The step of training at least one of the weights of the first codec model or the weights of the second codec model based on the channel codec algorithm information and the source codec algorithm information includes: Based on the channel coding and decoding algorithm information and the source coding and decoding algorithm information, the weights corresponding to the second sub-model structure of the first coding and decoding model and the weights corresponding to the model structure of the second coding and decoding model are trained.

5. The method according to claim 1, characterized in that, The model structure of the second codec model includes a first sub-model structure and a second sub-model structure. The source codec algorithm information is further used to indicate the weights corresponding to the first sub-model structure of the second codec model. The step of training at least one of the weights of the first codec model or the weights of the second codec model based on the channel codec algorithm information and the source codec algorithm information includes: Based on the channel coding and decoding algorithm information and the source coding and decoding algorithm information, the weights corresponding to the second sub-model structure of the second coding and decoding model and the weights corresponding to the model structure of the first coding and decoding model are trained.

6. The method according to claim 1, characterized in that, The model structure of the first codec model includes a first sub-model structure and a second sub-model structure. The channel codec algorithm information is further used to indicate the weights corresponding to the first sub-model structure of the first codec model. The model structure of the second codec model includes a first sub-model structure and a second sub-model structure. The source codec algorithm information is further used to indicate the weights corresponding to the first sub-model structure of the second codec model. The step of training at least one of the weights of the first codec model or the weights of the second codec model based on the channel codec algorithm information and the source codec algorithm information includes: Based on the channel coding and decoding algorithm information and the source coding and decoding algorithm information, the weights corresponding to the second sub-model structure of the second coding and decoding model and the weights corresponding to the second sub-model structure of the first coding and decoding model are trained.

7. The method according to claim 1, characterized in that, The model structure of the first codec model includes a first sub-model structure and a second sub-model structure. The channel codec algorithm information is further used to indicate the weights corresponding to the first sub-model structure of the first codec model, and the source codec algorithm information is further used to indicate the weights corresponding to the model structure of the second codec model. The step of training at least one of the weights of the first codec model or the weights of the second codec model based on the channel codec algorithm information and the source codec algorithm information includes: Based on the channel coding and decoding algorithm information and the source coding and decoding algorithm information, the weights corresponding to the second sub-model structure of the first coding and decoding model are trained.

8. The method according to claim 1, characterized in that, The model structure of the second codec model includes a first sub-model structure and a second sub-model structure. The source codec algorithm information is further used to indicate the weights corresponding to the first sub-model structure of the second codec model, and the channel codec algorithm information is further used to indicate the weights corresponding to the model structure of the first codec model. The step of training at least one of the weights of the first codec model or the weights of the second codec model based on the channel codec algorithm information and the source codec algorithm information includes: Based on the channel coding and decoding algorithm information and the source coding and decoding algorithm information, the weights corresponding to the second sub-model structure of the second coding and decoding model are trained.

9. The method according to any one of claims 1-8, characterized in that, The step of training at least one of the weights of the first encoding / decoding model or the weights of the second encoding / decoding model based on the channel encoding / decoding algorithm information and the source encoding / decoding algorithm information includes: Based on the channel coding and decoding algorithm information, the source coding and decoding algorithm information, and auxiliary information, at least one of the weights of the first coding and decoding model or the weights of the second coding and decoding model is trained. The auxiliary information includes at least one of the following: Channel information, used to indicate the impact of the channel on data transmission; or, Operation information is provided to indicate at least one of the following operations: a first operation performed by the channel coding device before encoding the data input to the channel coding device; a second operation performed by the channel coding device after encoding the data input to the channel coding device; a third operation performed by the channel decoding device before decoding the data input to the channel decoding device; or a fourth operation performed by the channel decoding device after decoding the data input to the channel decoding device; or, Auxiliary data information is used to indicate at least one of the following auxiliary data: first auxiliary data generated between the source coding device and the channel coding device, second auxiliary data generated between the channel coding device and the channel, third auxiliary data generated between the channel and the channel decoding device, or fourth auxiliary data generated between the channel decoding device and the source decoding device, wherein the first auxiliary data is used to assist the channel coding device in performing the first operation, the second auxiliary data is used to assist the channel coding device in performing the second operation, the third auxiliary data is used to assist the channel decoding device in performing the third operation, and the fourth auxiliary data is used to assist the channel decoding device in performing the fourth operation; or... Source data information is used to indicate the input data of the source encoding device.

10. The method according to claim 9, characterized in that, Before training at least one of the weights of the first codec model or the weights of the second codec model based on the channel codec algorithm information and the source codec algorithm information, the method further includes: Receive the auxiliary information.

11. The method according to any one of claims 1-10, characterized in that, The method further includes: Send at least one of the weights of the first codec model or the weights of the second codec model obtained from training.

12. The method according to any one of claims 1-11, characterized in that, Before acquiring the channel coding / decoding algorithm information and the source coding / decoding algorithm information, the method further includes: Send model training capability information, which indicates that the model training device has the ability to train encoding and decoding models.

13. A model training device, characterized in that, The device includes a processor and a communication interface, the processor being coupled to the communication interface, the processor being used for: The system acquires channel coding / decoding algorithm information and source coding / decoding algorithm information. The channel coding / decoding algorithm information is used to indicate the model structure of a first coding / decoding model, and the source coding / decoding algorithm information is used to indicate the model structure of a second coding / decoding model. The first coding / decoding model includes at least one of a first encoding model or a first decoding model. The first encoding model is used by the channel coding device to encode data output by the source coding device, and the first decoding model is used by the channel decoding device to decode data output by the channel coding device and transmitted through the channel. The second coding / decoding model includes at least one of a second encoding model or a second decoding model. The second encoding model is used by the source coding device to encode data input by the source coding device, and the second decoding model is used by the source decoding device to decode data output by the channel decoding device. Based on the channel coding and decoding algorithm information and the source coding and decoding algorithm information, at least one of the weights of the first coding and decoding model or the weights of the second coding and decoding model is trained.

14. The apparatus according to claim 13, characterized in that, The channel coding / decoding algorithm information is also used to indicate the weights corresponding to the model structure of the first coding / decoding model, and the processor is specifically used for: Based on the channel coding and decoding algorithm information and the source coding and decoding algorithm information, the weights corresponding to the model structure of the second coding and decoding model are trained.

15. The apparatus according to claim 13, characterized in that, The source encoding / decoding algorithm information is also used to indicate the weights corresponding to the model structure of the second encoding / decoding model, and the processor is specifically used for: Based on the channel coding and decoding algorithm information and the source coding and decoding algorithm information, the weights corresponding to the model structure of the first coding and decoding model are trained.

16. The apparatus according to claim 13, characterized in that, The model structure of the first codec model includes a first sub-model structure and a second sub-model structure. The channel codec algorithm information is also used to indicate the weights corresponding to the first sub-model structure of the first codec model. The processor is specifically used for: Based on the channel coding and decoding algorithm information and the source coding and decoding algorithm information, the weights corresponding to the second sub-model structure of the first coding and decoding model and the weights corresponding to the model structure of the second coding and decoding model are trained.

17. The apparatus according to claim 13, characterized in that, The model structure of the second codec model includes a first sub-model structure and a second sub-model structure. The source codec algorithm information is also used to indicate the weights corresponding to the first sub-model structure of the second codec model. The processor is specifically used for: Based on the channel coding and decoding algorithm information and the source coding and decoding algorithm information, the weights corresponding to the second sub-model structure of the second coding and decoding model and the weights corresponding to the model structure of the first coding and decoding model are trained.

18. The apparatus according to claim 13, characterized in that, The model structure of the first codec model includes a first sub-model structure and a second sub-model structure. The channel codec algorithm information is further used to indicate the weights corresponding to the first sub-model structure of the first codec model. The model structure of the second codec model includes a first sub-model structure and a second sub-model structure. The source codec algorithm information is further used to indicate the weights corresponding to the first sub-model structure of the second codec model. The processor is specifically used for: Based on the channel coding and decoding algorithm information and the source coding and decoding algorithm information, the weights corresponding to the second sub-model structure of the second coding and decoding model and the weights corresponding to the second sub-model structure of the first coding and decoding model are trained.

19. The apparatus according to claim 13, characterized in that, The model structure of the first codec model includes a first sub-model structure and a second sub-model structure. The channel codec algorithm information is further used to indicate the weights corresponding to the first sub-model structure of the first codec model, and the source codec algorithm information is further used to indicate the weights corresponding to the model structure of the second codec model. The processor is specifically used for: Based on the channel coding and decoding algorithm information and the source coding and decoding algorithm information, the weights corresponding to the second sub-model structure of the first coding and decoding model are trained.

20. The apparatus according to claim 13, characterized in that, The model structure of the second codec model includes a first sub-model structure and a second sub-model structure. The source codec algorithm information is further used to indicate the weights corresponding to the first sub-model structure of the second codec model, and the channel codec algorithm information is further used to indicate the weights corresponding to the model structure of the first codec model. The processor is specifically used for: Based on the channel coding and decoding algorithm information and the source coding and decoding algorithm information, the weights corresponding to the second sub-model structure of the second coding and decoding model are trained.

21. The apparatus according to any one of claims 13-20, characterized in that, The processor is specifically used for: Based on the channel coding and decoding algorithm information, the source coding and decoding algorithm information, and auxiliary information, at least one of the weights of the first coding and decoding model or the weights of the second coding and decoding model is trained. The auxiliary information includes at least one of the following: Channel information, used to indicate the impact of the channel on data transmission; or, Operation information is provided to indicate at least one of the following operations: a first operation performed by the channel coding device before encoding the data input to the channel coding device; a second operation performed by the channel coding device after encoding the data input to the channel coding device; a third operation performed by the channel decoding device before decoding the data input to the channel decoding device; or a fourth operation performed by the channel decoding device after decoding the data input to the channel decoding device; or, Auxiliary data information is used to indicate at least one of the following auxiliary data: first auxiliary data generated between the source coding device and the channel coding device, second auxiliary data generated between the channel coding device and the channel, third auxiliary data generated between the channel and the channel decoding device, or fourth auxiliary data generated between the channel decoding device and the source decoding device, wherein the first auxiliary data is used to assist the channel coding device in performing the first operation, the second auxiliary data is used to assist the channel coding device in performing the second operation, the third auxiliary data is used to assist the channel decoding device in performing the third operation, and the fourth auxiliary data is used to assist the channel decoding device in performing the fourth operation; or... Source data information is used to indicate the input data of the source encoding device.

22. The apparatus according to claim 21, characterized in that, The processor is also used for: Before training at least one of the weights of the first codec model or the weights of the second codec model based on the channel codec algorithm information and the source codec algorithm information, the auxiliary information is received through the communication interface.

23. The apparatus according to any one of claims 13-22, characterized in that, The processor is also used for: At least one of the weights of the first codec model or the weights of the second codec model obtained through training is sent through the communication interface.

24. The apparatus according to any one of claims 13-23, characterized in that, The processor is also used for: Before acquiring channel encoding / decoding algorithm information and source encoding / decoding algorithm information, model training capability information is sent through the communication interface. The model training capability information is used to indicate that the model training device has the ability to train encoding / decoding models.

25. A computer-readable storage medium, characterized in that, Used to store a computer program, which, when executed by a processor, implements the method as described in any one of claims 1-12.

26. A computer program product, characterized in that, When the computer program product is run on a processor, it implements the method as described in any one of claims 1-12.

Images