Model training method and apparatus

Abstract

Provided in the embodiments of the present application are a model training method and apparatus, by means of which a source encoding / decoding model and a channel encoding / decoding model can be jointly trained. The method comprises: a model training apparatus acquiring channel encoding / decoding algorithm information and source encoding / decoding algorithm information, wherein the channel encoding / decoding algorithm information is used for indicating the model structure of a first encoding / decoding model, and the source encoding / decoding algorithm information is used for indicating the model structure of a second encoding / decoding model; and on the basis of the channel encoding / decoding algorithm information and the source encoding / decoding algorithm information, the model training apparatus performing training to obtain at least one of the weight of the first encoding / decoding model and the weight of the second encoding / decoding model.

Description

Model training method and device

[0001] The present application claims priority to the Chinese Patent Application No. 202411120355.4, filed on August 14, 2024, and titled “Model training method and device”, the entire content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the field of coding technology, and more particularly, to a model training method and device. BACKGROUND

[0003] In a traditional coding system, source coding and channel coding are divided into two modules for independent design. For example, a sender respectively performs source coding and channel coding on media data to obtain coded data, and transmits the coded data through a channel; correspondingly, a receiver receives the coded data through the channel, respectively performs channel decoding and source decoding on the coded data, and obtains the media data.

[0004] However, since the source coding and the channel coding in the prior art are independent modules, the influence of the source coding and the channel coding on each other is not considered, and therefore, there is room for improvement in the performance of the coding system. SUMMARY

[0005] Embodiments of the present application provide a model training method and device, which can jointly train a source coding model and a channel coding model, thereby improving the transmission performance of the system.

[0006] In a first aspect, embodiments of the present application provide a coding method, which can include: a model training device obtaining channel coding algorithm information and source coding algorithm information, the channel coding algorithm information being used to indicate a model structure of a first coding model, the source coding algorithm information being used to indicate a model structure of a second coding model, wherein the first coding model includes at least one of a first encoding model or a first decoding model, the first encoding model being used for a channel coding device to code data output by a source coding device, the first decoding model being used for a channel decoding device to decode data output by the channel coding device and transmitted through a channel, the second coding model including at least one of a second encoding model or a second decoding model, the second encoding model being used for the source coding device to code data input by the source coding device, the second decoding model being used for a source decoding device to decode data output by the channel decoding device; the model training device training at least one of a weight of the first coding model or a weight of the second coding model based on the channel coding algorithm information and the source coding algorithm information.

[0007] The model training method and apparatus provided in this application can be used to jointly train the source coding and decoding model and the channel coding and decoding model, thereby improving the system performance.

[0008] Optionally, the specific form of the model training device is not limited in the embodiments of this application.

[0009] In one possible implementation, the model training device can be a source coding device or a source decoding device.

[0010] In another possible implementation, the model training device can be equipment associated with the source encoding and / or source decoding devices. For example, server equipment owned by the designer or manufacturer of the source encoding and / or source decoding devices.

[0011] In another possible implementation, the model training device can be a channel coding device or a channel decoding device.

[0012] In another possible implementation, the model training device can be equipment associated with the channel coding and / or channel decoding devices. For example, server equipment owned by the designer or manufacturer of the channel coding and / or channel decoding devices.

[0013] In another possible implementation, the model training device can be a third-party device. For example, a computing device or server device with model training capabilities.

[0014] Optionally, the model training device can obtain channel coding and decoding algorithm information in various ways, and the embodiments of this application do not limit this.

[0015] In one possible implementation, the model training device can receive channel coding and decoding algorithm information from channel-related devices (such as training devices for channel coding and decoding models).

[0016] In another possible implementation, if the channel coding / decoding algorithm information is used to indicate the model structure of the first coding model, the model training device can receive the channel coding / decoding algorithm information from the channel coding device.

[0017] In another possible implementation, if the channel coding / decoding algorithm information is used to indicate the model structure of the first decoding model, the model training device can receive the channel coding / decoding algorithm information from the channel decoding device.

[0018] In another possible implementation, if the channel coding / decoding algorithm information is used to indicate the structure of the first coding model and the model structure of the first decoding model, the model training device can receive the channel coding / decoding algorithm information from the channel coding device and the channel decoding device.

[0019] Optionally, the channel coding algorithm information can also be pre-agreed or standardized, and embodiments of the present application do not limit this.

[0020] Optionally, the channel coding algorithm information can indicate the model structure of the first coding model in multiple ways, and embodiments of the present application do not limit this.

[0021] In a possible implementation, the channel coding algorithm information can include a first model structure identifier, which is used to indicate the model structure of the first coding model.

[0022] In another possible implementation, the channel coding algorithm information can include the model structure of the first coding model.

[0023] Optionally, the model structure (such as the model structure of the first coding model) described in embodiments of the present application can include multiple layers.

[0024] Optionally, the layers described in embodiments of the present application can include but are not limited to the following layers: a fully connected layer, a convolutional layer, a pooling layer, or a deconvolutional layer, etc.

[0025] Optionally, the model structure of the coding model described in embodiments of the present application can include at least one of the model structure of the encoding model or the model structure of the decoding model.

[0026] Optionally, the model training apparatus can obtain the source coding algorithm information in multiple ways, and embodiments of the present application do not limit this.

[0027] In a possible implementation, the model training apparatus can receive the source coding algorithm information from a source-related device (such as a training device of a source coding model).

[0028] In another possible implementation, if the source coding algorithm information is used to indicate the model structure of the second encoding model, the model training apparatus can receive the source coding algorithm information from a source encoding apparatus.

[0029] In yet another possible implementation, if the source coding algorithm information is used to indicate the model structure of the second decoding model, the model training apparatus can receive the source coding algorithm information from a source decoding apparatus.

[0030] In yet another possible implementation, if the source coding algorithm information is used to indicate the model structure of the second encoding model and the model structure of the second decoding model, the model training apparatus can receive the source coding algorithm information from the source encoding apparatus and the source decoding apparatus.

[0031] Optionally, the source codec algorithm information can also be pre-agreed or standardized, and embodiments of the present application do not limit this.

[0032] Optionally, the source codec algorithm information can indicate the model structure of the second codec model in various ways, and embodiments of the present application do not limit this.

[0033] In a possible implementation, the source codec algorithm information can include a second model structure identifier, which is used to indicate the model structure of the second codec model.

[0034] In another possible implementation, the source codec algorithm information can include the model structure of the second codec model.

[0035] It should be noted that the model described in embodiments of the present application can include a model structure and weights corresponding to the model structure.

[0036] It should also be noted that the weights of the model described in embodiments of the present application can include weights corresponding to the model structure or weights corresponding to a sub-model structure. The sub-model structure can include any partial layer in the model structure.

[0037] Optionally, embodiments of the present application do not limit the number of layers included in the sub-model structure, and the position and order of each layer in the model structure.

[0038] The above model training apparatus trains at least one of the weights of the first codec model or the weights of the second codec model based on the channel codec algorithm information and the source codec algorithm information, which can include but is not limited to the following seven possible implementations.

[0039] In a first possible implementation, the channel codec algorithm information is also used to indicate the weights corresponding to the model structure of the first codec model, and the model training apparatus trains at least one of the weights of the first codec model or the weights of the second codec model based on the channel codec algorithm information and the source codec algorithm information can include: the model training apparatus trains the weights corresponding to the model structure of the second codec model based on the channel codec algorithm information and the source codec algorithm information.

[0040] In a second possible implementation, the source codec algorithm information is further used to indicate weights corresponding to a model structure of the second codec model, and the model training apparatus trains at least one of the weights of the first codec model or the weights of the second codec model based on the channel codec algorithm information and the source codec algorithm information can include that the model training apparatus trains weights corresponding to a model structure of the first codec model based on the channel codec algorithm information and the source codec algorithm information.

[0041] In a third possible implementation, the model structure of the first codec model includes a first sub-model structure and a second sub-model structure, the channel codec algorithm information is further used to indicate weights corresponding to the first sub-model structure of the first codec model, and the model training apparatus trains at least one of the weights of the first codec model or the weights of the second codec model based on the channel codec algorithm information and the source codec algorithm information can include that the model training apparatus trains weights corresponding to the second sub-model structure of the first codec model and weights corresponding to a model structure of the second codec model based on the channel codec algorithm information and the source codec algorithm information.

[0042] In a fourth possible implementation, the model structure of the second codec model includes a first sub-model structure and a second sub-model structure, the source codec algorithm information is further used to indicate weights corresponding to the first sub-model structure of the second codec model, and the model training apparatus trains at least one of the weights of the first codec model or the weights of the second codec model based on the channel codec algorithm information and the source codec algorithm information can include that the model training apparatus trains weights corresponding to the second sub-model structure of the second codec model and weights corresponding to a model structure of the first codec model based on the channel codec algorithm information and the source codec algorithm information.

[0043] In a fifth possible implementation, the model structure of the first codec model comprises a first sub-structure and a second sub-structure, the channel codec algorithm information is further used for indicating the weight corresponding to the first sub-structure of the first codec model, the model structure of the second codec model comprises a first sub-structure and a second sub-structure, the source codec algorithm information is further used for indicating the weight corresponding to the first sub-structure of the second codec model, and the model training apparatus training at least one of the weight of the first codec model or the weight of the second codec model based on the channel codec algorithm information and the source codec algorithm information can comprise: the model training apparatus training the weight corresponding to the second sub-structure of the second codec model and the weight corresponding to the second sub-structure of the first codec model based on the channel codec algorithm information and the source codec algorithm information.

[0044] In a sixth possible implementation, the model structure of the first codec model comprises a first sub-structure and a second sub-structure, the channel codec algorithm information is further used for indicating the weight corresponding to the first sub-structure of the first codec model, the source codec algorithm information is further used for indicating the weight corresponding to the model structure of the second codec model, and the model training apparatus training at least one of the weight of the first codec model or the weight of the second codec model based on the channel codec algorithm information and the source codec algorithm information can comprise: the model training apparatus training the weight corresponding to the second sub-structure of the first codec model based on the channel codec algorithm information and the source codec algorithm information.

[0045] In a seventh possible implementation, the model structure of the second codec model comprises a first sub-structure and a second sub-structure, the source codec algorithm information is further used for indicating the weight corresponding to the first sub-structure of the second codec model, and the channel codec algorithm information is further used for indicating the weight corresponding to the model structure of the first codec model, and the model training apparatus training at least one of the weight of the first codec model or the weight of the second codec model based on the channel codec algorithm information and the source codec algorithm information can comprise: the model training apparatus training the weight corresponding to the second sub-structure of the second codec model based on the channel codec algorithm information and the source codec algorithm information.

[0046] Optionally, the application embodiments do not limit the receiving manner of the auxiliary information.

[0047] In a possible implementation, the model training apparatus can receive the auxiliary information at one time.

[0048] In another possible implementation, the model training apparatus can receive the auxiliary information in multiple times. That is, the model training apparatus receives part of the auxiliary information each time until all the auxiliary information is received.

[0049] Optionally, the embodiments of the present application do not limit the source of the auxiliary information.

[0050] In a possible implementation, the model training apparatus can receive the auxiliary information from the same device (or apparatus).

[0051] For example, the model training apparatus can receive the auxiliary information from the channel-related device.

[0052] In another possible implementation, the model training apparatus can receive the auxiliary information from multiple devices (or apparatuses).

[0053] For example, the model training apparatus can receive the operation information from the channel encoding apparatus and the channel decoding apparatus.

[0054] For example, the model training apparatus can receive the source data information from the source encoding apparatus.

[0055] For example, the model training apparatus can receive the channel information from the channel decoding apparatus.

[0056] By using the model training method provided by the embodiments of the present application, since the auxiliary information can more closely simulate the use scenario of the encoding and decoding model, further combining the auxiliary information to jointly train the source encoding and decoding model and the channel encoding and decoding model can improve the transmission performance of the system.

[0057] Optionally, the method can further include that the model training apparatus sends the weight of the first encoding and decoding model and / or the weight of the second encoding and decoding model obtained by training.

[0058] Optionally, the model training apparatus can directly or indirectly send the weight of the first encoding and decoding model obtained by training to the user of the first encoding and decoding model through other devices; and / or, directly or indirectly send the weight of the second encoding and decoding model obtained by training to the user of the second encoding and decoding model through other devices.

[0059] Optionally, the method can further include: the model training apparatus sending indication information, the indication information including at least one of the following: training completion indication information, used to indicate that the training of the first codec model and / or the second codec model has been completed; or joint coding indication information, used to indicate that the channel coding apparatus and the source coding apparatus can perform joint coding; or joint coding start indication information, used to start joint coding between the channel coding apparatus and the source coding apparatus; or time indication information, used to indicate a start time of joint coding performed by the channel coding apparatus and the source coding apparatus.

[0060] In a second aspect, an embodiment of the present application provides a codec apparatus, which can include: an obtaining unit and a training unit. The obtaining unit is configured to obtain channel coding algorithm information and source coding algorithm information. The channel coding algorithm information is used to indicate a model structure of a first codec model. The source coding algorithm information is used to indicate a model structure of a second codec model. The first codec model includes at least one of a first encoding model or a first decoding model. The first encoding model is used for a channel coding apparatus to encode data output by a source coding apparatus. The first decoding model is used for a channel decoding apparatus to decode data output by the channel coding apparatus and transmitted through a channel. The second codec model includes at least one of a second encoding model or a second decoding model. The second encoding model is used for the source coding apparatus to encode data input by the source coding apparatus. The second decoding model is used for a source decoding apparatus to decode data output by the channel decoding apparatus. The training unit is configured to train at least one of a weight of the first codec model or a weight of the second codec model based on the channel coding algorithm information and the source coding algorithm information.

[0061] In a possible implementation, the channel coding algorithm information is further used to indicate a weight corresponding to the model structure of the first codec model. The training unit is specifically configured to train a weight corresponding to the model structure of the second codec model based on the channel coding algorithm information and the source coding algorithm information.

[0062] In a possible implementation, the source coding algorithm information is further used to indicate a weight corresponding to the model structure of the second codec model. The training unit is specifically configured to train a weight corresponding to the model structure of the first codec model based on the channel coding algorithm information and the source coding algorithm information.

[0063] In a possible implementation, the model structure of the first codec model comprises a first sub-model structure and a second sub-model structure, and the channel codec algorithm information is further used for indicating the weight corresponding to the first sub-model structure of the first codec model, and the training unit is specifically configured to: train the weight corresponding to the second sub-model structure of the first codec model and the weight corresponding to the model structure of the second codec model based on the channel codec algorithm information and the source codec algorithm information.

[0064] In a possible implementation, the model structure of the second codec model comprises a first sub-model structure and a second sub-model structure, and the source codec algorithm information is further used for indicating the weight corresponding to the first sub-model structure of the second codec model, and the training unit is specifically configured to: train the weight corresponding to the second sub-model structure of the second codec model and the weight corresponding to the model structure of the first codec model based on the channel codec algorithm information and the source codec algorithm information.

[0065] In a possible implementation, the model structure of the first codec model comprises a first sub-model structure and a second sub-model structure, the channel codec algorithm information is further used for indicating the weight corresponding to the first sub-model structure of the first codec model, the model structure of the second codec model comprises a first sub-model structure and a second sub-model structure, and the source codec algorithm information is further used for indicating the weight corresponding to the first sub-model structure of the second codec model, and the training unit is specifically configured to: train the weight corresponding to the second sub-model structure of the second codec model and the weight corresponding to the second sub-model structure of the first codec model based on the channel codec algorithm information and the source codec algorithm information.

[0066] In a possible implementation, the model structure of the first codec model comprises a first sub-model structure and a second sub-model structure, the channel codec algorithm information is further used for indicating the weight corresponding to the first sub-model structure of the first codec model, and the source codec algorithm information is further used for indicating the weight corresponding to the model structure of the second codec model, and the training unit is specifically configured to: train the weight corresponding to the second sub-model structure of the first codec model based on the channel codec algorithm information and the source codec algorithm information.

[0067] In a possible implementation, the model structure of the second codec model comprises a first sub-model structure and a second sub-model structure, the source codec algorithm information is further used for indicating the weight corresponding to the first sub-model structure of the second codec model, and the channel codec algorithm information is further used for indicating the weight corresponding to the model structure of the first codec model, and the training unit is specifically configured to: train the weight corresponding to the second sub-model structure of the second codec model based on the channel codec algorithm information and the source codec algorithm information.

[0068] In a possible implementation, the training unit is specifically configured to: train at least one of the weight of the first coding model or the weight of the second coding model based on the channel coding algorithm information, the source coding algorithm information, and auxiliary information; and wherein the auxiliary information comprises at least one of the following: channel information used to indicate an influence of a channel on data transmission; operation information used to indicate at least one of the following operations: a first operation performed before the channel coding device encodes data input into the channel coding device, a second operation performed after the channel coding device encodes data input into the channel coding device, a third operation performed before the channel decoding device decodes data input into the channel decoding device, or a fourth operation performed after the channel decoding device decodes data input into the channel decoding device; auxiliary data information used to indicate at least one of the following auxiliary data: first auxiliary data generated between the source coding device and the channel coding device, second auxiliary data generated between the channel coding device and the channel, third auxiliary data generated between the channel and the channel decoding device, or fourth auxiliary data generated between the channel decoding device and the source decoding device, the first auxiliary data being used to assist the channel coding device in performing the first operation, the second auxiliary data being used to assist the channel coding device in performing the second operation, the third auxiliary data being used to assist the channel decoding device in performing the third operation, and the fourth auxiliary data being used to assist the channel decoding device in performing the fourth operation; or source data information used to indicate input data of the source coding device.

[0069] Optionally, the apparatus can further include a receiving unit configured to: receive the auxiliary information before the training unit trains at least one of the weight of the first coding model or the weight of the second coding model based on the channel coding algorithm information and the source coding algorithm information.

[0070] Optionally, the apparatus 300 can further include a sending unit configured to: send at least one of the weight of the first coding model or the weight of the second coding model trained.

[0071] In a possible implementation, the sending unit is further configured to: send model training capability information before the obtaining unit obtains the channel coding algorithm information and the source coding algorithm information, the model training capability information being used to indicate that a model training device has the capability of training a coding model.

[0072] In a third aspect, the present application also provides a coding and decoding apparatus, which comprises a processor and a communication interface, the processor and the communication interface are coupled, the communication interface is configured to provide information and / or data for the processor, and the processor is configured to run computer program instructions to perform the coding and decoding method provided in the above aspects or various possible implementation manners thereof.

[0073] In a fourth aspect, the present application also provides a computer readable storage medium, which stores instructions, when the instructions are run on a computer, the computer is caused to perform the method provided in the above aspects or any possible implementation manner thereof.

[0074] In a fifth aspect, the present application also provides a computer program product comprising instructions, when the instructions are run on a computer, the computer is caused to perform the method provided in the above aspects or any possible implementation manner thereof.

[0075] It can be understood that any of the coding and decoding apparatus, computer storage medium or computer program product provided above are used to perform the corresponding method provided above, and thus the beneficial effects achieved thereby can refer to the beneficial effects of the corresponding method provided above, which will not be described here again. BRIEF DESCRIPTION OF DRAWINGS

[0076] FIG. 1 is a schematic block diagram of a coding and decoding system 100 provided by an embodiment of the present application;

[0077] FIG. 2 is a schematic block diagram of a model training method 200 provided by an embodiment of the present application;

[0078] FIG. 3 is a schematic block diagram of a model training apparatus 300 provided by an embodiment of the present application;

[0079] FIG. 4 is a schematic block diagram of a model training apparatus 400 provided by an embodiment of the present application. DETAILED DESCRIPTION

[0080] The technical solutions in the present application will be described below with reference to the accompanying drawings.

[0081] Firstly, the coding and decoding system provided by an embodiment of the present application will be introduced.

[0082] FIG. 1 shows a schematic diagram of a coding and decoding system 100 provided by an embodiment of the present application. As shown in FIG. 1, the system 100 can comprise an encoding end 110 and a decoding end 120, the encoding end 110 can comprise a source encoding apparatus 111 and a channel encoding apparatus 112, and the decoding end 120 can comprise a channel decoding apparatus 121 and a source decoding apparatus 122.

[0083] The source coding device 111 is configured to encode the input raw data (e.g., media data) based on a source coding model to obtain encoded data 1; and transmit the encoded data 1 to the channel coding device 112.

[0084] The channel coding device 112 is configured to encode the encoded data 1 based on a channel coding model to obtain encoded data 2; and transmit the encoded data 2 to the channel decoding device 121 through a channel to become encoded data 2a.

[0085] The channel decoding device 121 is configured to decode the encoded data 2a based on a channel decoding model to obtain decoded data 1a; and transmit the decoded data 1a to the source decoding device 122.

[0086] The source decoding device 122 is configured to decode the decoded data 1a based on a source decoding model to obtain decoded data 3, which includes the raw data.

[0087] Optionally, there can be other devices between the source coding device 111 and the channel coding device 112, between the channel coding device 112 and the channel decoding device 121, and between the channel decoding device 121 and the source decoding device 122.

[0088] It can be understood that the encoded data 1a and the encoded data 1 can be the same, or can be different due to channel transmission and / or other processing therein; the encoded data 2a and the encoded data 2 can be the same, or can be different due to channel transmission and / or other processing therein; and the raw data included in the decoded data 3 and the raw data can be the same, or can be different due to channel transmission and / or other processing therein.

[0089] Optionally, each of the above-mentioned encoding devices can perform preprocessing on the input data before encoding the input data; and / or each of the above-mentioned encoding devices can perform post-processing on the encoded data after encoding the input data. Optionally, the specific operation of the above-mentioned preprocessing can refer to the prior art, which will not be described here.

[0090] Optionally, each of the above-mentioned decoding devices can perform preprocessing on the input data before decoding the input data; and / or each of the above-mentioned decoding devices can perform post-processing on the decoded data after decoding the input data. Optionally, the specific operation of the above-mentioned post-processing can refer to the prior art, which will not be described here.

[0091] Optionally, one or more of the above-mentioned source coding model (i.e., the second encoding model), the channel coding model (i.e., the first encoding model), the channel decoding model (i.e., the first decoding model), or the source decoding model (i.e., the second decoding model) can be obtained by the model training method provided by the embodiments of the present application, which will be described in detail below.

[0092] Optionally, the embodiments of the present application do not limit the specific forms of the source coding device 111, the channel coding device 112, the channel decoding device 121 and the source decoding device 122.

[0093] In one possible implementation, the source coding device 111 can be an independent device A, the channel coding device 112 can be an independent device B, and the channel decoding device 121 and the source decoding device 122 can be integrated in a device C as functional modules or chip devices.

[0094] For example, the device A can be a video server, the device B can be a base station, and the device C can be a mobile phone. That is, the video server generates a media stream, the media stream is transmitted to the mobile phone via the base station for decoding and playing.

[0095] In another possible implementation, the source coding device 111 can be an independent device D, the channel coding device 112 can be an independent device E, the channel decoding device 121 can be an independent device F, and the source decoding device 122 can be an independent device G.

[0096] For example, the device D can be a video server, the device E can be a base station, the device F can be a mobile phone, and the device G can be a television. That is, the video server generates a media stream, the media stream is transmitted to the mobile phone via the base station, and the mobile phone forwards the media stream to the television for decoding and playing as a hotspot.

[0097] In yet another possible implementation, the source coding device 111 and the channel coding device 112 can be integrated in a device H as functional modules or chip devices, and the channel decoding device 121 and the source decoding device 122 can be integrated in a device I as functional modules or chip devices.

[0098] For example, the device H can be a speakerphone A, and the device I can be a speakerphone B. That is, the device A and the device B perform D2D communication.

[0099] In one possible implementation, the encoding end 110 and the decoding end 120 can communicate through a channel.

[0100] Optionally, the technical solutions of the embodiments of the present application can be applied to various communication systems. For example, global system for mobile communications (GSM), long term evolution (LTE) system, universal mobile telecommunication system (UMTS), 4th generation (4G) mobile communication system, 4.5th generation (4.5G) mobile communication system, worldwide interoperability for microwave access (WiMAX) communication system, 5th generation (5G) mobile communication system or new radio Access Technology (NR), and with the continuous development of communication technology, the technical solutions of the embodiments of the present application can also be applied to subsequent evolved communication systems, such as 6th generation (6G) mobile communication system, 7th generation (7G) mobile communication system, etc. The technical solutions of the embodiments of the present application can also be applied to universal mobile telecommunications system (UMTS), code division multiple access (CDMA) system, wireless local area network (WLAN), open RAN (O-RAN or ORAN), cloud radio access network (CRAN), etc.The technical solutions of the embodiments of the present application can also be applied to vehicle-to-other device (vehicle-to-X, V2X), wherein the V2X can include vehicle to network (V2N), vehicle to vehicle (V2V), vehicle to infrastructure (V2I), vehicle to pedestrian (V2P), Long Term Evolution-Vehicle (LTE-V), Internet of Vehicles, machine type communication (MTC), Internet of Things (IoT), Long Term Evolution-Machine (LTE-M), Machine to Machine (M2M), etc.

[0101] In a conventional codec system, source coding and channel coding are divided into two modules and designed independently, for example, a sending end performs source coding and channel coding on media data respectively, obtains coded data, and transmits the coded data through a channel. Correspondingly, source decoding and channel decoding are also divided into two modules and designed independently, for example, a receiving end receives coded data through a channel, performs channel decoding and source decoding on the coded data respectively, and obtains the media data.

[0102] However, since source coding and decoding and channel coding and decoding are independent modules in the prior art, the influence of source coding and decoding on channel coding and decoding and vice versa is not considered, and therefore, there is still room for improvement in the transmission performance of the codec system.

[0103] The application provides a model training method and device. The method can include: obtaining channel coding algorithm information and source coding algorithm information, the channel coding algorithm information being used to indicate a model structure of a first coding model, and the source coding algorithm information being used to indicate a model structure of a second coding model, wherein the first coding model includes at least one of a first encoding model or a first decoding model, the first encoding model being used for a channel encoding device to encode data output by a source encoding device, the first decoding model being used for a channel decoding device to decode data output by the channel encoding device and transmitted through a channel, the second coding model includes at least one of a second encoding model or a second decoding model, the second encoding model being used for the source encoding device to encode data input by the source encoding device, and the second decoding model being used for a source decoding device to decode data output by the channel decoding device; and training at least one of a weight of the first coding model or a weight of the second coding model based on the channel coding algorithm information and the source coding algorithm information. The model training method and device provided in the embodiments of the application can jointly train a source coding model and a channel coding model, thereby improving the performance of a system.

[0104] The model training method provided in the embodiments of the application will be further described below with reference to the accompanying drawings.

[0105] FIG. 2 shows a schematic flowchart of a model training method 200 provided in the embodiments of the application, which can be executed by a model training device.

[0106] Optionally, the embodiments of the application do not limit the specific form of the model training device.

[0107] In one possible implementation, the model training device can be the source encoding device 111 or the source decoding device 122 in the system 100.

[0108] In another possible implementation, the model training device can be a device related to the source encoding device 111 and / or the source decoding device 122 in the system 100. For example, the model training device can be a server device owned by a designer or a manufacturer of the source encoding device 111 and / or the source decoding device 122 in the system 100.

[0109] In yet another possible implementation, the model training device can be the channel encoding device 112 or the channel decoding device 121 in the system 100.

[0110] In yet another possible implementation, the model training apparatus can be a device related to the channel encoding apparatus 112 and / or the channel decoding apparatus 121 in the system 100. For example, the model training apparatus can be a server device owned by a designer or a producer of the channel encoding apparatus 112 and / or the channel decoding apparatus 121 in the system 100.

[0111] In yet another possible implementation, the model training apparatus can be a third-party device other than the apparatuses in the system 100. For example, a computing device or a server device with model training capability.

[0112] As shown in FIG. 2, the method 200 can include S201 and S202. The steps of the method 200 are described in detail as follows.

[0113] S201. The model training apparatus obtains channel codec algorithm information and source codec algorithm information, the channel codec algorithm information being used to indicate a model structure of a first codec model, the source codec algorithm information being used to indicate a model structure of a second codec model, wherein the first codec model includes at least one of a first encoding model or a first decoding model, the first encoding model being used for a channel encoding apparatus to encode data output by a source encoding apparatus, the first decoding model being used for a channel decoding apparatus to decode data output by the channel encoding apparatus and transmitted through a channel, the second codec model includes at least one of a second encoding model or a second decoding model, the second encoding model being used for the source encoding apparatus to encode data input by the source encoding apparatus, the second decoding model being used for a source decoding apparatus to decode data output by the channel decoding apparatus.

[0114] It should be noted that “A and / or B”, or “at least one of A or B” in the embodiments of the present application can include any of the following cases: (1) A; (2) B; (3) A and B.

[0115] It should be further noted that the codec model (such as a source codec model or a channel codec model) in the embodiments of the present application can include any of the following cases: (1) an encoding model; (2) a decoding model; (3) an encoding model and a decoding model.

[0116] It should be further noted that the first codec model in the embodiments of the present application can also be referred to as a channel codec model, wherein the first encoding model can also be referred to as a channel encoding model, and the first decoding model can also be referred to as a channel decoding model; the second codec model in the embodiments of the present application can also be referred to as a source codec model, wherein the second encoding model can also be referred to as a source encoding model, and the second decoding model can also be referred to as a source decoding model.

[0117] Optionally, the model training apparatus can obtain the channel coding algorithm information in various manners, which are not limited in the embodiments of the present application.

[0118] In a possible implementation, the model training apparatus can receive the channel coding algorithm information from a channel-related device, such as a training device of a channel coding model.

[0119] In another possible implementation, if the channel coding algorithm information is used to indicate the model structure of the first encoding model, the model training apparatus can receive the channel coding algorithm information from a channel encoding apparatus.

[0120] In yet another possible implementation, if the channel coding algorithm information is used to indicate the model structure of the first decoding model, the model training apparatus can receive the channel coding algorithm information from a channel decoding apparatus.

[0121] In yet another possible implementation, if the channel coding algorithm information is used to indicate the model structure of the first encoding model and the model structure of the first decoding model, the model training apparatus can receive the channel coding algorithm information from a channel encoding apparatus and a channel decoding apparatus.

[0122] Optionally, the channel coding algorithm information can also be pre-agreed or standardized, which is not limited in the embodiments of the present application.

[0123] Optionally, the channel coding algorithm information can indicate the model structure of the first encoding-decoding model in various manners, which are not limited in the embodiments of the present application.

[0124] In a possible implementation, the channel coding algorithm information can include a first model structure identifier, which is used to indicate the model structure of the first encoding-decoding model.

[0125] In another possible implementation, the channel coding algorithm information can include the model structure of the first encoding-decoding model.

[0126] Optionally, the model structure (such as the model structure of the first encoding-decoding model) described in the embodiments of the present application can include multiple layers.

[0127] Optionally, the layers described in the embodiments of the present application can include, but are not limited to, the following layers: a fully connected layer, a convolutional layer, a pooling layer, or a deconvolutional layer, etc.

[0128] Optionally, the model structure of the encoding-decoding model described in the embodiments of the present application can include at least one of the model structure of an encoding model or the model structure of a decoding model.

[0129] Optionally, the model training apparatus can obtain the source coding algorithm information in various manners, which are not limited in the embodiments of the present application.

[0130] In a possible implementation, the model training apparatus can receive the source coding algorithm information from a source-related device (e.g., a training device of a source coding model).

[0131] In another possible implementation, if the source coding algorithm information is used to indicate the model structure of the second encoding model, the model training apparatus can receive the source coding algorithm information from the source encoding apparatus.

[0132] In yet another possible implementation, if the source coding algorithm information is used to indicate the model structure of the second decoding model, the model training apparatus can receive the source coding algorithm information from the source decoding apparatus.

[0133] In yet another possible implementation, if the source coding algorithm information is used to indicate the model structure of the second encoding model and the model structure of the second decoding model, the model training apparatus can receive the source coding algorithm information from the source encoding apparatus and the source decoding apparatus.

[0134] Optionally, the source coding algorithm information can also be pre-agreed or standardized, which is not limited in the embodiments of the present application.

[0135] Optionally, the source coding algorithm information can indicate the model structure of the second coding model in various manners, which are not limited in the embodiments of the present application.

[0136] In a possible implementation, the source coding algorithm information can include a second model structure identifier, which is used to indicate the model structure of the second coding model.

[0137] In another possible implementation, the source coding algorithm information can include the model structure of the second coding model.

[0138] Optionally, if the model training apparatus is the source encoding apparatus, the source decoding apparatus or the source-related device, before S201, the method 200 can further include: the model training apparatus sending a channel coding algorithm information request message to a provider of channel coding algorithm information (e.g., a channel-related device), the request message being used to request the provider of channel coding algorithm information to send the channel coding algorithm information.

[0139] Optionally, if the model training apparatus is a source encoding apparatus, a source decoding apparatus, or a source-related device, before S201, the method 200 can further include: the model training apparatus sending model training capability information to a source channel coding and decoding algorithm information providing object, the model training capability information being used to indicate that the source channel coding and decoding algorithm information providing object has the capability of training a coding and decoding model.

[0140] Optionally, if the model training apparatus is a source encoding apparatus, a source decoding apparatus, or a source-related device, before S201, the method 200 can further include: the model training apparatus receiving model providing capability information from a source channel coding and decoding algorithm information providing object, the model providing capability information being used to indicate that the source channel coding and decoding algorithm information providing object has the capability of providing a model structure of the first coding and decoding model.

[0141] Optionally, if the model training apparatus is a channel encoding apparatus, a channel decoding apparatus, or a channel-related device, before S201, the method 200 can further include: the model training apparatus sending a source channel coding and decoding algorithm information request message to a source channel coding and decoding algorithm information providing object (such as a source-related device), the request message being used to request the source channel coding and decoding algorithm information providing object to send the source channel coding and decoding algorithm information.

[0142] Optionally, if the model training apparatus is a channel encoding apparatus, a channel decoding apparatus, or a channel-related device, before S201, the method 200 can further include: the model training apparatus sending model training capability information to a source channel coding and decoding algorithm information providing object, the model training capability information being used to indicate that the source channel coding and decoding algorithm information providing object has the capability of training a coding and decoding model.

[0143] Optionally, if the model training apparatus is a channel encoding apparatus, a channel decoding apparatus, or a channel-related device, before S201, the method 200 can further include: the model training apparatus receiving model providing capability information from a source channel coding and decoding algorithm information providing object, the model providing capability information being used to indicate that the source channel coding and decoding algorithm information providing object has the capability of providing a model structure of the second coding and decoding model.

[0144] S202. The model training apparatus trains at least one of a weight of the first coding and decoding model or a weight of the second coding and decoding model based on the channel coding and decoding algorithm information and the source channel coding and decoding algorithm information.

[0145] It should be noted that the model in the embodiments of the present application can include a model structure and a weight corresponding to the model structure.

[0146] It should be further noted that the weights of the model in the embodiments of the present application can include the weights corresponding to the model structure or the weights corresponding to the sub-model structure. The sub-model structure can include any partial layer in the model structure.

[0147] Optionally, the embodiments of the present application do not limit the number of layers included in the sub-model structure, and the position and order of each layer in the model structure.

[0148] For example, the model structure of the first coding model can include layer 1, layer 2, layer 3, layer 10, wherein the first sub-model structure can include layer 1, layer 2, layer 3, layer 5 and layer 8, and the second sub-model structure can include layer 4, layer 6 and layer 7. Correspondingly, the weights of the model structure can include the weights corresponding to layer 1-layer 10, the weights corresponding to the first sub-model structure can include the weights corresponding to layer 1, layer 2, layer 3, layer 5 and layer 8, and the weights of the second sub-model structure can include the weights corresponding to layer 4, layer 6 and layer 7.

[0149] Optionally, S202 can include, but is not limited to, the following seven possible implementation manners:

[0150] In the first possible implementation manner, the channel coding algorithm information is also used to indicate the weights corresponding to the model structure of the first coding model, and S202 can include that the model training apparatus trains the weights corresponding to the model structure of the second coding model based on the channel coding algorithm information and the source coding algorithm information.

[0151] In the second possible implementation manner, the source coding algorithm information is also used to indicate the weights corresponding to the model structure of the second coding model, and S202 can include that the model training apparatus trains the weights corresponding to the model structure of the first coding model based on the channel coding algorithm information and the source coding algorithm information.

[0152] In the third possible implementation manner, the model structure of the first coding model includes a first sub-model structure and a second sub-model structure, the channel coding algorithm information is also used to indicate the weights corresponding to the first sub-model structure of the first coding model, and S202 can include that the model training apparatus trains the weights corresponding to the second sub-model structure of the first coding model and the weights corresponding to the model structure of the second coding model based on the channel coding algorithm information and the source coding algorithm information.

[0153] In a fourth possible implementation, the model structure of the second codec model comprises a first sub-model structure and a second sub-model structure, and the source codec algorithm information is further used to indicate the weight corresponding to the first sub-model structure of the second codec model. S202 can comprise: the model training apparatus trains the weight corresponding to the second sub-model structure of the second codec model and the weight corresponding to the model structure of the first codec model based on the channel codec algorithm information and the source codec algorithm information.

[0154] In a fifth possible implementation, the model structure of the first codec model comprises a first sub-structure and a second sub-structure, and the channel codec algorithm information is further used to indicate the weight corresponding to the first sub-structure of the first codec model. The model structure of the second codec model comprises a first sub-structure and a second sub-structure, and the source codec algorithm information is further used to indicate the weight corresponding to the first sub-structure of the second codec model. S202 can comprise: the model training apparatus trains the weight corresponding to the second sub-model structure of the second codec model and the weight corresponding to the second sub-model structure of the first codec model based on the channel codec algorithm information and the source codec algorithm information.

[0155] In a sixth possible implementation, the model structure of the first codec model comprises a first sub-model structure and a second sub-model structure, and the channel codec algorithm information is further used to indicate the weight corresponding to the first sub-model structure of the first codec model. The source codec algorithm information is further used to indicate the weight corresponding to the model structure of the second codec model. S202 can comprise: the model training apparatus trains the weight corresponding to the second sub-model structure of the first codec model based on the channel codec algorithm information and the source codec algorithm information.

[0156] In a seventh possible implementation, the model structure of the second codec model comprises a first sub-model structure and a second sub-model structure, and the source codec algorithm information is further used to indicate the weight corresponding to the first sub-model structure of the second codec model. The channel codec algorithm information is further used to indicate the weight corresponding to the model structure of the first codec model. S202 can comprise: the model training apparatus trains the weight corresponding to the second sub-model structure of the second codec model based on the channel codec algorithm information and the source codec algorithm information.

[0157] Optionally, S202 can comprise: the model training apparatus trains at least one of the weight of the first codec model or the weight of the second codec model based on the channel codec algorithm information, the source codec algorithm information, and the auxiliary information.

[0158] Optionally, the auxiliary information can comprise at least one of the following information:

[0159] channel information for indicating an influence of a channel on data transmission; or

[0160] operation information for indicating at least one of a first operation performed before encoding of data input to the channel encoding device, a second operation performed after encoding of data input to the channel encoding device, a third operation performed before decoding of data input to the channel decoding device, or a fourth operation performed after decoding of data input to the channel decoding device; or

[0161] auxiliary data information for indicating at least one of first auxiliary data generated between the source encoding device and the channel encoding device, second auxiliary data generated between the channel encoding device and the channel, third auxiliary data generated between the channel and the channel decoding device, or fourth auxiliary data generated between the channel decoding device and the source decoding device, the first auxiliary data being used to assist the channel encoding device to perform the first operation, the second auxiliary data being used to assist the channel encoding device to perform the second operation, the third auxiliary data being used to assist the channel decoding device to perform the third operation, and the fourth auxiliary data being used to assist the channel decoding device to perform the fourth operation; or

[0162] source data information for indicating input data of the source encoding device.

[0163] It can be understood that the data input to the channel encoding device refers to data input to the channel encoding device.

[0164] For example, the channel information can be used to indicate at least one of the following: a type of the channel, such as an additive white Gaussian noise (AWGN) channel, a Rayleigh fading channel, etc.; a channel simulator, such as a generative adversarial networks (GAN) based channel simulator; channel signal-to-noise ratio information, such as a signal-to-noise ratio range, etc.

[0165] For example, the first operation or the second operation can comprise at least one of the following operations: segmentation, merging, packetization, depacketization, interleaving, modulation, cyclic redundancy check (CRC), etc.

[0166] For example, the auxiliary data information can include at least one of the following: the first auxiliary data, a data distribution feature of the first auxiliary data, a generation method of the first auxiliary data, a generator of the first auxiliary data, or packet header information required by the channel coding device when packetizing.

[0167] For example, the source data information can include at least one of the following: the source data, a data distribution feature of the source data, a generation method of the source data, or a generator of the source data.

[0168] In a possible implementation, part or all of the auxiliary information can be combined with the channel coding algorithm information to be presented in the form of software or an application programming interface (API) for training the weight of the second coding model.

[0169] In another possible implementation, part or all of the auxiliary information can be combined with the source coding algorithm information to be presented in the form of software or an API for training the weight of the first coding model.

[0170] Optionally, the embodiments of the present application do not limit the receiving manner of the auxiliary information.

[0171] In a possible implementation, the model training apparatus can receive the auxiliary information at one time.

[0172] In another possible implementation, the model training apparatus can receive the auxiliary information in multiple times. That is, the model training apparatus receives part of the auxiliary information each time until all the auxiliary information is received.

[0173] Optionally, the embodiments of the present application do not limit the source of the auxiliary information.

[0174] In a possible implementation, the model training apparatus can receive the auxiliary information from the same device (or apparatus).

[0175] For example, the model training apparatus can receive the auxiliary information from the channel-related device.

[0176] In another possible implementation, the model training apparatus can receive the auxiliary information from multiple devices (or apparatuses).

[0177] For example, the model training apparatus can receive the operation information from the channel coding device and the channel decoding device.

[0178] For example, the model training apparatus can receive the source data information from the source coding device.

[0179] For example, the model training apparatus can receive channel information from the channel decoding apparatus.

[0180] Optionally, after S202, the method 200 can further include that the model training apparatus sends the trained weight of the first codec model and / or the trained weight of the second codec model.

[0181] Optionally, the model training apparatus can directly or indirectly send the trained weight of the first codec model to the user of the first codec model through other devices; and / or, directly or indirectly send the trained weight of the second codec model to the user of the second codec model through other devices.

[0182] For example, if the model training apparatus is a third-party device, the user of the first encoding model can be a channel encoding apparatus, the user of the first decoding model can be a channel decoding apparatus, the user of the second encoding model can be a source encoding apparatus, and the user of the second decoding model can be a source decoding apparatus.

[0183] Optionally, if the model training apparatus is a source encoding apparatus, after S202, the method 200 can further include that the source encoding apparatus encodes input data based on the second encoding model to obtain first encoded data; and sends the first encoded data to the channel encoding apparatus.

[0184] Optionally, if the model training apparatus is a channel encoding apparatus, after S202, the method 200 can further include that the channel encoding apparatus encodes input data based on the first encoding model to obtain second encoded data; and sends the second encoded data to the channel decoding apparatus through a channel.

[0185] Optionally, if the model training apparatus is a channel decoding apparatus, after S202, the method 200 can further include that the channel decoding apparatus decodes input data based on the first decoding model to obtain first decoded data; and sends the first decoded data to the source decoding apparatus.

[0186] Optionally, if the model training apparatus is a source decoding apparatus, after S202, the method 200 can further include that the source decoding apparatus decodes input data based on the second decoding model to obtain second decoded data.

[0187] Optionally, after S202, the method 200 can further include: the model training apparatus sending indication information, the indication information comprising at least one of the following information: training completion indication information, used to indicate that the training of the first codec model and / or the second codec model has been completed; or joint coding indication information, used to indicate that the channel coding apparatus and the source coding apparatus can perform joint coding; or joint coding start indication information, used to start the joint coding between the channel coding apparatus and the source coding apparatus; or time indication information, used to indicate the start time of the joint coding performed by the channel coding apparatus and the source coding apparatus.

[0188] In a possible implementation, the joint coding between the channel coding apparatus and the source coding apparatus in the embodiments of the present application refers to that the source coding apparatus encodes the input data by using the trained second encoding model; and / or the channel coding apparatus encodes the data output by the source coding apparatus by using the trained first encoding model, and transmits the data to the channel decoding apparatus through a channel; and / or the channel decoding apparatus decodes the received data by using the trained first decoding model; and / or the source decoding apparatus decodes the data output by the channel decoding apparatus by using the trained second decoding model.

[0189] In a possible implementation, taking the model training apparatus as an example of the source-related device, the model training method provided by the embodiments of the present application can include the following steps:

[0190] (1) The model training apparatus sends model training capability information to the channel-related device, the model training capability information being used to indicate that the model training apparatus has the capability of training a codec model.

[0191] (2) The channel-related device sends model providing capability information to the model training apparatus, the model providing capability information being used to indicate that the channel-related device has the capability of providing the model structure of the first codec model, the structure of the first codec model comprising a first encoding model structure and a first decoding model structure, the first encoding model being used for the channel coding apparatus to encode the data output by the source coding apparatus, and the first decoding model being used for the channel decoding apparatus to decode the data output by the channel coding apparatus and transmitted through a channel.

[0192] (3) The model training apparatus receives channel codec algorithm information from the channel-related device, the channel codec algorithm information being used to indicate the model structure of the first encoding model and the model structure of the first decoding model.

[0193] (4) The model training device receives source coding and decoding algorithm information from the source encoding device and the source decoding device, and the source coding and decoding algorithm information is used to indicate the model structure of the second encoding model and the model structure of the second decoding model, the second encoding model is used for the source encoding device to encode the data input by the source encoding device, and the second decoding model is used for the source decoding device to decode the data output by the channel decoding device.

[0194] (5) The model training device trains the weights corresponding to the model structure of the first encoding model, the weights corresponding to the model structure of the first decoding model, the weights corresponding to the model structure of the second encoding model, and the weights corresponding to the model structure of the second decoding model based on the model structure of the first encoding model, the model structure of the first decoding model, the model structure of the second encoding model, the model structure of the second decoding model and the auxiliary information.

[0195] (6) The model training device sends the second encoding model to the source encoding device and sends the second decoding model to the source decoding device.

[0196] (7) The model training device sends the first encoding model to the channel encoding device and sends the first decoding model to the channel decoding device.

[0197] (8) The model training device sends indication information to the source encoding device, the source decoding device, the channel encoding device and the channel decoding device, and the indication information includes at least one of the following information: training completion indication information, used to indicate that the training of the first and / or second codec model has been completed; or joint coding indication information, used to indicate that the channel coding and decoding device and the source coding and decoding device can perform joint coding; or joint coding start indication information, used to start the joint coding between the channel coding and decoding device and the source coding and decoding device; or time indication information, used to indicate the start time of joint coding of the channel coding and decoding device and the source coding and decoding device.

[0198] Correspondingly, the source encoding device, the channel encoding device, the channel decoding device and the source decoding device perform joint coding based on the indication information from the start time.

[0199] For example, the joint coding can include: the source coding device encodes input data based on the second encoding model to obtain encoded data 1; the source coding device sends the encoded data 1 to the channel coding device; the channel coding device encodes the encoded data 1 based on the first encoding model to obtain encoded data 2, and sends the encoded data 2 to the channel decoding device through a channel; the channel decoding device decodes the encoded data 2 based on the first decoding model to obtain decoded data 1, and sends the decoded data 1 to the source decoding device; and the source decoding device decodes the decoded data 1 based on the second decoding model to obtain decoded data 2.

[0200] The model training method provided in the embodiments of the present application is described above in combination with FIG. 2, and the model training device provided in the embodiments of the present application will be further described below.

[0201] FIG. 3 shows a schematic block diagram of a model training device 300 provided in the embodiments of the present application. As shown in FIG. 3, the device 300 can include an obtaining unit 301 and a training unit 302.

[0202] Optionally, the device 300 can be used as the model training device in the method 200 embodiments described above, and further, the device 300 can be a virtual device formed by software executed by a processor or a controller of the model training device in the method 200 embodiments described above.

[0203] The obtaining unit 301 is configured to obtain channel coding algorithm information and source coding algorithm information, the channel coding algorithm information being used to indicate a model structure of a first coding model, the source coding algorithm information being used to indicate a model structure of a second coding model, wherein the first coding model includes at least one of a first encoding model or a first decoding model, the first encoding model being used for a channel coding device to encode data output by a source coding device, the first decoding model being used for a channel decoding device to decode data output by the channel coding device and transmitted through a channel, the second coding model includes at least one of a second encoding model or a second decoding model, the second encoding model being used for the source coding device to encode data input by the source coding device, the second decoding model being used for a source decoding device to decode data output by the channel decoding device.

[0204] The training unit 302 is configured to train at least one of a weight of the first coding model or a weight of the second coding model based on the channel coding algorithm information and the source coding algorithm information.

[0205] In a possible implementation, the channel coding algorithm information is further used to indicate weights corresponding to a model structure of the first coding model, and the training unit 302 is specifically configured to: train weights corresponding to a model structure of the second coding model based on the channel coding algorithm information and the source coding algorithm information.

[0206] In a possible implementation, the source coding algorithm information is further used to indicate weights corresponding to a model structure of the second coding model, and the training unit 302 is specifically configured to: train weights corresponding to a model structure of the first coding model based on the channel coding algorithm information and the source coding algorithm information.

[0207] In a possible implementation, the model structure of the first coding model includes a first sub-model structure and a second sub-model structure, the channel coding algorithm information is further used to indicate weights corresponding to the first sub-model structure of the first coding model, and the training unit 302 is specifically configured to: train weights corresponding to the second sub-model structure of the first coding model and weights corresponding to a model structure of the second coding model based on the channel coding algorithm information and the source coding algorithm information.

[0208] In a possible implementation, the model structure of the second coding model includes a first sub-model structure and a second sub-model structure, the source coding algorithm information is further used to indicate weights corresponding to the first sub-model structure of the second coding model, and the training unit 302 is specifically configured to: train weights corresponding to the second sub-model structure of the second coding model and weights corresponding to a model structure of the first coding model based on the channel coding algorithm information and the source coding algorithm information.

[0209] In a possible implementation, the model structure of the first coding model includes a first sub-model structure and a second sub-model structure, the channel coding algorithm information is further used to indicate weights corresponding to the first sub-model structure of the first coding model, the model structure of the second coding model includes a first sub-model structure and a second sub-model structure, the source coding algorithm information is further used to indicate weights corresponding to the first sub-model structure of the second coding model, and the training unit 302 is specifically configured to: train weights corresponding to the second sub-model structure of the second coding model and weights corresponding to the second sub-model structure of the first coding model based on the channel coding algorithm information and the source coding algorithm information.

[0210] In a possible implementation, the model structure of the first coding model comprises a first sub-model structure and a second sub-model structure, the channel coding algorithm information is further used for indicating weights corresponding to the first sub-model structure of the first coding model, and the source coding algorithm information is further used for indicating weights corresponding to the model structure of the second coding model. The training unit 302 is specifically configured to train, based on the channel coding algorithm information and the source coding algorithm information, the weights corresponding to the second sub-model structure of the first coding model.

[0211] In a possible implementation, the model structure of the second coding model comprises a first sub-model structure and a second sub-model structure, the source coding algorithm information is further used for indicating weights corresponding to the first sub-model structure of the second coding model, and the channel coding algorithm information is further used for indicating weights corresponding to the model structure of the first coding model. The training unit 302 is specifically configured to train, based on the channel coding algorithm information and the source coding algorithm information, the weights corresponding to the second sub-model structure of the second coding model.

[0212] In a possible implementation, the training unit 302 is specifically configured to train, based on the channel coding algorithm information, the source coding algorithm information, and auxiliary information, at least one of the weights of the first coding model or the weights of the second coding model. The auxiliary information comprises at least one of the following information: channel information used for indicating an influence of a channel on data transmission; or operation information used for indicating at least one of the following operations: a first operation performed before the channel encoding device encodes data input into the channel encoding device, a second operation performed after the channel encoding device encodes the data input into the channel encoding device, a third operation performed before the channel decoding device decodes data input into the channel decoding device, or a fourth operation performed after the channel decoding device decodes the data input into the channel decoding device; or auxiliary data information used for indicating at least one of the following auxiliary data: first auxiliary data generated between the source encoding device and the channel encoding device, second auxiliary data generated between the channel encoding device and the channel, third auxiliary data generated between the channel and the channel decoding device, or fourth auxiliary data generated between the channel decoding device and the source decoding device, the first auxiliary data being used for assisting the channel encoding device to perform the first operation, the second auxiliary data being used for assisting the channel encoding device to perform the second operation, the third auxiliary data being used for assisting the channel decoding device to perform the third operation, and the fourth auxiliary data being used for assisting the channel decoding device to perform the fourth operation; or source data information used for indicating input data of the source encoding device.

[0213] Optionally, the apparatus 300 further can comprise a receiving unit 303.

[0214] In a possible implementation, the receiving unit 303 is configured to receive the auxiliary information before the training unit 302 trains at least one of the weight of the first coding model or the weight of the second coding model based on the channel coding algorithm information and the source coding algorithm information.

[0215] Optionally, the apparatus 300 further can comprise a sending unit 304.

[0216] In a possible implementation, the sending unit 304 is configured to send at least one of the trained weight of the first coding model or the trained weight of the second coding model.

[0217] In a possible implementation, the sending unit 304 is further configured to send model training capability information before the obtaining unit 301 obtains the channel coding algorithm information and the source coding algorithm information, the model training capability information being used to indicate that the model training apparatus has the capability of training a coding model.

[0218] It should be noted that the information interaction between the above apparatuses, the execution process, and the like, are based on the same concept as the method 200 embodiments, and the specific functions and the technical effects brought by the same can be referred to the method embodiments part, which will not be repeated here. In an optional example, the apparatus 300 can be specifically the model training apparatus in the above method 200 embodiments, and the apparatus 300 can be configured to execute each process and / or step corresponding to the model training apparatus in the above method 200 embodiments. To avoid repetition, details will not be repeated here.

[0219] One or more of each module in the embodiments shown in FIG. 3 can be implemented by software, hardware, firmware, or a combination thereof. The software or firmware includes, but is not limited to, computer program instructions or codes, and can be executed by a hardware processor. The hardware includes, but is not limited to, various integrated circuits, such as a central processing unit (CPU), a digital signal processor (DSP), a field programmable gate array (FPGA), or an application specific integrated circuit (ASIC).

[0220] FIG. 4 shows a schematic block diagram of a model training apparatus 400 provided in an embodiment of the present application. The apparatus 400 can include a processor 401 and a communication interface 402, which are coupled.

[0221] In an optional example, the apparatus 400 can be specifically a model training apparatus in the method 200 embodiments described above, and the apparatus 400 can be an entity hardware structure of the model training apparatus. The apparatus 400 can be used to execute various processes and / or steps corresponding to the model training apparatus in the method 200 embodiments described above, and thus details are not repeated here.

[0222] The processor 401 in the embodiments of the present application can include one or more processing units. Optionally, the processing unit includes but is not limited to a CPU, a general-purpose processor, a DSP, an ASIC, an FPGA, a discrete gate or transistor logic device, or a discrete hardware component, etc. The general-purpose processor can be a microprocessor, a microcontroller, or any conventional processor, etc.

[0223] For example, the processor 401 is configured to obtain channel codec algorithm information and source codec algorithm information, the channel codec algorithm information is used to indicate a model structure of a first codec model, and the source codec algorithm information is used to indicate a model structure of a second codec model, wherein the first codec model includes at least one of a first encoding model or a first decoding model, the first encoding model is used for a channel encoding device to encode data output by a source encoding device, and the first decoding model is used for a channel decoding device to decode data output by the channel encoding device and transmitted through a channel, the second codec model includes at least one of a second encoding model or a second decoding model, the second encoding model is used for the source encoding device to encode data input by the source encoding device, and the second decoding model is used for a source decoding device to decode data output by the channel decoding device; and at least one of a weight of the first codec model or a weight of the second codec model is trained based on the channel codec algorithm information and the source codec algorithm information.

[0224] Optionally, the apparatus 400 can further include a memory 403.

[0225] The memory 403 can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memory. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically EPROM (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM) used as an external cache. By way of example, and not limitation, many forms of RAM can be used, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous dynamic RAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0226] Specifically, the memory 403 is configured to store program codes and instructions of the apparatus 400. Optionally, the memory 403 is further configured to store data obtained in the execution of the method 200 by the processor 401, such as channel coding algorithm information or source coding algorithm information.

[0227] Optionally, the memory 403 can be a separate device or integrated in the processor 401.

[0228] It should be noted that FIG. 4 only shows a simplified design of the apparatus 400. In actual applications, the apparatus 400 can further include necessary other elements, including but not limited to any number of communication interfaces, processors, selectors, memories, etc., and all apparatuses 400 that can implement the present application are within the protection scope of the present application.

[0229] In a possible design, the apparatus 400 can be a chip. Optionally, the chip can further include one or more memories for storing computer-executable instructions, and when the chip apparatus is running, the processor can execute the computer-executable instructions stored in the memory to enable the chip to perform the steps performed by the model training apparatus in the method 200.

[0230] Optionally, the chip device can be a field programmable gate array, an application specific integrated chip, a system chip, a central processing unit, a network processing unit, a digital signal processing circuit, a micro controller, and can also be a programmable controller or other integrated chip.

[0231] The embodiment of the present application further provides a computer readable storage medium, which stores computer instructions, and when the computer instructions run on a computer, the model training method described in the above method embodiment is realized.

[0232] The embodiment of the present application further provides a computer program product, and when the computer program product runs on a processor, the model training method described in the above method embodiment is realized.

[0233] The model training device, the computer readable storage medium, the computer program product or the chip provided by the embodiment of the present application are all used for executing the corresponding method provided above, and therefore the beneficial effects that can be achieved can refer to the beneficial effects described in the corresponding method provided above, and will not be described here again.

[0234] It should be understood that, in various embodiments of the present application, the size of the serial number of each process does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0235] Those skilled in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0236] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiment, and will not be described here again.

[0237] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other manners. For example, the described device embodiments are merely schematic. The division of the units is merely logical function division. There can be another division manner for the actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0238] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0239] In addition, each functional unit in the various embodiments of the present application can be integrated into a processing unit, or each unit can be a physically independent unit, or two or more units can be integrated into one unit.

[0240] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the part of the technical solutions that essentially contribute to the prior art can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes various media that can store program codes, such as U disk, mobile hard disk, ROM, RAM, magnetic disk or optical disk, etc.

[0241] The above description is merely a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A model training method, characterized in that, The method comprises: obtaining channel codec algorithm information and source codec algorithm information, the channel codec algorithm information being used to indicate a model structure of a first codec model, and the source codec algorithm information being used to indicate a model structure of a second codec model, wherein the first codec model comprises at least one of a first encoding model or a first decoding model, the first encoding model being used for a channel encoding device to encode data output by a source encoding device, and the first decoding model being used for a channel decoding device to decode data output by the channel encoding device and transmitted through a channel, and the second codec model comprises at least one of a second encoding model or a second decoding model, the second encoding model being used for the source encoding device to encode data input by the source encoding device, and the second decoding model being used for a source decoding device to decode data output by the channel decoding device; training at least one of a weight of the first codec model or a weight of the second codec model based on the channel codec algorithm information and the source codec algorithm information.

2. The method of claim 1, wherein, The channel codec algorithm information is further used to indicate a weight corresponding to the model structure of the first codec model, and the training of the at least one of the weight of the first codec model or the weight of the second codec model based on the channel codec algorithm information and the source codec algorithm information comprises: training a weight corresponding to the model structure of the second codec model based on the channel codec algorithm information and the source codec algorithm information.

3. The method of claim 1, wherein, The source codec algorithm information is further used to indicate a weight corresponding to the model structure of the second codec model, and the training of the at least one of the weight of the first codec model or the weight of the second codec model based on the channel codec algorithm information and the source codec algorithm information comprises: training a weight corresponding to the model structure of the first codec model based on the channel codec algorithm information and the source codec algorithm information.

4. The method of claim 1, wherein, The model structure of the first codec model comprises a first sub-model structure and a second sub-model structure, the channel codec algorithm information is further used to indicate a weight corresponding to the first sub-model structure of the first codec model, and the training of the at least one of the weight of the first codec model or the weight of the second codec model based on the channel codec algorithm information and the source codec algorithm information comprises: training a weight corresponding to the second sub-model structure of the first codec model and a weight corresponding to the model structure of the second codec model based on the channel codec algorithm information and the source codec algorithm information.

5. The method of claim 1, wherein, The model structure of the second codec model comprises a first sub-model structure and a second sub-model structure, the source codec algorithm information is further used to indicate a weight corresponding to the first sub-model structure of the second codec model, and the training of the at least one of the weight of the first codec model or the weight of the second codec model based on the channel codec algorithm information and the source codec algorithm information comprises: The second sub-model structure corresponding to the weight of the second coding model is trained based on the channel coding algorithm information and the source coding algorithm information.

6. The method of claim 1, wherein, The model structure of the first coding model includes a first sub-model structure and a second sub-model structure, and the channel coding algorithm information is further used to indicate the weight corresponding to the first sub-model structure of the first coding model; the model structure of the second coding model includes a first sub-model structure and a second sub-model structure, and the source coding algorithm information is further used to indicate the weight corresponding to the first sub-model structure of the second coding model; and the at least one of the weight of the first coding model or the weight of the second coding model is trained based on the channel coding algorithm information and the source coding algorithm information. The second sub-model structure corresponding to the weight of the second coding model is trained based on the channel coding algorithm information and the source coding algorithm information.

7. The method of claim 1, wherein, The model structure of the first coding model includes a first sub-model structure and a second sub-model structure, and the channel coding algorithm information is further used to indicate the weight corresponding to the first sub-model structure of the first coding model; the source coding algorithm information is further used to indicate the weight corresponding to the model structure of the second coding model; and the at least one of the weight of the first coding model or the weight of the second coding model is trained based on the channel coding algorithm information and the source coding algorithm information. The second sub-model structure corresponding to the weight of the first coding model is trained based on the channel coding algorithm information.

8. The method of claim 1, wherein, The model structure of the second coding model includes a first sub-model structure and a second sub-model structure, and the source coding algorithm information is further used to indicate the weight corresponding to the first sub-model structure of the second coding model; the channel coding algorithm information is further used to indicate the weight corresponding to the model structure of the first coding model; and the at least one of the weight of the first coding model or the weight of the second coding model is trained based on the channel coding algorithm information and the source coding algorithm information. The second sub-model structure corresponding to the weight of the second coding model is trained based on the channel coding algorithm information and the source coding algorithm information.

9. The method according to any one of claims 1-8, characterized in that, The at least one of the weight of the first coding model or the weight of the second coding model is trained based on the channel coding algorithm information and the source coding algorithm information. The at least one of the weight of the first coding model or the weight of the second coding model is trained based on the channel coding algorithm information, the source coding algorithm information and auxiliary information; The auxiliary information includes at least one of the following information: channel information, used to indicate the influence of a channel on data transmission; or operation information used for indicating at least one of the following operations: a first operation performed before the channel coding device encodes data input into the channel coding device, a second operation performed after the channel coding device encodes data input into the channel coding device, a third operation performed before the channel decoding device decodes data input into the channel decoding device, or a fourth operation performed after the channel decoding device decodes data input into the channel decoding device; or auxiliary data information used for indicating at least one of the following auxiliary data: first auxiliary data generated between the source coding device and the channel coding device, second auxiliary data generated between the channel coding device and the channel, third auxiliary data generated between the channel and the channel decoding device, or fourth auxiliary data generated between the channel decoding device and the source decoding device, the first auxiliary data being used for assisting the channel coding device to perform the first operation, the second auxiliary data being used for assisting the channel coding device to perform the second operation, the third auxiliary data being used for assisting the channel decoding device to perform the third operation, and the fourth auxiliary data being used for assisting the channel decoding device to perform the fourth operation; or source data information used for indicating input data of the source coding device.

10. The method of claim 9, wherein, Before the training of at least one of the weight of the first coding and decoding model or the weight of the second coding and decoding model based on the channel coding algorithm information and the source coding algorithm information, the method further comprises: receiving the auxiliary information.

11. The method according to any one of claims 1-10, characterized in that, The method further comprises: sending at least one of the weight of the first coding and decoding model or the weight of the second coding and decoding model trained.

12. The method according to any one of claims 1-11, characterized in that, Before the obtaining of the channel coding algorithm information and the source coding algorithm information, the method further comprises: sending model training capability information used for indicating that the model training device has the capability of training a coding and decoding model.

13. A model training apparatus, comprising: The device comprises a processor and a communication interface, the processor is coupled with the communication interface, and the processor is used for: obtaining channel coding algorithm information and source coding algorithm information, the channel coding algorithm information being used for indicating a model structure of a first coding and decoding model, and the source coding algorithm information being used for indicating a model structure of a second coding and decoding model, wherein the first coding and decoding model comprises at least one of a first coding model or a first decoding model, the first coding model being used for a channel coding device to encode data output by a source coding device, and the first decoding model being used for a channel decoding device to decode data output by the channel coding device and transmitted through a channel, and the second coding and decoding model comprises at least one of a second coding model or a second decoding model, the second coding model being used for the source coding device to encode data input into the source coding device, and the second decoding model being used for a source decoding device to decode data output by the channel decoding device. Train at least one of the weight of the first coding model or the weight of the second coding model based on the channel coding algorithm information and the source coding algorithm information.

14. The apparatus of claim 13, wherein, The channel coding algorithm information is also used to indicate the weight corresponding to the model structure of the first coding model, and the processor is specifically configured to: Train the weight corresponding to the model structure of the second coding model based on the channel coding algorithm information and the source coding algorithm information.

15. The apparatus of claim 13, wherein, The source coding algorithm information is also used to indicate the weight corresponding to the model structure of the second coding model, and the processor is specifically configured to: Train the weight corresponding to the model structure of the first coding model based on the channel coding algorithm information and the source coding algorithm information.

16. The apparatus of claim 13, wherein, The model structure of the first coding model includes a first sub-model structure and a second sub-model structure, the channel coding algorithm information is also used to indicate the weight corresponding to the first sub-model structure of the first coding model, and the processor is specifically configured to: Train the weight corresponding to the second sub-model structure of the first coding model and the weight corresponding to the model structure of the second coding model based on the channel coding algorithm information and the source coding algorithm information.

17. The apparatus of claim 13, wherein, The model structure of the second coding model includes a first sub-model structure and a second sub-model structure, the source coding algorithm information is also used to indicate the weight corresponding to the first sub-model structure of the second coding model, and the processor is specifically configured to: Train the weight corresponding to the second sub-model structure of the second coding model and the weight corresponding to the model structure of the first coding model based on the channel coding algorithm information and the source coding algorithm information.

18. The apparatus of claim 13, wherein, The model structure of the first coding model includes a first sub-model structure and a second sub-model structure, the channel coding algorithm information is also used to indicate the weight corresponding to the first sub-model structure of the first coding model, the model structure of the second coding model includes a first sub-model structure and a second sub-model structure, and the source coding algorithm information is also used to indicate the weight corresponding to the first sub-model structure of the second coding model, and the processor is specifically configured to: Train the weight corresponding to the second sub-model structure of the second coding model and the weight corresponding to the second sub-model structure of the first coding model based on the channel coding algorithm information and the source coding algorithm information.

19. The apparatus of claim 13, wherein, The model structure of the first coding model includes a first sub-model structure and a second sub-model structure, the channel coding algorithm information is also used to indicate the weight corresponding to the first sub-model structure of the first coding model, and the source coding algorithm information is also used to indicate the weight corresponding to the model structure of the second coding model, and the processor is specifically configured to: Train the weight corresponding to the second sub-model structure of the first coding model based on the channel coding algorithm information and the source coding algorithm information.

20. The apparatus of claim 13, wherein, The model structure of the second codec model comprises a first sub-model structure and a second sub-model structure, the source codec algorithm information is further used for indicating the weight corresponding to the first sub-model structure of the second codec model, and the channel codec algorithm information is further used for indicating the weight corresponding to the model structure of the first codec model. The processor is specifically configured to:

21. The apparatus of any of claims 13-20, wherein, train the weight corresponding to the second sub-model structure of the second codec model based on the channel codec algorithm information and the source codec algorithm information. The processor is specifically configured to: train at least one of the weight of the first codec model or the weight of the second codec model based on the channel codec algorithm information, the source codec algorithm information and auxiliary information; The auxiliary information comprises at least one of the following information: channel information, used for indicating the influence of a channel on data transmission; or operation information, used for indicating at least one of the following operations: a first operation performed before the channel coding device encodes data input into the channel coding device, a second operation performed after the channel coding device encodes data input into the channel coding device, a third operation performed before the channel decoding device decodes data input into the channel decoding device or a fourth operation performed after the channel decoding device decodes data input into the channel decoding device; or auxiliary data information, used for indicating at least one of the following auxiliary data: first auxiliary data generated between the source coding device and the channel coding device, second auxiliary data generated between the channel coding device and the channel, third auxiliary data generated between the channel and the channel decoding device or fourth auxiliary data generated between the channel decoding device and the source decoding device, the first auxiliary data being used for assisting the channel coding device to perform the first operation, the second auxiliary data being used for assisting the channel coding device to perform the second operation, the third auxiliary data being used for assisting the channel decoding device to perform the third operation, and the fourth auxiliary data being used for assisting the channel decoding device to perform the fourth operation; or 22. The apparatus of claim 21, wherein, source data information, used for indicating input data of the source coding device. The processor is further configured to:

23. The apparatus of any of claims 13-22, wherein, receive the auxiliary information through the communication interface before training at least one of the weight of the first codec model or the weight of the second codec model based on the channel codec algorithm information and the source codec algorithm information. The processor is further configured to:

24. The apparatus of any of claims 13-23, wherein, send at least one of the weight of the first codec model or the weight of the second codec model trained through the communication interface. The processor is further configured to: send model training capability information through the communication interface before acquiring the channel codec algorithm information and the source codec algorithm information, the model training capability information being used for indicating that a model training device has the capability of training a codec model.

25. A computer readable storage medium, characterized in that, A computer program product for storing a computer program which, when run by a processor, implements the method according to any one of claims 1-12.

26. A computer program product, characterised in that, A computer program product for storing a computer program which, when run by a processor, implements the method according to any one of claims 1-12.

Images