Communication system, communication method, learning device, learning method, and program

Abstract

Provided is a communication system comprising: a tokenization unit that tokenizes, into a plurality of tokens, a bit string for transmission; a compression unit that generates compressed data by dimensionally reducing the plurality of tokens by using a transformer encoder; a compressed data transmission unit that transmits the compressed data; a compressed data reception unit that receives the compressed data transmitted by the compressed data transmission unit; and an output data generation unit that, by using a transformer decoder, restores the compressed data received by the compressed data reception unit, and thereby generates output data.

Description

Communication System, Communication Method, Learning Device, Learning Method, and Program

[0001] The present invention relates to a communication system, a communication method, a learning device, a learning method, and a program.

[0002] In Patent Document 1, when an information processing device causes a cloud server to process all or part of the processing of an application including an AI (Artificial Intelligence) / ML (Machine Learning) model via wireless communication, the processing of the AI / ML model is described as being processed by the NWDAF (Network Data Analytics Function) of a 5G system. Patent Document 2 describes a decoding device including a circuit and a memory connected to the circuit. In operation, the circuit decodes expression data indicating information expressed by a person, and generates a person-corresponding image, which is an image corresponding to the person, via a neural network according to the expression data and at least one profile image of the person, and outputs the person-corresponding image. [Prior Art Documents] [Patent Documents] [Patent Document 1] Japanese Unexamined Patent Application Publication No. 2024-029281 [Patent Document 2] Japanese Unexamined Patent Application Publication No. 2023-195426

[0003] In a communication protocol, in each layer of the TCP (Transmission Control Protocol) / IP (Internet Protocol) model, a PDU (Protocol Data Unit) in each layer is formed by attaching a header to the data (payload) of the upper layer. A packet corresponds to the Internet layer, and a frame corresponds to the link layer. Although there are compression techniques in each PDU formation, there is room for improvement in compression efficiency.

[0004] In the communication system according to this embodiment, compression is performed using an Encoder-Decoder Transformer model. For example, in the communication system, PDUs are compressed using an Encoder-Decoder Transformer model. This not only compresses the data, but also learns and optimizes the structure of the information necessary when forming the PDU, effectively utilizing data patterns that could not be captured by conventional compression techniques, and potentially achieving a dramatic improvement in compression efficiency.

[0005] According to one embodiment of the present invention, a communication system is provided. The communication system may include a tokenization unit that tokenizes a bit sequence to be transmitted into a plurality of tokens. The communication system may include a compression unit that generates compressed data by reducing the dimensionality of the plurality of tokens using a TransformerEncoder. The communication system may include a compressed data transmission unit that transmits the compressed data. The communication system may include a compressed data receiving unit that receives the compressed data transmitted by the compressed data transmission unit. The communication system may include an output data generation unit that restores the compressed data received by the compressed data receiving unit using a TransformerDecoder and generates output data.

[0006] In the communication system, the tokenization unit may tokenize the bit sequence of the HTTP response body to be transmitted into the plurality of tokens, and the compressed data transmission unit may attach a header to the compressed data and transmit it. The communication system may include a first communication device having the tokenization unit, the compression unit, and the compressed data transmission unit, and a second communication device having the receiving unit and the output data generation unit, the second communication device may further include a request transmission unit that transmits an HTTP request to the first communication device in which the compression algorithm includes at least the TransformerEncoder, the first communication device may further include a request receiving unit that receives the HTTP request, and the tokenization unit may tokenize the bit sequence of the HTTP response body to be transmitted as a response to the HTTP request into the plurality of tokens.

[0007] Any of the aforementioned communication systems may further include a learning execution unit that generates the TransformerEncoder and the TransformerDecoder by performing machine learning using the bit sequences of multiple HTTP response bodies.

[0008] Any of the aforementioned communication systems may include an information processing infrastructure having a RAN control unit for controlling RAN and an AI processing unit for performing AI processing, and the AI ​​processing unit may have the learning execution unit.

[0009] In any of the above communication systems, the tokenization unit may tokenize the bit sequence of the Ethernet® frame to be transmitted into the plurality of tokens. The communication system may include a first communication device having the tokenization unit, the compression unit, and the compressed data transmission unit, and a second communication device having the receiving unit and the output data generation unit, wherein the second communication device may further include a request transmission unit that transmits an HTTP request to the first communication device, the compression algorithm of which includes at least the TransformerEncoder, and the first communication device may further include a request receiving unit that receives the HTTP request, and the tokenization unit may tokenize the bit sequence of the Ethernet frame to be transmitted as a response to the HTTP request into the plurality of tokens.

[0010] Any of the aforementioned communication systems may further include a learning execution unit that generates the TransformerEncoder and the TransformerDecoder by performing machine learning using the bit sequences of multiple Ethernet frames.

[0011] Any of the aforementioned communication systems may include an information processing infrastructure having a RAN control unit for controlling RAN and an AI processing unit for performing AI processing, and the AI ​​processing unit may have the learning execution unit.

[0012] Any of the above communication systems may further include a learning execution unit that generates a first TransformerEncoder and a first TransformerDecoder by performing machine learning using the bit sequences of a plurality of HTTP response bodies, and generates a second TransformerEncoder and a second TransformerDecoder by performing machine learning using the bit sequences of a plurality of Ethernet frames, and the compression unit may generate the compressed data by reducing the dimensionality using the first TransformerEncoder when the tokenization unit tokenizes the bit sequence of the HTTP response body to be transmitted into the plurality of tokens, and generate the compressed data by reducing the dimensionality using the second TransformerEncoder when the tokenization unit tokenizes the bit sequence of the Ethernet frame to be transmitted into the plurality of tokens. When the learning execution unit generates the second TransformerEncoder and the second TransformerDecoder, it may prioritize the accuracy of compression and decompression over the accuracy of compression and decompression when generating the first TransformerEncoder and the first TransformerDecoder.

[0013] According to one embodiment of the present invention, a communication method is provided that is performed by a communication system comprising a first communication device and a second communication device. The communication method may include a tokenization step in which the first communication device tokenizes a bit sequence to be transmitted into a plurality of tokens. The communication method may include a compression step in which the first communication device generates compressed data by reducing the dimensionality of the plurality of tokens using a TransformerEncoder. The communication method may include a compressed data transmission step in which the first communication device transmits the compressed data. The communication method may include a compressed data reception step in which the second communication device receives the compressed data transmitted by the compressed data transmission unit. The communication method may include an output data generation step in which the second communication device restores the compressed data received in the compressed data reception step using a TransformerDecoder to generate output data.

[0014] According to one embodiment of the present invention, a learning device is provided. The learning device may include a learning data acquisition unit that acquires learning data including bit sequences communicated between a plurality of communication devices. The learning device may include a learning execution unit that, for each of the plurality of bit sequences included in the learning data, tokenizes the bit sequence into a plurality of tokens, uses the plurality of tokens as input to a TransformerEncoder, uses the output to be decompressed from the TransformerDecoder as the plurality of tokens, sets the pre-registered compression / decompression accuracy and compression efficiency as loss functions, and performs learning to generate a trained TransformerEncoder and a trained TransformerDecoder.

[0015] According to one embodiment of the present invention, a learning method performed by a computer is provided. The learning method may include a learning data acquisition step of acquiring learning data that includes bit sequences communicated between a plurality of communication devices. The learning method may include a learning execution step of tokenizing each of the plurality of bit sequences contained in the learning data into a plurality of tokens, using the plurality of tokens as input to a TransformerEncoder, using the output to be decompressed from the TransformerDecoder as the plurality of tokens, setting the pre-registered compression / decompression accuracy and compression efficiency as loss functions, and performing learning to generate a trained TransformerEncoder and a trained TransformerDecoder.

[0016] According to one embodiment of the present invention, a program is provided for causing a computer to execute the learning method.

[0017] AI processing can be categorized into two types: AI processing related to RAN control (sometimes referred to as RAN-controlled AI processing) and AI processing not related to RAN control (sometimes referred to as non-RAN-controlled AI processing).

[0018] An example of AI-based RAN control processing is RIC (RAN Intelligent Controller). RIC is a technology that uses AI to optimize RAN wireless resources and automate RAN operations. RIC includes Non-RT RIC (Non-Real Time RIC) and Near-RT RIC (Near-Real Time RIC). Non-RT RIC is sometimes called Centralized RIC. Non-RT RIC is located within the SMO (Service Management and Orchestration) that manages and orchestrates the RAN. Non-RT RIC generates and notifies policies related to RAN control and transmits information to Near-RT RIC. For example, a Non-RT RIC generates a learning model for RAN control by performing machine learning using data collected from the RAN, and sends it to a Near-RT RIC. A Near-RT RIC is sometimes called a Distributed RIC. Compared to a Non-RT RIC, a Near-RT RIC is located closer to the RAN nodes (RU (Radio Unit), DU (Distributed Unit), CU (Central Unit)) and performs control of the RAN nodes and resources. Compared to a Non-RT RIC, a Near-RT RIC performs processing with higher real-time capabilities. For example, a Near-RT RIC performs inference processing related to RAN control using the learning model obtained from a Non-RT RIC. RAN control AI processing is not limited to RICs.

[0019] Non-RAN controlled AI processing may be a so-called MEC AI application. Non-RAN controlled AI processing includes learning and inference processing of any AI that is not related to RAN control.

[0020] It should be noted that the above summary of the invention does not enumerate all the necessary features of the present invention. Furthermore, subcombinations of these features may also constitute an invention.

[0021] A schematic example of the communication system 10 is shown. A schematic example of the processing content in the communication system 10 is shown. A schematic example of the processing flow in the communication system 10 is shown. A schematic example of the processing content in the communication system 10 is shown. A schematic example of the learning content in the communication system 10 is shown. A schematic example of the functional configuration of the learning device 300 is shown. A schematic example of the functional configuration of the server 100 is shown. A schematic example of the functional configuration of the communication terminal 200 is shown. A schematic example of the environment to which the communication system 10 is applied is shown. A schematic example of the functional configuration of the distributed infrastructure 500 is shown. A schematic example of the hardware configuration of the computer 1200 that functions as the server 100, communication terminal 200, learning device 300, management infrastructure 400, or distributed infrastructure 500 is shown.

[0022] The present invention will be described below through embodiments, but these embodiments are not intended to limit the scope of the claims. Furthermore, not all combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0023] Figure 1 schematically shows an example of a communication system 10. The communication system 10 includes a server 100. The communication system 10 includes a communication terminal 200. The server 100 may be an example of a first communication device, and the communication terminal 200 may be an example of a second communication device.

[0024] The server 100 and the communication terminal 200 communicate via the network 50. The network 50 may include a cloud network. The network 50 may include the internet. The network 50 may include a mobile communication network. The mobile communication network may conform to any of the following mobile communication systems: LTE (Long Term Evolution) communication system, 5G (5th Generation) communication system, 3G (3rd Generation) communication system, and 6G (6th Generation) communication system or later.

[0025] Server 100 may be a server that provides data to the communication terminal 200. Server 100 may be, for example, an HTTP (Hyper Text Transfer Protocol) server.

[0026] The communication terminal 200 can be any type of device as long as it is capable of communicating with the server 100. For example, the communication terminal 200 may be a smartphone, a tablet, or a PC (Personal Computer).

[0027] The learning device 300 performs learning to compress part or all of the data exchanged between the server 100 and the communication terminal 200 using Transformer. For example, the learning device 300 generates a TransformerEncoder for compressing the data and a TransformerDecoder for restoring the data by performing machine learning using a large amount of part or all of the data exchanged between the server 100 and the communication terminal 200.

[0028] The learning device 300 provides the generated TransformerEncoder and TransformerDecoder to the server 100 and the communication terminal 200.

[0029] The learning device 300, for example, provides a TransformerEncoder to the server 100 and a TransformerDecoder to the communication terminal 200. As a result, the server 100 compresses the data it provides to the communication terminal 200 using the TransformerEncoder, transmits the compressed data to the communication terminal 200, and the communication terminal 200 can decompress the received compressed data using the TransformerDecoder.

[0030] The learning device 300, for example, provides a TransformerEncoder to the communication terminal 200 and a TransformerDecoder to the server 100. As a result, the communication terminal 200 compresses the data to be sent to the server 100 using the TransformerEncoder, sends the compressed data to the server 100, and the server 100 can decompress the received compressed data using the TransformerDecoder.

[0031] The learning device 300 may provide both the TransformerEncoder and the TransformerDecoder to the server 100 and the communication terminal 200, respectively. This allows for bidirectional compression using the TransformerEncoder and decompression using the TransformerDecoder between the server 100 and the communication terminal 200.

[0032] Furthermore, when communicating between multiple communication terminals 200, a TransformerEncoder and a TransformerDecoder may be used. In this case, the first communication terminal 200 and the second communication terminal 200 that communicate may be examples of the first communication device and the second communication device.

[0033] Figure 2 schematically shows an example of the processing content in the communication system 10. Here, we will explain a case in which, when the server 100 sends an Ethernet frame 60 to the communication terminal 200 in response to an HTTP request from the communication terminal 200, the bit sequence of the HTTP response body 64 is compressed using a TransformerEncoder.

[0034] Server 100 tokenizes the bit sequence of the HTTP response body 64 into multiple tokens 72. For example, Server 100 tokenizes the bit sequence of the HTTP response body 64 into 16-bit (2-byte) units, generating 65,536 types of tokens 72, which are then input to the TransformerEncoder. The TransformerEncoder generates compressed data 80 by reducing the dimensionality of the multiple tokens 72. Server 100 adds a header 62 to the compressed data 80 and transmits it to the communication terminal 200.

[0035] This tokenization is equivalent to text tokenization in LLM (Large Language Model) using Transformer, for example. In LLM using Transformer, relationships between distant tokens can be efficiently learned for continuous data such as language. By applying this to bit sequences, the bit sequence can be compressed into an intermediate representation while taking context into account.

[0036] In this way, by making the HTTP response body 64 the target of compression, it effectively becomes equivalent to the HTTP compression currently implemented in gzip and brotil, making implementation and deployment to existing networks easier.

[0037] Figure 3 schematically shows an example of the processing flow in the communication system 10. Here, the TransformerEncoder 110 and TransformerDecoder 210 generated by the learning device 300 are provided to the server 100 and the communication terminal 200, and the starting state is when the user 20 accesses the server 100 via the communication terminal 200 and checks the contents of the data provided by the server 100 using the browser 202.

[0038] In response to a page transition instructed by user 20, the communication terminal 200 generates an HTTP request. In this example, the communication terminal 200 generates an HTTP request that includes tf, which indicates compression by Transformer, in Accept-Encoding, and sends it to the server 100. In the example shown in Figure 3, the communication terminal 200 includes gzip, br, and tf in Accept-Encoding. Note that tf is just one example, and any expression that indicates compression by Transformer is acceptable.

[0039] If server 100 supports tf, server 100 performs compression using tfF. In this example, server 100 supports tf. Server 100 inputs an encoding request for the HTTP response body 64 to be sent to communication terminal 200 to TransformerEncoder 110 and obtains the encoded HTTP response body 64 from TransformerEncoder 110. Server 100 sends an HTTP response to communication terminal 200 with a header 62 added to the HTTP response body 64. Server 100 includes tf in Content-Encoding.

[0040] The communication terminal 200 inputs a decoding request for the encoded HTTP response body 64 contained in the HTTP response to the TransformerDecoder 210, and obtains the decoded HTTP response body 64 from the TransformerDecoder 210. Then, the communication terminal 200 reconstructs the HTML and displays it to the user 20.

[0041] Conventionally, AI-based image compression, audio compression, and video compression technologies have been known, and the development of compression models for individual application data has progressed. In contrast, the communication system 10 according to this embodiment can provide a general-purpose compression technology by targeting the HTTP response body 64 for compression. In other words, the communication system 10 according to this embodiment can advance AI-based efficient data compression technology.

[0042] Figure 4 schematically shows an example of the processing content in the communication system 10. Here, a case where, in response to an http request from the communication terminal 200 to the server 100, the server 100 transmits an Ethernet frame 60 to the communication terminal 200 and compresses the bit sequence of the Ethernet frame 60 using a Transformer Encoder will be described.

[0043] The server 100 tokenizes the bit sequence of the Ethernet frame 60 into a plurality of tokens 72. For example, the server 100 tokenizes the bit sequence of the Ethernet frame 60 every 16 bits (2 bytes), generates 65,536 types of tokens 72, and inputs them to the Transformer Encoder. The Transformer Encoder generates compressed data 80 by reducing the dimension of the plurality of tokens 72. The server 100 transmits the compressed data 80 to the communication terminal 200.

[0044] As shown in FIG. 4, by making the Ethernet frame 60 the compression target, the versatility can be further enhanced and the compression efficiency can also be improved.

[0045] Figure 5 schematically shows an example of the learning content by the learning device 300. In the learning in the communication system 10, the input and the output to be expanded are the same and can be obtained as the bit sequences of all PDUs in the communication between the server 100 and the communication terminal 200. The learning device 300 may perform machine learning using the acquired large number of bit sequences of PDUs.

[0046] As shown in the following mathematical formula 1, the learning device 300 may set a loss function considering both the accuracy of compression and decompression (cross-entropy loss, packet retransmission rate, subjective accuracy, etc.) and the compression efficiency, that is, the substantial data size (number of non-zero elements, low-rank approximation, etc.) in the intermediate representation, and perform learning.

[0047]

[0048] L ａｃｃ represents the loss of the accuracy of compression and decompression. L ｃｏｍｐThis represents the actual data size loss in the intermediate representation.

[0049] In its simplest form, only the ratio λ is set, as shown in Equation 2 below. The ratio λ may be set according to the priority given to the accuracy of compression and decompression, and the compression efficiency. It may also be a more complex function than Equation 2.

[0050]

[0051] The learning device 300 may adjust, for example, the compression and decompression accuracy and compression efficiency depending on the compression target. For example, when the learning device 300 is compressing Ethernet frames, it may prioritize the compression and decompression accuracy over the compression and decompression accuracy when learning compared to when the compression and decompression accuracy is compressing HTTP response bodies. This allows for prioritizing compression efficiency and contributing to improved communication efficiency in cases where degradation that is imperceptible to humans is acceptable, such as with HTTP response bodies, or where human-perceptible degradation is acceptable if it is small. However, in cases where high accuracy close to 100% is required, such as with Ethernet frames, for example in applications at the data link layer, learning can be performed with weighted emphasis on compression and decompression accuracy, taking into account frame discard and retransmission due to decompression failure.

[0052] Figure 6 schematically shows an example of the functional configuration of the learning device 300. The learning device 300 comprises a storage unit 312, a learning data acquisition unit 314, a learning execution unit 316, and a transmission unit 318.

[0053] The learning data acquisition unit 314 acquires learning data. The learning data acquisition unit 314 may receive learning data from an external source. The learning data acquisition unit 314 may acquire learning data from a storage medium. The learning data acquisition unit 314 stores the acquired learning data in the storage unit 312.

[0054] The training data includes multiple bit sequences communicated between multiple communication devices. The training data may include multiple bit sequences communicated between communication devices that provide the TransformerEncourager and TransformerDecoder to be generated. For example, the training data may include bit sequences of multiple PDUs. The training data may include bit sequences of multiple Ethernet frames. The training data may include, for example, bit sequences of multiple HTTP response bodies.

[0055] The learning execution unit 316 performs machine learning using the learning data stored in the memory unit 312 to generate a TransformerEncoder and a TransformerDecoder. The learning execution unit 316 may perform machine learning using multiple bit sequences included in the learning data. For each of the multiple bit sequences included in the learning data, the learning execution unit 316 tokenizes the bit sequence into multiple tokens, uses the multiple tokens as input to the TransformerEncoder, uses the output to be decompressed from the TransformerDecoder as the multiple tokens, sets the previously registered compression / decompression accuracy and compression efficiency as the loss function, and performs learning to generate a trained TransformerEncoder and a trained TransformerDecoder.

[0056] For example, the learning execution unit 316 performs machine learning using the bit sequences of multiple http response bodies. The learning execution unit 316 may perform machine learning using the bit sequences of multiple http response bodies by setting the pre-registered compression / decompression accuracy and compression efficiency as loss functions. For example, the learning execution unit 316 tokenizes each bit sequence of the multiple response bodies into multiple tokens, uses the multiple tokens as input to the TransformerEncoder, sets the output to be decompressed from the TransformerDecoder as the multiple tokens, sets the pre-registered compression / decompression accuracy and compression efficiency as loss functions, and performs learning. The learning execution unit 316 may perform learning so that the TransformerEncoder generates an intermediate representation for the input multiple tokens in order to achieve the pre-registered compression / decompression accuracy and compression efficiency.

[0057] For example, the learning execution unit 316 performs machine learning using the bit sequences of multiple Ethernet frames. The learning execution unit 316 may perform machine learning using the bit sequences of multiple Ethernet frames by setting the pre-registered compression / decompression accuracy and compression efficiency as loss functions. For example, for each bit sequence of multiple Ethernet frames, the learning execution unit 316 tokenizes the bit sequence into multiple tokens, uses the multiple tokens as input to the TransformerEncoder, sets the output to be decompressed from the TransformerDecoder as the multiple tokens, sets the pre-registered compression / decompression accuracy and compression efficiency as loss functions, and performs learning. The learning execution unit 316 may perform learning so that the TransformerEncoder generates an intermediate representation for the input multiple tokens in order to achieve the pre-registered compression / decompression accuracy and compression efficiency.

[0058] The learning execution unit 316 may adjust the compression accuracy and compression efficiency depending on the compression target. For example, when the learning execution unit 316 is compressing an Ethernet frame, it may prioritize the compression accuracy over the compression efficiency during learning. For example, when the learning execution unit 316 is compressing an HTTP response body, it may prioritize the compression efficiency over the compression accuracy during learning. When the learning execution unit 316 is compressing an Ethernet frame, it may prioritize the compression accuracy over the compression efficiency during learning compared to when the HTTP response body is compressing.

[0059] The learning execution unit 316 stores the generated TransformerEncoder and TransformerDecoder in the storage unit 312.

[0060] The transmitting unit 318 transmits the TransformerEncoder and TransformerDecoder stored in the storage unit 312. The transmitting unit 318 may transmit the TransformerEncoder to one of the two communicating devices and the TransformerDecoder to the other. The transmitting unit 318 may transmit the TransformerEncoder and the TransformerDecoder to both of the two communicating devices.

[0061] Figure 7 schematically shows an example of the functional configuration of server 100. Figure 8 schematically shows an example of the functional configuration of communication terminal 200.

[0062] The server 100 includes a storage unit 112, a receiving unit 114, a transmitting unit 116, a tokenization unit 118, and a compression unit 120. The communication terminal 200 includes a storage unit 212, a receiving unit 214, a transmitting unit 216, an output data generation unit 218, and an output control unit 220.

[0063] The storage unit 112 stores various types of data. The storage unit 112 may also store data to be provided to other devices. For example, the storage unit 112 stores page data and the like to be provided to other devices. The receiving unit 114 receives various types of data and stores them in the storage unit 112. The transmitting unit 116 transmits the data stored in the storage unit 112.

[0064] The receiving unit 114 may receive a TransformerEncoder from the learning device 300. The receiving unit 114 may also receive a TransformerDecoder from the learning device 300.

[0065] The storage unit 212 stores various types of data. The storage unit 212 may also store data to be provided to other devices. The receiving unit 214 receives various types of data and stores them in the storage unit 212. The transmitting unit 216 transmits the data stored in the storage unit 112.

[0066] The receiving unit 214 may receive a TransformerDecoder from the learning device 300. The receiving unit 214 may also receive a TransformerEncoder from the learning device 300.

[0067] The output data generation unit 218 generates output data. The output control unit 220 controls the output data generated by the output data generation unit 218 to be output. For example, the output control unit 220 displays the output data generated by the output data generation unit 218 on the display of the communication terminal 200.

[0068] The transmission unit 216 sends, for example, an HTTP request to the server 100 requesting page data. The transmission unit 216 may send an HTTP request to the server 100 that includes at least a TransformerEncoder in its compression algorithm. As a specific example, the transmission unit 216 generates an HTTP request that includes at least a TF indicating compression by Transformer in the Accept-Encoding and sends it to the server 100. The transmission unit 216 may be an example of a request transmission unit.

[0069] The receiving unit 114 receives the HTTP request sent by the transmitting unit 216. The receiving unit 114 may be an example of a request receiving unit.

[0070] The tokenization unit 118 tokenizes the bit sequence to be transmitted in response to the HTTP request received by the receiving unit 114 into multiple tokens. The compression unit 120 generates compressed data by reducing the dimensions of the multiple tokens tokenized by the tokenization unit 118 using the TransformerEncoder stored in the storage unit 112. The transmission unit 116 transmits the compressed data generated by the compression unit 120 to the communication terminal 200. The transmission unit 116 may be an example of a compressed data transmission unit. The receiving unit 214 receives the compressed data transmitted by the transmission unit 116. The receiving unit 214 may be an example of a compressed data reception unit. The output data generation unit 218 restores the compressed data received by the receiving unit 214 using the TransformerDecoder and generates output data. The output control unit 220 controls the output data generated by the output data generation unit 218 to output.

[0071] For example, the tokenization unit 118 tokenizes the bit sequence of the HTTP response body that the receiving unit 114 sends as a response to the HTTP request received by the receiving unit 114 into multiple tokens. The compression unit 120 generates compressed data by reducing the dimensions of the multiple tokens tokenized by the tokenization unit 118 using a TransformerEncoder stored in the storage unit 112. The transmission unit 116 adds a header to the compressed data generated by the compression unit 120 and transmits it to the communication terminal 200.

[0072] For example, the tokenization unit 118 tokenizes the bit sequence of the Ethernet frame that the receiving unit 114 sends as a response to the HTTP request it receives into multiple tokens. The compression unit 120 generates compressed data by reducing the dimensions of the multiple tokens tokenized by the tokenization unit 118 using a TransformerEncoder stored in the storage unit 112. The transmission unit 116 transmits the compressed data generated by the compression unit 120 to the communication terminal 200.

[0073] Figure 9 schematically shows an example of an environment to which the communication system 10 is applied. The environment shown in Figure 9 comprises a management infrastructure 400, a plurality of distributed infrastructures 500, and a plurality of wireless base stations 600. In this environment, the management infrastructure 400 and the plurality of distributed infrastructures 500 may cooperate to control the RAN 610 and perform AI processing. The RAN 610 provides mobile communication services to the communication terminal 200.

[0074] RAN610 may be a virtualized vRAN (Virtual RAN). RAN610 may also be a physical RAN. In this example, we will mainly explain the case where RAN610 is a vRAN.

[0075] The AI ​​processing performed by the management infrastructure 400 and the multiple distributed infrastructures 500 may include RAN control AI processing. The AI ​​processing performed by the communication system 10 may include non-RAN control AI processing.

[0076] The distributed infrastructure 500 may be data centers located in various locations. The distributed infrastructure 500 may be composed of multiple devices. The distributed infrastructure 500 may be implemented on a virtualization infrastructure consisting of multiple devices. The distributed infrastructure 500 may be implemented by a single device. That is, the distributed infrastructure 500 may be a distributed device. The distributed infrastructure 500 may function as a BBU (BaseBand Unit), and the wireless base station 600 may function as an RRU (Remote Radio Unit). The distributed infrastructure 500 may implement a CU. The distributed infrastructure 500 may implement a DU. The distributed infrastructure 500 may implement a UPF (User Plane Function).

[0077] The management infrastructure 400 may be a data center that manages multiple distributed infrastructures 500. The management infrastructure 400 may be composed of multiple devices. The management infrastructure 400 may be implemented on a virtualization infrastructure consisting of multiple devices. The management infrastructure 400 may be implemented by a single device. In other words, the management infrastructure 400 may be a management device.

[0078] The management infrastructure 400 may be called the Core Brain, and the distributed infrastructure 500 may be called the Regional Brain. Note that Figure 9 illustrates a case where a single-layer distributed infrastructure 500 is located below the management infrastructure 400, but it is not limited to this. The distributed infrastructure 500 may have multiple layers. For example, if two layers of distributed infrastructure 500 are located below the management infrastructure 400, the management infrastructure 400 may be called the Core Brain, the lower-layer distributed infrastructure 500 may be called the Regional Brain, and the further lower-layer distributed infrastructure 500 may be called the Sub-Regional Brain.

[0079] The distributed infrastructure 500 may have one or more CPUs (Central Processing Units). The distributed infrastructure 500 may have one or more GPUs (Graphics Processing Units). The distributed infrastructure 500 may have multiple superchips, each connected to a CPU and a GPU by an interconnect. This interconnect may be memory consistent and capable of achieving high bandwidth and low latency. Thus, the distributed infrastructure 500 may have CPU resources and GPU resources as computing resources.

[0080] The communication system 10 may include a distributed infrastructure 500. The distributed infrastructure 500 may function as a learning device 300. The distributed infrastructure 500 may be an example of an information processing infrastructure. The communication system 10 may also include a management infrastructure 400, and the management infrastructure 400 may function as a learning device 300. In this case, the management infrastructure 400 may be an example of an information processing infrastructure.

[0081] Figure 10 schematically shows an example of the functional configuration of the distributed infrastructure 500. The distributed infrastructure 500 includes a RAN control unit 510 that controls the RAN and an AI processing unit 520 that performs AI processing. The AI ​​processing unit 520 may implement a learning device 300. That is, the AI ​​processing unit 520 may have a storage unit 312, a learning data acquisition unit 314, a learning execution unit 316, and a transmission unit 318.

[0082] Figure 11 schematically shows an example of the hardware configuration of a computer 1200 that functions as a server 100, a communication terminal 200, a learning device 300, a management infrastructure 400, or a distributed infrastructure 500. A program installed on the computer 1200 can cause the computer 1200 to function as one or more "parts" of the apparatus according to this embodiment, or to cause the computer 1200 to execute operations associated with the apparatus according to this embodiment or such one or more "parts", and / or to cause the computer 1200 to execute a process or a stage of such process according to this embodiment. Such a program may be executed by the CPU 1212 to cause the computer 1200 to execute specific operations associated with some or all of the blocks in the flowcharts and block diagrams described herein.

[0083] The computer 1200 according to this embodiment includes a CPU 1212, a GPU 1213, a RAM 1214, and a graphics controller 1216, which are interconnected by a host controller 1210. The computer 1200 also includes input / output units such as a communication interface 1222, a storage device 1224, a DVD drive 1226, and an IC card drive, which are connected to the host controller 1210 via an input / output controller 1220. The DVD drive 1226 may be a DVD-ROM drive and a DVD-RAM drive, etc. The storage device 1224 may be a hard disk drive and a solid-state drive, etc. The computer 1200 also includes legacy input / output units such as a ROM 1230 and a keyboard, which are connected to the input / output controller 1220 via an input / output chip 1240.

[0084] The CPU 1212 operates according to the programs stored in the ROM 1230 and RAM 1214, thereby controlling each unit. The graphics controller 1216 acquires the image data generated by the CPU 1212 and stores it in the frame buffer provided in the RAM 1214 or within itself, so that the image data is displayed on the display device 1218.

[0085] The communication interface 1222 communicates with other electronic devices via a network. The storage device 1224 stores programs and data used by the CPU 1212 in the computer 1200. The DVD drive 1226 reads programs or data from the DVD-ROM 1227, etc., and provides them to the storage device 1224. The IC card drive reads programs and data from the IC card and / or writes programs and data to the IC card.

[0086] The ROM 1230 stores boot programs and / or hardware-dependent programs of the computer 1200, which are executed by the computer 1200 when activated. The input / output chip 1240 may also connect various input / output units to the input / output controller 1220 via USB ports, parallel ports, serial ports, keyboard ports, mouse ports, etc.

[0087] The program is provided on a computer-readable storage medium such as a DVD-ROM 1227 or an IC card. The program is read from the computer-readable storage medium and installed on a storage device 1224, RAM 1214, or ROM 1230, which are examples of computer-readable storage media, and executed by the CPU 1212. The information processing described within these programs is read by the computer 1200, resulting in coordination between the program and the various types of hardware resources described above. The apparatus or method may be configured to realize the operation or processing of information in accordance with the use of the computer 1200.

[0088] For example, when communication is performed between a computer 1200 and an external device, the CPU 1212 may execute a communication program loaded into the RAM 1214 and, based on the processing described in the communication program, instruct the communication interface 1222 to perform communication processing. Under the control of the CPU 1212, the communication interface 1222 reads transmission data stored in a transmission buffer area provided in a recording medium such as the RAM 1214, storage device 1224, DVD-ROM 1227, or IC card, transmits the read transmission data to the network, or writes received data received from the network to a reception buffer area or the like provided on the recording medium.

[0089] Furthermore, the CPU 1212 may read all or necessary parts of a file or database stored on an external recording medium such as a storage device 1224, a DVD drive 1226 (DVD-ROM 1227), or an IC card into the RAM 1214, and perform various types of processing on the data in the RAM 1214. The CPU 1212 may then write the processed data back to the external recording medium.

[0090] Various types of information, such as various types of programs, data, tables, and databases, may be stored on the recording medium and subjected to information processing. The CPU 1212 may perform various types of processing on the data read from the RAM 1214, including various types of operations, information processing, conditional judgments, conditional branching, unconditional branching, information retrieval / replacement, etc., as described throughout this disclosure and specified by the program instruction sequence, and write the results back to the RAM 1214. The CPU 1212 may also retrieve information in files, databases, etc., within the recording medium. For example, if a plurality of entries are stored in the recording medium, each having an attribute value of a first attribute associated with an attribute value of a second attribute, the CPU 1212 may search among the plurality of entries for an entry that matches the specified condition for the attribute value of the first attribute, read the attribute value of the second attribute stored in that entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

[0091] The program or software module described above may be stored on or near the computer 1200 in a computer-readable storage medium. Alternatively, a recording medium such as a hard disk or RAM provided within a server system connected to a dedicated communication network or the Internet can be used as a computer-readable storage medium, thereby providing the program to the computer 1200 via the network.

[0092] In this embodiment, blocks in the flowchart and block diagram may represent a stage in a process in which an operation is performed or a "part" of a device that has the role of performing an operation. A particular stage and "part" may be implemented by a dedicated circuit, a programmable circuit supplied with computer-readable instructions stored on a computer-readable storage medium, and / or a processor supplied with computer-readable instructions stored on a computer-readable storage medium. The dedicated circuit may include digital and / or analog hardware circuits, and may include integrated circuits (ICs) and / or discrete circuits. The programmable circuit may include reconfigurable hardware circuits, such as field-programmable gate arrays (FPGAs) and programmable logic arrays (PLAs), which include logical AND, logical OR, exclusive OR, negated AND, negated OR, and other logical operations, flip-flops, registers, and memory elements.

[0093] A computer-readable storage medium may include any tangible device capable of storing instructions to be executed by a suitable device, and as a result, a computer-readable storage medium having instructions stored therein will comprise a product that includes instructions that can be executed to create means for performing operations specified in a flowchart or block diagram. Examples of computer-readable storage media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, etc. More specific examples of computer-readable storage media may include floppy disks (registered trademark), diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), electrically erasable programmable read-only memory (EEPROM), static random access memory (SRAM), compact disk read-only memory (CD-ROM), digital versatile disk (DVD), Blu-ray (registered trademark) disk, memory stick, integrated circuit card, etc.

[0094] Computer-readable instructions may include assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk®, Java®, C++, and conventional procedural programming languages ​​such as the C programming language or similar programming languages.

[0095] Computer-readable instructions may be provided locally or via a wide area network (WAN) such as a local area network (LAN) or the internet to a processor or programmable circuit of a general-purpose computer, a special-purpose computer, or another programmable data processing device, so that the processor or programmable circuit of the programmable data processing device, such as a computer, may execute the computer-readable instructions to generate means for performing operations specified in a flowchart or block diagram. Here, the computer may be a PC (personal computer), a tablet computer, a smartphone, a workstation, a server computer, a general-purpose computer, or a special-purpose computer, and may also be a computer system in which multiple computers are connected. Such a computer system in which multiple computers are connected is also called a distributed computing system and is a computer in a broad sense. In a distributed computing system, multiple computers execute a program collectively by each computer executing a part of the program and passing data during program execution between computers as needed.

[0096] Examples of processors include computer processors, central processing units (CPUs), processing units, microprocessors, digital signal processors, controllers, and microcontrollers. A computer may have one or more processors. In a multiprocessor system with multiple processors, each processor executes a portion of the program, and the processors collectively execute the program by passing program execution data between them as needed. For example, in the execution of multitasks, each of the multiple processors may execute a portion of each task in small chunks by switching tasks at each time slice. In this case, which part of a program each processor executes changes dynamically. Which part of a program each of the multiple processors executes may also be statically determined by multiprocessor-aware programming.

[0097] By using the invention according to this embodiment, the compression efficiency of PDU can be improved, which can contribute to achieving Sustainable Development Goal (SDG) 9, "Build resilient infrastructure, promote inclusive and sustainable industrialization and foster innovation."

[0098] Although the present invention has been described above using embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various modifications or improvements can be made to the above embodiments. It will be clear from the claims that such modified or improved forms may also be included in the technical scope of the present invention.

[0099] It should be noted that the execution order of operations, procedures, steps, and stages in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not explicitly stated as "before" or "prior to," and that these can be performed in any order unless the output of a previous operation is used in a later operation. Even if the operation flow in the claims, specifications, and drawings is described using phrases such as "first," and "next," for convenience, this does not mean that it is mandatory to perform the operations in that order.

[0100] 10 System, 20 User, 50 Network, 60 Ethernet Frame, 62 Header, 64 HTTP Response Body, 72 Token, 80 Compressed Data, 100 Server, 110 TransformerEncoder, 112 Storage Unit, 114 Receiving Unit, 116 Transmitting Unit, 118 Tokenization Unit, 120 Compression Unit, 200 Communication Terminal, 202 Browser, 210 TransformerDecoder, 212 Storage Unit, 214 Receiving Unit, 216 Transmitting Unit, 218 Output Data Generation Unit, 220 Output Control Unit, 300 Learning Device, 312 Storage Unit, 314 Learning Data Acquisition Unit, 316 Learning Execution Unit, 318 Transmitting Unit, 400 Management Infrastructure, 500 Distributed Infrastructure, 510 RAN Control Unit, 520 AI processing unit, 600 Wireless base station, 610 RAN, 1200 Computer, 1210 Host controller, 1212 CPU, 1213 GPU, 1214 RAM, 1216 Graphics controller, 1218 Display device, 1220 Input / Output controller, 1222 Communication interface, 1224 Storage device, 1226 DVD drive, 1227 DVD-ROM, 1230 ROM, 1240 Input / Output chip

Claims

1. A communication system comprising: a tokenization unit that tokenizes a bit string to be transmitted into multiple tokens; a compression unit that generates compressed data by reducing the dimensions of the multiple tokens using a Transformer Encoder; a compressed data transmission unit that transmits the compressed data; a compressed data receiving unit that receives the compressed data transmitted by the compressed data transmission unit; and an output data generation unit that restores the compressed data received by the compressed data receiving unit using a Transformer Decoder to generate output data.

2. The communication system according to claim 1, wherein the tokenization unit tokenizes the bit sequence of the HTTP response body to be transmitted into the plurality of tokens, and the compressed data transmission unit transmits the compressed data with a header attached.

3. The communication system according to claim 2, comprising: a first communication device having the tokenization unit, the compression unit, and the compressed data transmission unit; and a second communication device having the compressed data reception unit and the output data generation unit, wherein the second communication device further has a request transmission unit that transmits an HTTP request to the first communication device, the compression algorithm comprising at least the TransformerEncoder; the first communication device further has a request reception unit that receives the HTTP request; and the tokenization unit tokenizes the bit sequence of the HTTP response body to be transmitted as a response to the HTTP request into the plurality of tokens.

4. The communication system according to claim 2 or 3, further comprising a learning execution unit that generates the TransformerEncoder and the TransformerDecoder by performing machine learning using the bit sequences of multiple HTTP response bodies.

5. The communication system according to claim 4, comprising an information processing platform having a RAN control unit for controlling RAN and an AI processing unit for performing AI processing, wherein the AI ​​processing unit has the learning execution unit.

6. The communication system according to claim 1, wherein the tokenization unit tokenizes the bit sequence of the Ethernet frame to be transmitted into the plurality of tokens.

7. The communication system according to claim 6, comprising: a first communication device having the tokenization unit, the compression unit, and the compressed data transmission unit; and a second communication device having the compressed data reception unit and the output data generation unit, wherein the second communication device further has a request transmission unit that transmits an HTTP request to the first communication device, the compression algorithm comprising at least the TransformerEncoder; the first communication device further has a request reception unit that receives the HTTP request; and the tokenization unit tokenizes the bit sequence of the Ethernet frame to be transmitted as a response to the HTTP request into the plurality of tokens.

8. The communication system according to claim 6 or 7, further comprising a learning execution unit that generates the TransformerEncoder and the TransformerDecoder by performing machine learning using the bit sequences of multiple Ethernet frames.

9. The communication system according to claim 8, comprising an information processing platform having a RAN control unit for controlling RAN and an AI processing unit for performing AI processing, wherein the AI ​​processing unit has the learning execution unit.

10. The communication system according to claim 1, further comprising a learning execution unit that generates a first TransformerEncoder and a first TransformerDecoder by performing machine learning using the bit sequences of a plurality of HTTP response bodies, and generates a second TransformerEncoder and a second TransformerDecoder by performing machine learning using the bit sequences of a plurality of Ethernet frames, wherein the compression unit generates the compressed data by reducing the dimensionality using the first TransformerEncoder when the tokenization unit tokenizes the bit sequence of the HTTP response body to be transmitted into the plurality of tokens, and generates the compressed data by reducing the dimensionality using the second TransformerEncoder when the tokenization unit tokenizes the bit sequence of the Ethernet frame to be transmitted into the plurality of tokens.

11. The communication system according to claim 10, wherein when the learning execution unit generates the second TransformerEncoder and the second TransformerDecoder, it prioritizes the accuracy of compression and decompression over the accuracy of compression and decompression of compression and decompression, compared to when it generates the first TransformerEncoder and the first TransformerDecoder.

12. A communication method performed by a communication system comprising a first communication device and a second communication device, comprising: a tokenization step in which the first communication device tokenizes a bit sequence to be transmitted into a plurality of tokens; a compression step in which the first communication device generates compressed data by reducing the dimensionality of the plurality of tokens using a TransformerEncoder; a compressed data transmission step in which the first communication device transmits the compressed data; a compressed data reception step in which the second communication device receives the compressed data transmitted in the compressed data transmission step; and an output data generation step in which the second communication device restores the compressed data received in the compressed data reception step using a TransformerDecoder to generate output data.

13. A learning device comprising: a learning data acquisition unit that acquires learning data including bit sequences communicated between multiple communication devices; and a learning execution unit that tokenizes each of the multiple bit sequences contained in the learning data into multiple tokens, uses the multiple tokens as input to a TransformerEncoder, uses the output to be decompressed from the TransformerDecoder as the multiple tokens, sets the pre-registered compression / decompression accuracy and compression efficiency as loss functions, and performs learning to generate a trained TransformerEncoder and a trained TransformerDecoder.

14. A learning method performed by a computer, comprising: a learning data acquisition step of acquiring learning data including bit sequences communicated between multiple communication devices; and a learning execution step of tokenizing each of the multiple bit sequences contained in the learning data into multiple tokens, using the multiple tokens as input to a TransformerEncoder, using the output to be decompressed from the TransformerDecoder as the multiple tokens, setting the pre-registered compression / decompression accuracy and compression efficiency as loss functions, and performing learning to generate a trained TransformerEncoder and a trained TransformerDecoder.

15. A program for causing a computer to perform the learning method described in claim 14.

Images