Image transmitter, digital communication system, image transmission method, and program

Abstract

An image transmitter (10, 10a) comprises: an image acquisition unit (11) that acquires an image; a compression rate management unit that manages a plurality of compression rates at which desired image quality can be obtained for the acquired image; a compression rate selection unit (32) that selects an optimal compression rate from among the plurality of compression rates managed by the compression rate management unit; a first encoding unit (transmission encoding unit (33)) that encodes the acquired image using a machine learning model; a compression rate selection result embedding unit (34) that embeds metadata indicating the compression rate selected by the compression rate selection unit (32) in the image that is encoded by the first encoding unit (33); and a transmission unit (signal transmission unit (35)) that transmits the image in which the metadata is embedded by the compression rate selection result embedding unit (34) and which is encoded by the first encoding unit (transmission encoding unit (33)).

Description

Image transmitter, digital communication system, image transmission method and program

[0001] The present disclosure relates to an image transmitter, a digital communication system, an image transmission method, and a program.

[0002] In modern digital communication systems, a two-stage encoding process prevails for video transmission.

[0003] For example, Non-Patent Document 1 discloses a transmitter and a receiver.

[0004] At the transmitter, data is typically compressed before being sent to the receiver. This reduces the amount of data, saving bandwidth and allowing for efficient use of storage capacity. The compressed data is then added with redundancy by a channel encoder. This redundancy is known as an error-correcting code. By adding redundancy, the channel encoder increases resistance to noise and errors that occur on the communication channel. This coding process provides a certain level of protection against noise and errors that occur during communication, and adding redundancy improves the reliability and quality of communication. The encoded data is then modulated. Modulation is the process of converting a digital or analog signal into a format suitable for the communication channel.

[0005] The receiver receives the modulated signal from the transmitter and decodes it in the reverse order of transmission, removing redundancies from the modulated signal to restore the original compressed data.

[0006] Non-Patent Document 2 discloses a joint source-channel encoder using deep learning. The joint source-channel encoder does not use coding for compression and error correction, but instead uses Deep Joint Source-Channel Coding Modulation (Deep JSCC), in which the encoder and decoder functions are parameterized by two convolutional neural networks (CNNs).

[0007] CE Shannon, "A mathematical theory of communication," in The Bell System Technical Journal, vol. 27, no. 3, pp. 379-423, July 1948, doi: 10.1002 / j.1538-7305.1948.tb01338.xE Bourtsoulatze, D. Burth Kurka and D. Gunduz, "Deep Joint Source-Channel Coding for Wireless Image Transmission," in IEEE Transactions on Cognitive Communications and Networking, vol. 5, no. 3, pp. 567-579, Sept. 2019, doi: 10.1109 / TCCN.2019.2919300.

[0008] For example, as described in Non-Patent Document 1, when considering the transmission of digital image data over the communication lines of a fifth-generation mobile system, it is common to combine JPEG coding, which is a source coding, with LDPC (Low-Density Parity-check Code) coding, which is a channel coding. However, the JPEG coding method suffers from a phenomenon known as the cliff effect, in which a small number of bit errors corrupts most of the image, making it impossible to reproduce. Therefore, a joint source-channel coder using Deep JSCC, as described in Non-Patent Document 2, has attracted attention.

[0009] When compressing an image using Deep JSCC, unlike JPEG compression, the compression rate is fixed and does not depend on the entropy of the image. In other words, if the compression rate is set to suit a complex image, there is ample room for compression for images with low entropy, such as sky or sea. On the other hand, if the compression rate is set to suit an image with low entropy, there is an issue that the quality of complex images with high entropy will be significantly degraded.

[0010] Therefore, an object of the present disclosure is to provide an image transmitter or the like that can suppress degradation of image quality and obtain a desired image quality.

[0011] In order to achieve the above object, an image transmitter according to one aspect of the present disclosure includes an image acquisition unit that acquires an image, a compression rate management unit that manages multiple compression rates that result in a desired image quality for the acquired image, a compression rate selection unit that selects an optimal compression rate from the multiple compression rates managed by the compression rate management unit, a first encoding unit that encodes the acquired image using a machine learning model, a compression rate selection result embedding unit that embeds metadata indicating the compression rate selected by the compression rate selection unit into the image to be encoded by the first encoding unit, and a transmitting unit that transmits the image in which the compression rate selection result embedding unit embeds the metadata and which has been encoded by the first encoding unit.

[0012] In addition, in order to achieve the above-mentioned object, a digital communication system according to one embodiment of the present disclosure comprises an image transmitter and an image receiver, wherein the image receiver has a receiving unit that receives the image transmitted by the transmitting unit of the image transmitter, a decoding unit that decodes the received image using a machine learning model, and a compression rate selection result reading unit that reads the metadata embedded in the image.

[0013] In addition, in order to achieve the above-mentioned object, an image transmission method according to one aspect of the present disclosure includes an image acquisition unit acquiring an image, a compression rate management unit managing multiple compression rates that result in a desired image quality for the acquired image, a compression rate selection unit selecting an optimal compression rate from the multiple compression rates managed by the compression rate management unit, a first encoding unit encoding the acquired image using a machine learning model, a compression rate selection result embedding unit embedding metadata indicating the compression rate selected by the compression rate selection unit into the image to be encoded by the first encoding unit, and a transmitting unit transmitting the image encoded by the first encoding unit after the compression rate selection result embedding unit has embedded the metadata.

[0014] In order to achieve the above object, a program according to one aspect of the present disclosure is a program for causing a computer to execute an image transmission method.

[0015] According to the image transmitter and the like according to the present disclosure, it is possible to suppress degradation of image quality and obtain a desired image quality.

[0016] FIG. 1 is a block diagram showing an image transmitter according to Embodiment 1. FIG. 2 is an explanatory diagram showing an example of Deep JSCC. FIG. 3 is a flowchart showing an example of operation of the image transmitter according to Embodiment 1. FIG. 4 is a flowchart showing an example of operation of the image receiver according to Embodiment 1. FIG. 5 is a block diagram showing an image transmitter according to Modification 1 of Embodiment 1. FIG. 6 is a flowchart showing an example of operation of the image transmitter according to Modification 1 of Embodiment 1. FIG. 7A is a block diagram showing an image transmitter according to Modification 2 of Embodiment 1. FIG. 7B is a flowchart showing an example of operation of the image transmitter according to Modification 2 of Embodiment 1. FIG. 8A is a block diagram showing an image transmitter according to Modification 2 of Embodiment 1. FIG. 8B is a flowchart showing an example of operation of the image transmitter according to Modification 2 of Embodiment 1. FIG. 9 is a block diagram showing an image transmitter according to Embodiment 2. FIG. 10 is an explanatory diagram showing a case where an image is divided into a plurality of segment regions and then combined, and a case where an image is divided into a plurality of segment regions so that they overlap and then combined. FIG. 11A is a flowchart showing an example of operation of the image transmitter according to Embodiment 2. FIG. 11B is a flowchart showing an example of operation of the image receiver according to Embodiment 2. Fig. 12 is a block diagram showing an image transmitter according to Modification 1 of Embodiment 2. Fig. 13 is a flowchart showing an example of operation of an image receiver according to Modification 1 of Embodiment 2. Fig. 14 is a block diagram showing an image transmitter according to Modification 2 of Embodiment 2. Fig. 15 is a flowchart showing an example of operation of an image receiver according to Modification 2 of Embodiment 2. Fig. 16 is a block diagram showing an image transmitter according to Modification 3 of Embodiment 2. Fig. 17 is a flowchart showing an example of operation of an image transmitter according to Modification 3 of Embodiment 2.

[0017] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. Note that each of the embodiments described below represents a specific example of the present disclosure. Therefore, the numerical values, shapes, materials, components, component arrangements and connection forms, steps, step order, etc. shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Therefore, among the components in the following embodiments, components not recited in independent claims will be described as optional components.

[0018] Furthermore, each figure is a schematic diagram and is not necessarily an exact illustration. Therefore, for example, the scales and the like do not necessarily match in each figure. Furthermore, in each figure, substantially the same configurations are assigned the same reference numerals, and duplicate explanations are omitted or simplified.

[0019] (First embodiment) <Configuration and Functions> First, the configuration and functions of an image digital communication system 1 will be described with reference to Fig. 1. Images include moving images and still images.

[0020] FIG. 1 is a block diagram showing an image transmitter 10 according to the first embodiment.

[0021] The digital communication system 1 comprises an image transmitter 10 and an image receiver 40 .

[0022] The image transmitter 10 includes an image acquisition unit 11, a compression rate adjustment unit 12, an adjustment encoding unit 13, a noise application unit 14, an adjustment decoding unit 15, a calculation unit 30, a judgment unit 31, a compression rate selection unit 32, a transmission encoding unit 33, a compression rate selection result embedding unit 34, and a signal transmission unit 35.

[0023] The image acquisition unit 11 is configured to acquire an original image (hereinafter, may be simply referred to as an image). The image acquisition unit 11 outputs the acquired image to the compression rate adjustment unit 12.

[0024] The compression rate adjustment unit 12 manages a plurality of compression rates so that a desired image quality can be obtained for the acquired image. Specifically, the compression rate adjustment unit 12 adjusts the compression rate for the acquired image so that a desired image quality can be obtained in the image receiver 40. More specifically, the compression rate adjustment unit 12 acquires the result of calculating the image quality of the image from the calculation unit 30, and can adjust the compression rate in a brute-force manner by changing the compression rate taking into account the result of calculating the image quality. Each time the compression rate adjustment unit 12 adjusts the compression rate, it outputs the repeatedly adjusted compression rate to the adjustment encoding unit 13. The compression rate adjustment unit 12 adjusts the compression rate within a preset range.

[0025] The adjustment encoding unit 13 is an encoder configured to encode the image acquired by the image acquisition unit 11 using a machine learning model. Specifically, the adjustment encoding unit 13 uses the machine learning model to compress the image at the compression rate adjusted by the compression rate adjustment unit 12, and encodes the compressed image. The machine learning model is, for example, Deep JSCC (source-channel joint coding or source-channel joint coding), a deep learning network, a support vector machine, a random forest, a linear regression model, or the like. The adjustment encoding unit 13 outputs the encoded image to the noise injection unit 14. The adjustment encoding unit 13 is an example of a second encoding unit.

[0026] The noise injection unit 14 is configured to inject noise into the image coded by the adjustment coding unit 13 in order to estimate noise resistance. The noise injection unit 14 outputs the image to which noise has been injected to the adjustment decoding unit 15.

[0027] The adjustment decoding unit 15 is a decoder configured to decode the image to which noise has been added by the noise addition unit 14 using a machine learning model, that is, to decode the image while taking into account the adjusted compression rate. The adjustment decoding unit 15 outputs the decoded image to the calculation unit 30. The machine learning model is, for example, Deep JSCC, a deep learning network, a support vector machine, a random forest, a linear regression model, or the like. The adjustment decoding unit 15 is an example of a decoding unit.

[0028] In this embodiment, it is preferable that the machine learning model used in the adjustment encoding unit 13 and the adjustment decoding unit 15 is Deep JSCC.

[0029] The calculation unit 30 calculates the image quality of the image decoded by the adjustment decoding unit 15. The calculation unit 30 calculates the image quality of the decoded image using, for example, the PSNR (Peak Signal to Noise Ratio) method. The calculation unit 30 outputs the result of calculating the image quality of the decoded image to the compression rate adjustment unit 12. That is, the calculation unit 30 feeds back to the compression rate adjustment unit 12 a result including the compression rate adjusted by the compression rate adjustment unit 12 and the image quality corresponding to the compression rate. Therefore, as described above, the compression rate adjustment unit 12 can adjust the compression rate in a brute-force manner by changing the compression rate while taking into account the result fed back from the calculation unit 30.

[0030] The calculation unit 30 also calculates the image quality of the image decoded for each changed compression rate and outputs the result to the determination unit 31 .

[0031] The determination unit 31 is configured to determine whether or not the multiple image qualities calculated by the calculation unit 30 for the image with the changed compression rate satisfy desired conditions. For example, the determination unit 31 determines, from the multiple calculated image qualities, whether or not the image quality of the image measured by the calculation unit 30 is equal to or greater than a preset threshold and satisfies the condition of the highest compression rate, using a PSNR (Peak Signal to Noise Ratio) method.

[0032] The determining unit 31 outputs the result of determining the image quality of the image to the compression rate selecting unit 32 .

[0033] The compression rate selection unit 32 is configured to select an optimal compression rate from among the multiple compression rates managed by the compression rate adjustment unit 12, based on the result of the determination by the determination unit 31. That is, the compression rate selection unit 32 selects the highest compression rate from among the multiple compression rates that result in excellent image quality equal to or higher than a preset threshold. The compression rate selection unit 32 outputs the image acquired by the image acquisition unit 11 to the transmission encoding unit 33, and outputs information indicating the selected compression rate to the compression rate selection result embedding unit 34. That is, the transmission encoding unit 33 may acquire the original image acquired by the image acquisition unit 11 via the compression rate selection unit 32 or the like, or may acquire it directly from the image acquisition unit 11.

[0034] The transmission encoding unit 33 is configured to encode the image acquired by the image acquisition unit 11 using a machine learning model. Specifically, the transmission encoding unit 33 encodes the image acquired by the image acquisition unit 11 and converts it into latent features possessed by the image. The machine learning model is, for example, Deep JSCC, a deep learning network, a support vector machine, a random forest, a linear regression model, or the like. The transmission encoding unit 33 is an example of a first encoding unit.

[0035] The determining unit 31 and the compression rate selecting unit 32 may be integrated into one device. Also, the processing may proceed from the transmission encoding unit 33 to the compression rate selection result embedding unit .

[0036] The compression rate selection result embedding unit 34 is configured to embed metadata indicating the compression rate selected by the compression rate selection unit 32 in a header or the like of an image acquired by the image acquisition unit 11 and to be coded by the transmission coding unit 33. In other words, the compression rate selection result embedding unit 34 embeds metadata indicating the compression rate selected by the compression rate selection unit 32 as a header in a latent feature of the image acquired by the image acquisition unit 11 and to be coded by the transmission coding unit 33.

[0037] The signal sending unit 35 is a transmitter configured to send a signal indicating the latent feature amount of the image in which the metadata has been embedded by the transmission encoding unit 33 and the compression rate selection result embedding unit 34 to the image receiver 40 via the actual communication path 3. Then, the signal sending unit 35 sends a signal indicating the latent feature amount of the image (sometimes referred to as a signal indicating the image) to the image receiver 40. The signal sending unit 35 is an example of a sending unit.

[0038] The image receiver 40 includes a signal receiving unit 41 , a receiving decoding unit 42 , and a compression rate selection result reading unit 43 .

[0039] The signal receiving unit 41 is a receiver configured to receive a signal indicating an image transmitted by the image transmitter 10 via the actual communication path 3. The signal receiving unit 41 outputs the image indicated by the received signal to the receiving decoding unit 42, and outputs metadata of the image to the compression rate selection result reading unit 43.

[0040] The compression rate selection result reading unit 43 is configured to acquire image metadata from the signal receiving unit 41 and read the image metadata. As described above, the metadata contains information indicating the compression rate of the image. Therefore, the compression rate selection result reading unit 43 can recognize the compression rate corresponding to the image from the metadata. The compression rate selection result reading unit 43 outputs the read image metadata to the receiving decoding unit 42.

[0041] The receiving decoding unit 42 acquires an image from the signal receiving unit 41 and acquires metadata of the image read by the compression rate selection result reading unit 43. The receiving decoding unit 42 is a decoder configured to decode the image acquired from the signal receiving unit 41 using a machine learning model. In other words, the receiving decoding unit 42 decodes the image using the machine learning model so as to correspond to the compression rate indicated in the metadata. The machine learning model is, for example, Deep JSCC, a deep learning network, a support vector machine, a random forest, a linear regression model, or the like. In this way, the image receiver 40 can acquire the decoded image.

[0042] The image receiver 40 may also have a noise estimation unit that calculates estimated noise in the decoded image. The estimated noise is noise that is estimated when a signal is transmitted through the actual communication channel 3. In this case, as indicated by the dashed line in FIG. 1 , the image receiver 40 may transmit the calculated estimated noise to the noise injection unit 14 of the image transmitter 10. The noise injection unit 14 may learn the estimated noise by receiving feedback of the estimated noise. After learning, the noise injection unit 14 may inject noise into the image coded by the adjustment coding unit 13. This allows the noise injection unit 14 to inject noise more appropriately, thereby enabling the determination unit 31 to more accurately determine a compression rate that satisfies the desired image quality.

[0043] Here, a digital communication system using Deep JSCC will be described with reference to FIG.

[0044] Fig. 2 is an explanatory diagram showing an example of Deep JSCC. Fig. 2 shows the process from encoding to decoding. Fig. 2 shows a digital communication system made up of a Deep JSCC encoding unit 100, a communication path, and a Deep JSCC decoding unit 110. In Fig. 2, the encoding unit 100 corresponds to the transmission encoding unit 33 on the image transmitter 10 side, the communication path corresponds to the actual communication path 3 described above, and the decoding unit 110 corresponds to the reception decoding unit 42 on the image receiver 40 side.

[0045] In the encoding unit 100 and the decoding unit 110, a repetitive structure consisting of a convolutional layer (Conv.), a batch normalization layer (Batch norm.), and an activation function layer (PRelu) is formed after a normalization layer. The encoding unit 100 and the decoding unit 110 are connected via a communication channel. In the decoding unit 110, a denormalization layer is formed after the repetitive structure. The decoding unit 110 uses a transposed convolutional layer to gradually increase the resolution of the compressed image and scale it to the size of the original image, that is, reconstruct it to the resolution of the original image.

[0046] The activation function mainly uses the PRelu function, and the sigmoid function is used only in the final convolutional layer of the decoding unit 110.

[0047] For example, if the "5x5x16|2" shown in the first repetitive structure is changed to "FxFxK|S," F represents the spatial extent or size of the filter, S represents the stride, and K represents the number of filters in the convolutional layer. Therefore, in the case of "5x5x16|2," "5x5" represents the spatial extent or size, "16" represents the convolutional layer of the filter, and "2" represents the stride. The same applies to other repetitive structures.

[0048] The number of filters c in the final convolutional layer of the encoding unit 100 is set to be variable. Taking into consideration the process of dividing the original image into IQ symbols represented by complex numbers and how the original image is compressed depending on the stride of the convolutional layer, the number of IQ symbols Nps per original image is calculated as shown in the following (Equation 1).

[0049]

[0050] Mpr and Mpc are the vertical and horizontal sizes of the original image. c is the number of filters. si is the stride of the ith convolutional layer.

[0051] When applying a convolutional layer to an original image, the number of channels of the feature map detected by the convolutional layer can be adjusted by changing the number of filters c in the convolutional layer. In other words, the compression rate adjustment unit 12 can adjust the compression rate of the original image by controlling the number of filters c, so the number of filters c can also be said to be a value corresponding to the adjustment of the compression rate.

[0052] In this embodiment, the number of filters c is arranged in the final convolutional layer of the encoding unit 100, but this is not limitative. The convolutional layer in which the number of filters c is arranged may be arranged anywhere in the repetitive structure of the encoding unit 100.

[0053] <Operation Example> Next, an operation example of the image transmitter 10 in the digital communication system 1 will be described with reference to FIG.

[0054] FIG. 3 is a flowchart showing an example of the operation of the image transmitter 10 according to the first embodiment.

[0055] First, the flowchart of the image transmitter 10 will be described.

[0056] First, the image acquisition unit 11 acquires an original image (S11).

[0057] Next, the compression rate adjustment unit 12 adjusts the compression rate for the image acquired by the image acquisition unit 11 so as to obtain a desired image quality in the image receiver 40 (S12). Specifically, the compression rate adjustment unit 12 acquires the result of calculating the image quality of the image from the calculation unit 30, and adjusts the compression rate in a round-robin manner while changing the compression rate taking into account the result of calculating the image quality, and outputs the result to the adjustment coding unit 13 every time the compression rate is adjusted.

[0058] Next, the image transmitter 10 processes the image acquired by the image acquisition unit 11 (S13).

[0059] Specifically, the adjustment encoding unit 13 uses a machine learning model to adjust the compression rate and encode the image acquired by the image acquisition unit 11. The noise application unit 14 applies noise to the image encoded by the adjustment encoding unit 13. The adjustment decoding unit 15 uses a machine learning model to decode the image to which noise has been applied by the noise application unit 14 so as to correspond to the compression rate.

[0060] Next, the calculation unit 30 calculates the image quality of the image decoded by the adjustment decoding unit 15, and feeds back the result, including the compression ratio adjusted by the compression ratio adjustment unit 12 and the image quality relative to the compression ratio, to the compression ratio adjustment unit 12 (S14).

[0061] In this manner, steps S12 to S14 are repeated.

[0062] Next, the determination unit 31 determines, from among the calculated multiple image qualities, the image quality that is equal to or greater than a preset threshold and has the highest compression rate, for example, using the PSNR (Peak Signal to Noise Ratio) method (S15).

[0063] Next, the compression rate selection unit 32 selects a compression rate based on the result output by the determination unit 31 (S16).

[0064] Next, the transmission encoding unit 33 encodes the image acquired by the image acquisition unit 11 using the machine learning model (S17).

[0065] Next, the compression rate selection result embedding unit 34 embeds metadata indicating the compression rate selected by the compression rate selection unit 32 into the header or the like of the image acquired by the image acquisition unit 11. In other words, the compression rate selection result embedding unit 34 embeds metadata indicating the compression rate selected by the compression rate selection unit 32 as a header into latent features of the image acquired by the image acquisition unit 11 and to be encoded by the transmission encoding unit 33.

[0066] Next, the signal transmitting unit 35 transmits a signal in which the metadata is embedded and which indicates the latent features of the image to the image receiver 40 via the actual communication path 3 (S19). That is, the signal transmitting unit 35 embeds metadata as a header in the latent features of the image acquired by the image acquiring unit 11 and to be coded by the transmission coding unit 33, and transmits a signal indicative of the latent features of the image in which the metadata is embedded (a signal indicative of the image) to the image receiver 40.

[0067] As a result, a signal representing an image is transmitted from the image transmitter 10 to the image receiver 40 via the actual communication path 3. Then, the flow chart of FIG. 3 ends.

[0068] Next, a flowchart of the image receiver 40 will be described with reference to FIG.

[0069] FIG. 4 is a flowchart showing an example of the operation of the image receiver 40 according to the first embodiment.

[0070] First, the signal receiving unit 41 receives a signal indicating an image transmitted by the image transmitter 10 via the actual communication path 3 (S21).

[0071] Next, the compression rate selection result reading unit 43 acquires the image metadata from the signal receiving unit 41, and reads the image compression rate indicated by the image metadata (S22).

[0072] Next, the receiving decoding unit 42 acquires the image from the signal receiving unit 41 and acquires the compression rate of the image read by the compression rate selection result reading unit 43. The receiving decoding unit 42 decodes the image acquired from the signal receiving unit 41 using a machine learning model (S23).

[0073] As a result, a signal representing an image is transmitted from the image transmitter 10 to the image receiver 40 via the actual communication path 3. Then, the flow chart of FIG. 4 ends.

[0074] Next, the following modifications are included in this embodiment.

[0075] (Modification 1) First, a digital communication system 1a including an image transmitter 10a and the like in this modification will be described with reference to FIG.

[0076] FIG. 5 is a block diagram showing an image transmitter 10a according to a first modification of the first embodiment.

[0077] This modification differs from the above-described embodiment in that the image transmitter 10a does not include the adjustment encoding unit 13, noise application unit 14, adjustment decoding unit 15, calculation unit 30, and determination unit 31, and in that a compression ratio estimation unit 12a is used instead of the compression ratio adjustment unit 12. Since the basic configuration of the digital communication system 1a including the image transmitter 10a etc. in this modification is the same as the basic configuration of the above-described embodiment, the basic configuration of the digital communication system 1a including the image transmitter 10a etc. in this modification will be assigned the same reference numerals as those in the above-described embodiment, and description thereof will be omitted where appropriate.

[0078] The image transmitter 10 a includes an image acquisition unit 11 , a compression rate estimation unit 12 a , a compression rate selection unit 32 , a transmission encoding unit 33 , a compression rate selection result embedding unit 34 , and a signal transmission unit 35 .

[0079] The image acquisition unit 11 is configured to acquire an original image (hereinafter, may be simply referred to as an image) and outputs the acquired image to the compression ratio estimation unit 12a.

[0080] The compression rate estimation unit 12a manages multiple compression rates that achieve a desired image quality for the acquired image. Specifically, the compression rate estimation unit 12a uses a machine learning model to estimate a compression rate that achieves the image quality of the acquired image so that the desired image quality is obtained in the image receiver 40. More specifically, the compression rate estimation unit 12a estimates the compression rate each time the number of filters c described above in the machine learning model is changed. To estimate multiple compression rates for one image, the compression rate estimation unit 12a outputs the estimated multiple compression rates to the compression rate selection unit 32. The machine learning model is, for example, Deep JSCC, a deep learning network, a support vector machine, a random forest, a linear regression model, or the like. Note that when configured as a deep learning network, it may be configured as a simple fully connected layer, or a convolutional layer may be used.

[0081] The compression rate selection unit 32 is configured to select an optimal compression rate from the plurality of compression rates estimated by the compression rate estimation unit 12 a. Specifically, the compression rate selection unit 32 selects the highest compression rate from the plurality of image qualities that is equal to or greater than a predetermined threshold value so as to obtain a desired image quality. The compression rate selection unit 32 outputs the image acquired by the image acquisition unit 11 to the transmission encoding unit 33 and outputs information indicating the selected compression rate to the compression rate selection result embedding unit 34.

[0082] The configurations and functions of the transmission encoding unit 33, the compression rate selection result embedding unit 34, and the signal transmitting unit 35 are the same as those in the above-described embodiment.

[0083] The image receiver 40 may also have a noise estimation unit that calculates estimated noise in the decoded image. In this case, as indicated by the dashed line in FIG. 5 , the image receiver 40 may transmit the calculated estimated noise to the compression rate estimation unit 12a of the image transmitter 10a. The compression rate estimation unit 12a may learn the estimated noise by receiving the estimated noise as feedback. After learning, the compression rate estimation unit 12a can estimate whether image quality will deteriorate, taking the estimated noise into account. This allows the compression rate estimation unit 12a to more accurately estimate a compression rate that satisfies the desired image quality.

[0084] <Example of Operation> Next, an example of operation of the digital communication system 1a will be described with reference to FIG.

[0085] Fig. 6 is a flowchart showing an example of the operation of the image transmitter 10a according to the modified example of Embodiment 1. Note that the same processes as those in Fig. 3 are denoted by the same reference numerals, and the description thereof will be omitted as appropriate.

[0086] First, the image acquisition unit 11 acquires an original image (S11).

[0087] Next, the compression rate estimation unit 12a estimates the compression rate for the acquired image so that the desired image quality can be obtained in the image receiver 40 (S12a). That is, the compression rate estimation unit 12a estimates the compression rate of the image every time the number of filters c described above in the machine learning model is changed so that the desired image quality can be obtained. In this way, by repeating step S12a, the compression rate estimation unit 12a obtains multiple estimated compression rates for one image.

[0088] Then, after the processes of steps S16 to S19, the flow chart of FIG. 6 ends.

[0089] As a result, a signal representing an image is transmitted from the image transmitter 10 a to the image receiver 40 via the actual communication path 3 .

[0090] In this modification, the operation example of the image receiver 40 is the same as the flowchart in FIG. 4, and therefore the operation example will be omitted.

[0091] (Modification 2) First, a digital communication system 1b including an image transmitter 10b and the like in this modification will be described with reference to FIG. 7A.

[0092] FIG. 7A is a block diagram showing an image transmitter 10b according to the second modification of the first embodiment.

[0093] This modification differs from the first embodiment in that the image transmitter 10b includes an image reduction unit 11a arranged between the image acquisition unit 11 and the compression rate adjustment unit 12, and that the image acquired by the image acquisition unit 11 is output to the transmission encoding unit 33. The basic configuration and functions of the digital communication system 1b including the image transmitter 10b and the like in this modification are the same as those in the first embodiment, and therefore the basic configuration and functions of the digital communication system 1b including the image transmitter 10b and the like in this modification will be denoted by the same reference numerals as those in the first embodiment, and explanations thereof will be omitted where appropriate.

[0094] When the image reducing unit 11a acquires an original image from the image acquiring unit 11, it can process the acquired image. The image reducing unit 11a can use, for example, an image compression method such as the BPG (Better Portable Graphics) format, the JPEG format, or the JPEG 2000 format, or an image compression method using deep learning (AI). The image reducing unit 11a also reduces the number of pixels (resizes). For example, if the image acquiring unit 11 acquires an image with 8K image quality (original), the image reducing unit 11a can reduce the 8K resolution image to a 4K resolution image. In such a case, it is desirable for the image reducing unit 11a to compress the data volume of the image acquired by the image acquiring unit 11 to between 70% and 90%. As a result, the image reducing unit 11a performs compression processing on the original image based on the image format and reduces the number of pixels (resizes) on the original image, thereby reducing the processing load on the image transmitter 10b in this modified example. Specifically, this reduces the amount of calculation in the adjustment encoding unit 13 and the adjustment decoding unit 15. The image reducing unit 11a may perform both compression and resizing on the original image. The image reducing unit 11a then outputs the compressed image to the compression rate adjusting unit 12. The processes performed by the compression rate adjusting unit 12 and subsequent processes are performed on the compressed image, and the details of these processes have been described above, so a repeated description will be omitted here.

[0095] Furthermore, the image acquisition unit 11 outputs the original image to the transmission encoding unit 33. The transmission encoding unit 33 encodes the original image acquired from the image acquisition unit 11 using a machine learning model. The processing performed by the transmission encoding unit 33 and subsequent processes is as described above, and therefore will not be described again here.

[0096] <Operation Example> Next, an operation example of the image transmitter 10b will be described with reference to FIG. 7B.

[0097] 7B is a flowchart showing an example of the operation of the image transmitter 10b according to Modification 2 of Embodiment 1. In this example of operation, the same reference numerals are used for the same operations as in FIG. 3, and the description thereof will be omitted as appropriate.

[0098] First, the image acquisition unit 11 acquires an original image (S11).

[0099] Next, the image reducing unit 11a processes the original image acquired by the image acquiring unit 11 (S11a). The image reducing unit 11a outputs the compressed image to the compression rate adjusting unit 12. Then, the processing from step S12 onwards shown in Fig. 3 is performed, and therefore repeated explanations will be omitted here.

[0100] In this modification, the operation example of the image receiver 40 is the same as the flowchart in FIG. 4, and therefore the operation example will be omitted.

[0101] (Modification 3) First, a digital communication system 1c including an image transmitter 10c and the like in this modification will be described with reference to FIG. 8A.

[0102] FIG. 8A is a block diagram showing an image transmitter 10c according to a third modification of the first embodiment.

[0103] This modification differs from the above-described modification 1 of the first embodiment in that the image transmitter 10c includes an image reduction unit 11a arranged between the image acquisition unit 11 and the compression ratio estimation unit 12a, and in that the original image acquired by the image acquisition unit 11 is output to the transmission encoding unit 33. Since the basic configuration and functions of the digital communication system 1c including the image transmitter 10c and the like in this modification are the same as those in the above-described embodiment, the basic configuration and functions of the digital communication system 1c including the image transmitter 10c and the like in this modification will be assigned the same reference numerals as those in the above-described modification 1 of the first embodiment, and description thereof will be omitted where appropriate.

[0104] The image reducing unit 11a can process the acquired image when it acquires the original image from the image acquiring unit 11. The image reducing unit 11a outputs the compressed image to the compression ratio estimating unit 12a. The configuration and functions of the image reducing unit 11a are the same as those described in Modification 2 of Embodiment 1, and therefore will not be described again here.

[0105] Furthermore, the image acquisition unit 11 outputs the original image to the transmission encoding unit 33 as described above, and therefore a repeated description will be omitted here.

[0106] <Operation Example> Next, an operation example of the image transmitter 10c will be described with reference to FIG. 8B.

[0107] 8B is a flowchart showing an example of the operation of the image transmitter 10c according to Modification 3 of Embodiment 1. Note that the same processes as those in FIG. 6 are denoted by the same reference numerals, and the description thereof will be omitted as appropriate.

[0108] First, the image acquisition unit 11 acquires an original image (S11).

[0109] Next, the image reducing unit 11a processes the original image acquired by the image acquiring unit 11 (S11a). The image reducing unit 11a outputs the compressed image to the compression ratio estimating unit 12a. Then, the processing from step S12a shown in FIG. 6 is performed, and therefore, repeated explanations are omitted here.

[0110] In this modification, the operation example of the image receiver 40 is the same as the flowchart in FIG. 4, and therefore the operation example will be omitted.

[0111] (Embodiment 2) <Configuration and Function> First, the configuration and function of an image digital communication system 201 will be described with reference to Fig. 9. Images include moving images and still images.

[0112] FIG. 9 is a block diagram showing an image transmitter 210 according to the second embodiment.

[0113] The digital communication system 201 comprises an image transmitter 210 and an image receiver 240 .

[0114] The image transmitter 210 includes a first acquisition unit 211, a first encoding unit 212, a first noise application unit 213, a first decoding measurement unit 214, a second acquisition unit 221, a second encoding unit 222, a second noise application unit 223, a second decoding measurement unit 224, a comparison unit 231, an image selection unit 232, an encoding unit 233, an image selection result embedding unit 234, and a signal transmission unit 235.

[0115] The first acquisition unit 211 is configured to acquire a first divided image in which a plurality of non-overlapping divided regions (also referred to as patch images) are formed. For example, the first acquisition unit 211 may be capable of acquiring a first divided image in which a plurality of divided regions are formed by dividing an acquired original image (hereinafter, sometimes simply referred to as an image) into a plurality of divided regions. The first acquisition unit 211 may also be capable of acquiring a first divided image in which a plurality of divided regions are formed by dividing an image into a plurality of divided regions in advance. The first acquisition unit 211 outputs the acquired plurality of divided regions to the first encoding unit 212.

[0116] The first encoding unit 212 is an encoder configured to encode the first divided image acquired by the first acquisition unit 211 using a machine learning model. The machine learning model is, for example, a machine learning model such as Deep JSCC, a deep learning network, a support vector machine, a random forest, or a linear regression model. The first encoding unit 212 encodes the acquired first divided image and outputs the encoded image to the first noise application unit 213.

[0117] In order to estimate noise resistance, the first noise injection unit 213 is configured to inject noise into the first divided image coded by the first coding unit 212. The first noise injection unit 213 outputs the first divided image to which noise has been injected to the first decoding measurement unit 214.

[0118] The first decoding measurement unit 214 is a decoder configured to measure image quality by using a machine learning model to decode the first divided image to which noise has been applied by the first noise application unit 213. The first decoding measurement unit 214 outputs the calculated image quality of the first divided image to the comparison unit 231. The machine learning model is, for example, a machine learning model such as Deep JSCC, a deep learning network, a support vector machine, a random forest, or a linear regression model.

[0119] The second acquisition unit 221 is configured to acquire a second divided image in which a plurality of overlapping divided regions (also called patch images) are formed.

[0120] For example, the second acquisition unit 221 may acquire multiple divided regions by dividing the acquired image into multiple regions. Specifically, the second acquisition unit 221 may acquire multiple divided regions by dividing the acquired original image (hereinafter, sometimes simply referred to as the image) into multiple divided regions such that each divided region has adjacent divided regions and overlapping regions where the divided regions partially overlap. In other words, the second acquisition unit 221 divides the acquired image so that adjacent divided regions overlap. Therefore, the divided regions of the divided image have overlapping regions between adjacent divided regions, i.e., overlapping data.

[0121] The second acquisition unit 221 may also acquire second divided images in which an image has been divided into a plurality of overlapping divided regions in advance, and outputs the acquired divided regions to the second encoding unit 222.

[0122] The second encoding unit 222 is an encoder configured to encode the second divided image acquired by the second acquisition unit 221 using a machine learning model. The machine learning model is, for example, a machine learning model such as Deep JSCC, a deep learning network, a support vector machine, a random forest, or a linear regression model. The second encoding unit 222 encodes the acquired second divided image and outputs the encoded image to the second noise application unit 223.

[0123] In order to estimate noise resistance, the second noise injection unit 223 is configured to inject noise into the second divided image coded by the second coding unit 222. The second noise injection unit 223 outputs the second divided image to which noise has been injected to the second decoding measurement unit 224.

[0124] The second decoding measurement unit 224 is a decoder configured to measure image quality by using a machine learning model to decode the second divided image to which noise has been applied by the second noise application unit 223. The second decoding measurement unit 224 outputs the calculated image quality of the second divided image to the comparison unit 231. The machine learning model is, for example, a machine learning model such as Deep JSCC, a deep learning network, a support vector machine, a random forest, or a linear regression model.

[0125] The comparison unit 231 is configured to compare the image quality of the first image without overlap measured by the first decoding measurement unit 214 with the image quality of the second image with overlap measured by the second decoding measurement unit 224. The comparison unit 231 compares the image quality of the first image with the image quality of the second image using, for example, a PSNR (Peak Signal to Noise Ratio) method. The comparison unit 231 is an example of a processing unit.

[0126] The comparison unit 231 outputs the result of comparing the image quality of the first image with the image quality of the second image to the image selection unit 232 .

[0127] The image selection unit 232 is configured to select either the first segmented image acquired by the first acquisition unit 211 or the second segmented image acquired by the second acquisition unit 221 based on the result output by the comparison unit 231. For example, the image selection unit 232 acquires the result of the comparison unit 231 comparing the result (PSNR (dB)) of the first segmented image with the result (PSNR (dB)) of the second segmented image, and selects the image with the larger PSNR value. In other words, the image selection unit 232 selects the second segmented image if the image quality in the result output by the comparison unit 231 is improved beyond a predetermined threshold. In other words, the image selection unit 232 selects the segmented image with superior image quality. The image selection unit 232 outputs the selected segmented image to the encoding unit 233 and outputs information indicating whether the selected segmented image is an overlapped image to the image selection result embedding unit 234.

[0128] The encoding unit 233 is configured to encode the segmented image selected by the image selection unit 232 using a machine learning model. The encoding unit 233 encodes the segmented image selected by the image selection unit 232 and outputs the encoded image to the signal transmission unit 235. The machine learning model is, for example, Deep JSCC, a deep learning network, a support vector machine, a random forest, a linear regression model, or the like.

[0129] The image selection result embedding unit 234 is configured to embed metadata indicating whether or not the images are overlapped in the headers of the segmented images selected by the image selection unit 232 and to be coded by the coding unit 233. In other words, the image selection result embedding unit 234 embeds metadata indicating whether or not the images are overlapped as headers in latent features of the segmented images selected by the image selection unit 232 and to be coded by the coding unit 233. Note that the processing may proceed from the coding unit 233 to the image selection result embedding unit 234.

[0130] The signal sending unit 235 is a transmitter configured to send a signal indicating the latent feature amount of the segmented image into which the metadata has been embedded by the encoding unit 233 and the image selection result embedding unit 234 to the image receiver 240 via the actual communication path 3. Then, the signal sending unit 235 sends a signal indicating the latent feature amount of the segmented image (sometimes referred to as a signal indicated by the segmented image) to the image receiver 240. The signal sending unit 235 is an example of a sending unit.

[0131] The image receiver 240 includes a signal receiving unit 241 , a decoding unit 242 , an image selection result reading unit 243 , and an image reconstructing unit 244 .

[0132] The signal receiving unit 241 is a receiver configured to receive a signal indicating the divided image transmitted by the image transmitter 210 via the actual communication path 3. The signal receiving unit 241 outputs the divided image indicated by the received signal to the decoding unit 242, and outputs metadata of the divided image to the image selection result reading unit 243. The signal receiving unit 241 is an example of a receiving unit.

[0133] The decoding unit 242 is a decoder configured to acquire the segmented images from the signal receiving unit 241 and decode the segmented images using a machine learning model. The decoding unit 242 decodes the segmented images acquired from the signal receiving unit 241 and outputs them to the image reconstruction unit 244. The machine learning model is, for example, a machine learning model such as Deep JSCC, a deep learning network, a support vector machine, a random forest, or a linear regression model.

[0134] The image selection result reading unit 243 is configured to acquire metadata embedded in the segmented images from the signal receiving unit 241 and read the metadata embedded in the segmented images. As described above, the metadata indicates whether or not the multiple segmented areas indicated by the segmented images are overlapping areas, that is, whether or not the segmented images are overlapping images. Therefore, the image selection result reading unit 243 can recognize whether or not the segmented images are overlapping images from the metadata. The image selection result reading unit 243 outputs the results of the metadata of the read segmented images to the image reconstruction unit 244.

[0135] The image reconstructing unit 244 is configured to reconstruct the segmented images based on the segmented images decoded by the decoding unit 242 and the metadata of the segmented images read by the image selection result reading unit 243. In other words, the image reconstructing unit 244 can reconstruct the segmented images using either the non-overlapping first segmented images or the overlapping second segmented images, whichever has better image quality. In this way, the image receiver 240 can obtain the reconstructed image.

[0136] The image receiver 240 may include a noise estimation unit that calculates estimated noise in an image reconstructed from the segmented images. In this case, as indicated by the dashed lines in FIG. 9 , the image receiver 240 may transmit the calculated estimated noise to the first noise application unit 213 and the second noise application unit 223 of the image transmitter 210. The first noise application unit 213 and the second noise application unit 223 may learn the estimated noise by receiving the estimated noise as feedback. After learning, the first noise application unit 213 may apply noise to the first segmented image coded by the first encoding unit 212, and the second noise application unit 223 may apply noise to the second segmented image coded by the second encoding unit 222. This enables the first noise application unit 213 and the second noise application unit 223 to apply noise more appropriately, thereby enabling the comparison unit 231 to more accurately compare the image quality of the segmented images.

[0137] In this embodiment, too, in the digital communication system 201 using Deep JSCC, when a convolutional layer is applied to an original image as described above, the number of channels of the feature map detected by the convolutional layer can be adjusted by changing the number of filters c in the convolutional layer. In other words, the compression rate of the original image can be controlled by controlling the number of filters c.

[0138] The above describes the cases where the images do not overlap and the cases where the images overlap.

[0139] Now, referring to Figure 10, we will explain in more detail how to divide an image to generate a non-overlapping first divided image and an overlapping second divided image, and how to combine the first divided images and the second divided images.

[0140] FIG. 10 is an explanatory diagram showing the case where an image is simply divided into multiple divided regions and then joined together (non-overlapping case), and the case where an image is divided into multiple divided regions so that they overlap and then joined together (overlapping case).

[0141] First, an original image is prepared as shown in Fig. 10(a). The original image is the image before division.

[0142] Next, when there is no overlap, dividing the original image generates a first divided image consisting of a plurality of divided regions, as shown in FIG. 10(b1).

[0143] Next, as shown in (c1) of Figure 10, if there is no overlap, the encoding unit 233 encodes the first divided image using a machine learning model such as Deep JSCC, and transmits the encoded first divided image to the image receiver 240.

[0144] When the non-overlapping case is selected by the image selection unit 232 of the image transmitter 210, the image receiver 240 can receive the first divided images and combine them to obtain the combined image, as shown in (d1) of Figure 10.

[0145] On the other hand, in the case of overlap, when the original image is divided, a second divided image is generated, consisting of multiple divided areas each formed so that it has an overlapping area that partially overlaps with an adjacent divided area, as shown in (b2) of Figure 10.

[0146] Next, as shown in (c2) of Figure 10, even if there is overlap, the encoding unit 233 encodes the second divided image using a machine learning model such as Deep JSCC, and transmits the encoded second divided image to the image receiver 240.

[0147] When the overlapping case is selected by the image selection unit 232 of the image transmitter 210, the image receiver 240 removes or averages the overlapping areas of the second divided images, as shown in (d2) of Figure 10, and then combines them to obtain a combined image with reduced block noise.

[0148] Without overlap, noise known as block noise may occur at the boundaries between blocks when the divided images are combined. However, since there is no need to transmit overlapping pixels, the number of pixels that must be transmitted increases, which reduces transmission efficiency. Transmission efficiency refers to overall performance, including the reproducibility of images when decoded by the image receiver 240, transmission reliability, optimization of resources such as data processing time in the image transmitter 210, and power consumption of the image transmitter 210 and image receiver 240. For example, if overlapping reduces the image quality of the received image and prevents accurate reproduction of the image, the efficiency can be evaluated as low. On the other hand, if overlapping still allows the image receiver 240 to reproduce high-quality images, image transmission is fast, and there are few errors, the efficiency can be evaluated as high.

[0149] On the other hand, if there is overlap, the number of pixels that must be transmitted increases, reducing transmission efficiency, but block noise can be reduced by removing or averaging the overlapping areas before combining them.

[0150] <Operation Example> Next, an operation example of the image transmitter 210 in the digital communication system 201 will be described with reference to FIG. 11A.

[0151] FIG. 11A is a flowchart showing an example of the operation of the image transmitter 210 according to the second embodiment.

[0152] First, the flowchart of the image transmitter 210 will be described.

[0153] First, the image transmitter 210 processes a first divided image in which a plurality of non-overlapping divided regions are formed (S111).

[0154] Specifically, the first acquisition unit 211 acquires a first divided image formed with a plurality of non-overlapping divided regions. The first encoding unit 212 encodes the first divided image acquired by the first acquisition unit 211 using a machine learning model. The first noise application unit 213 applies noise to the first divided image encoded by the first encoding unit 212. The first decoding measurement unit 214 decodes the first divided image to which noise has been applied by the first noise application unit 213 using the machine learning model to measure image quality. The first decoding measurement unit 214 outputs the calculated image quality of the first image without overlap to the comparison unit 231.

[0155] The image transmitter 210 also processes the second divided image in which a plurality of overlapping divided regions are formed (S112).

[0156] Specifically, the second acquisition unit 221 acquires a second divided image in which a plurality of overlapping divided regions are formed. The second encoding unit 222 encodes the second divided image acquired by the second acquisition unit 221 using a machine learning model. The second noise application unit 223 applies noise to the second divided image encoded by the second encoding unit 222. The second decoding measurement unit 224 decodes the second divided image to which noise has been applied by the second noise application unit 223 using a machine learning model to measure image quality. The second decoding measurement unit 224 outputs the calculated image quality of the second image with overlap to the comparison unit 231.

[0157] The order of steps S111 and S112 may be reversed, or steps S112 may be performed simultaneously. Therefore, the flowchart in Fig. 11A is merely an example and is not limited to this embodiment.

[0158] Next, the comparison unit 231 compares the image quality of the first image without overlap measured by the first decoding measurement unit 214 with the image quality of the second image with overlap measured by the second decoding measurement unit 224 (S113).

[0159] Next, the image selection unit 232 selects either the first divided image or the second divided image based on the result output by the comparison unit 231 (S114).

[0160] Next, the encoding unit 233 encodes the segmented images selected by the image selection unit 232 using the machine learning model (S115).

[0161] Next, the image selection result embedding unit 234 embeds metadata indicating whether or not the image is an overlapped image in the header or the like of the segmented image selected by the image selecting unit 232 (S116). That is, the image selection result embedding unit 234 embeds metadata indicating whether or not the image is an overlapped image as a header in the latent feature of the segmented image selected by the image selecting unit 232 and to be coded by the coding unit 233.

[0162] Next, the signal transmitting unit 235 transmits a signal in which the metadata has been embedded and which indicates the latent feature amount possessed by the segmented image to the image receiver 240 via the actual communication path 3 (S117). That is, the signal transmitting unit 235 embeds metadata as a header in the latent feature amount possessed by the segmented image selected by the image selecting unit 232 and to be coded by the coding unit 233, and transmits a signal indicative of the latent feature amount possessed by the segmented image in which the metadata has been embedded (a signal indicated by the segmented image) to the image receiver 40.

[0163] As a result, a signal representing the divided image is transmitted from the image transmitter 210 to the image receiver 240 via the actual communication path 3. Then, the flow chart of FIG. 11A ends.

[0164] Next, a flow chart for the image receiver 240 will be described with reference to FIG. 11B.

[0165] FIG. 11B is a flowchart showing an example of the operation of the image receiver 240 according to the second embodiment.

[0166] First, the signal receiving unit 241 receives a signal indicating a divided image transmitted by the image transmitter 210 via the actual communication path 3 (S121).

[0167] Next, the decoding unit 242 acquires the segmented image from the signal receiving unit 241 and decodes the segmented image using the machine learning model (S122).

[0168] Next, the image selection result reading unit 243 acquires the metadata of the divided images from the signal receiving unit 241 and reads the metadata of the divided images (S123).

[0169] Next, the image reconstructing unit 244 reconstructs the divided images based on the divided images decoded by the decoding unit 242 and the metadata of the divided images read by the image selection result reading unit 243 (S124).

[0170] As a result, a signal representing the divided image is transmitted from the image transmitter 210 to the image receiver 240 via the actual communication path 3. Then, the flow chart of FIG. 11B ends.

[0171] Next, the following modifications are included in this embodiment.

[0172] (Modification 1) First, a digital communication system 201a including an image transmitter 210a and the like in this modification will be described with reference to FIG.

[0173] FIG. 12 is a block diagram showing an image transmitter 210a according to a first modification of the second embodiment.

[0174] This modification differs from the above-described embodiment in that the image transmitter 210a does not include the first encoding unit 212, first noise application unit 213, first decoding measurement unit 214, second encoding unit 222, second noise application unit 223, and second decoding measurement unit 224 of Fig. 9, and in that a determination unit 231a is used instead of the comparison unit 231 of Fig. 9. Since the basic configuration of the digital communication system 201a including the image transmitter 210a etc. in this modification is the same as the basic configuration of the above-described embodiment, the basic configuration of the digital communication system 201a including the image transmitter 210a etc. in this modification will be assigned the same reference numerals as those in the above-described embodiment, and description thereof will be omitted where appropriate.

[0175] The image transmitter 210 a includes a first acquisition unit 211 , a second acquisition unit 221 , a determination unit 231 a , an image selection unit 232 , an encoding unit 233 , an image selection result embedding unit 234 , and a signal transmission unit 235 .

[0176] The first acquisition unit 211 is configured to acquire a first divided image in which a plurality of non-overlapping divided regions are formed, and outputs the acquired plurality of divided regions to the determination unit 231 a.

[0177] The second acquisition unit 221 is configured to acquire a second divided image in which a plurality of overlapping divided regions are formed, and outputs the acquired plurality of divided regions to the determination unit 231 a.

[0178] The determination unit 231a uses a machine learning model to compare the efficiency of the first image segment with the efficiency of the second image segment and determine whether the efficiency of the first image segment and the efficiency of the second image segment are each within a threshold. In other words, the determination unit 231a determines whether overlapping the image segments to be transmitted reduces efficiency. Efficiency refers to compression efficiency, transmission efficiency, encoding efficiency, etc. The machine learning model may be, for example, Deep JSCC, a deep learning network, a support vector machine, a random forest, a linear regression model, or other machine learning models. Note that when using a deep learning network, it may be configured with a simple fully connected layer or may use a convolutional layer. The output result of the determination unit 231a may be an estimated value using the PSNR method, or may simply indicate whether overlapping should be performed, such as 0 / 1, using a sigmoid function or a softmax function. The determination unit 231a outputs the determination result to the image selection unit 232. The determination unit 231a is an example of a processing unit.

[0179] The configurations and functions of the image selection unit 232, the encoding unit 233, the image selection result embedding unit 234, and the signal transmission unit 235 are the same as those in the above-described embodiment.

[0180] The image receiver 240 may include a noise estimation unit that calculates estimated noise in an image reconstructed from the segmented images. The estimated noise is noise estimated when a signal is transmitted through the actual communication channel 3. In this case, as indicated by the dashed line in FIG. 12 , the image receiver 240 may transmit the calculated estimated noise to the determination unit 231a of the image transmitter 210a. The estimated noise may also be input to the machine learning model of the determination unit 231a. That is, the estimated noise, the first segmented image, and the second segmented image may be input to the determination unit 231a. The determination unit 231a can determine whether overlapping the segmented images to be transmitted reduces efficiency by taking the estimated noise into consideration. This allows the determination unit 231a to more appropriately determine whether the segmented images do not overlap or do overlap.

[0181] <Operation Example> Next, an operation example of the digital communication system 201a will be described with reference to FIG.

[0182] Fig. 13 is a flowchart showing an example of the operation of the image receiver 240 according to the first modification of the second embodiment. Note that the same processes as those in Fig. 11A are denoted by the same reference numerals, and the description thereof will be omitted as appropriate.

[0183] First, the first acquisition unit 211 acquires a first divided image in which a plurality of non-overlapping divided regions are formed (S111a).

[0184] Furthermore, the second acquisition unit 221 acquires a second divided image in which a plurality of overlapping divided regions are formed (S112a).

[0185] Next, the determination unit 231a uses the machine learning model to compare the efficiency of the first divided image acquired from the first acquisition unit 211 with the efficiency of the second divided image acquired from the second acquisition unit 221 (S113a), and determines whether the efficiency of the first divided image and the efficiency of the second divided image are each within a threshold value. The determination unit 231a outputs the determination result to the image selection unit 232.

[0186] Next, the image selection unit 232 selects either the first divided image or the second divided image based on the result output by the determination unit 231a (S114).

[0187] Then, after the processes of steps S115 to S117, the flow chart of FIG. 13 ends.

[0188] As a result, a signal representing the divided image is transmitted from the image transmitter 210 a to the image receiver 240 via the actual communication path 3 .

[0189] In this modification, the operation example of the image receiver 240 is the same as the flowchart in FIG. 11B, and therefore the operation example will be omitted.

[0190] (Modification 2) Next, a digital communication system 201b including an image transmitter 210b and the like in this modification will be described with reference to FIG.

[0191] FIG. 14 is a block diagram showing an image transmitter 210b according to the second modification of the second embodiment.

[0192] This modification differs from the above-described embodiment in that the image transmitter 210b further includes a compression rate adjustment unit 225 and an overlap ratio adjustment unit 226 in addition to the first acquisition unit 211, first encoding unit 212, first noise injection unit 213, first decoding measurement unit 214, second acquisition unit 221, second encoding unit 222, second noise injection unit 223, second decoding measurement unit 224, comparison unit 231, image selection unit 232, encoding unit 233, image selection result embedding unit 234, and signal transmission unit 235. The basic configuration of the digital communication system 201b including the image transmitter 210b and other components in this modification is similar to the basic configuration of the above-described embodiment. Therefore, the basic configuration of the digital communication system 201b including the image transmitter 210b and other components in this modification will be denoted by the same reference numerals as those in the above-described embodiment and the modifications included therein, and a description thereof will be omitted where appropriate. Furthermore, the configurations of the above-described embodiment may be applied to this modification.

[0193] The compression rate adjustment unit 225 is configured to use a machine learning model to adjust the compression rate of the overlapped second image, the image quality of which has been measured by the second decoding measurement unit 224. The compression rate adjustment unit 225 adjusts the compression rate within a preset range.

[0194] The overlap ratio adjustment unit 226 is configured to use a machine learning model to adjust the number of pixels of the overlapped second image, the image quality of which has been measured by the second decoding measurement unit 224, so as to obtain a desired image quality in the image receiver 240. The overlap ratio adjustment unit 226 adjusts the number of pixels within a preset range.

[0195] These machine learning models are, for example, Deep JSCC, deep learning networks, support vector machines, random forests, linear regression models, and the like.

[0196] The compression rate adjustment unit 225 and the overlap rate adjustment unit 226 may be separate, independent components, or may be a single component that cannot be separated without being destroyed.

[0197] The compression rate adjustment unit 225 adjusts the compression rate of the second image, and the overlap rate adjustment unit 226 adjusts the number of pixels for the second image, so that the comparison unit 231 compares the image quality of the first image without overlap measured by the first decoding measurement unit 214 with the image quality of the second image with overlap measured by the second decoding measurement unit 224, where the compression rate and number of pixels have been adjusted. The comparison unit 231 outputs the result of comparing the image quality of the first image with the image quality of the second image to the image selection unit 232.

[0198] Furthermore, the compression rate adjustment unit 225 may output to the second encoding unit 222 the result of decoding by the second decoding measurement unit 224 of the second image segments to which noise has been added so as to correspond to the compression rate. That is, the second encoding unit 222 feeds back to the second encoding unit 222 the result of decoding the second image segments by the second decoding measurement unit 224 so as to correspond to the adjusted compression rate. This allows the second encoding unit 222 to compress and encode the second image segments while taking into account the result acquired from the compression rate adjustment unit 225 each time the compression rate adjustment unit 225 adjusts the compression rate. In other words, the second encoding unit 222 can encode the second image segments in a brute-force manner while taking into account the result acquired from the compression rate adjustment unit 225 each time the compression rate adjustment unit 225 adjusts the compression rate. The result includes the adjusted compression rate. The machine learning model of the second encoding unit 222 can acquire the result from the compression rate adjustment unit 225 and learn to encode the second image segments.

[0199] Furthermore, the overlap ratio adjustment unit 226 may output the result of decoding the noise-added second divided image by the second decoding measurement unit 224 so as to correspond to the number of pixels to the second acquisition unit 221. In other words, the second acquisition unit 221 feeds back the result of decoding the second divided image by the second decoding measurement unit 224 so as to correspond to the adjusted number of pixels. In this way, each time the second acquisition unit 221 acquires the result from the overlap ratio adjustment unit 226, the second acquisition unit 221 divides the input image taking into account the result acquired from the overlap ratio adjustment unit 226, thereby obtaining various second divided images. The result includes the adjusted number of pixels.

[0200] The image receiver 240 may include a noise estimation unit that calculates estimated noise in an image reconstructed from the segmented images. In this case, as indicated by the dashed lines in FIG. 14 , the image receiver 240 may transmit the calculated estimated noise to the first noise application unit 213 and the second noise application unit 223 of the image transmitter 210b. The first noise application unit 213 and the second noise application unit 223 may learn the estimated noise by receiving the estimated noise as feedback. After learning, the first noise application unit 213 may apply noise to the first segmented image coded by the first encoding unit 212, and the second noise application unit 223 may apply noise to the second segmented image coded by the second encoding unit 222. This enables the first noise application unit 213 and the second noise application unit 223 to apply noise more appropriately, thereby enabling the comparison unit 231 to more accurately compare the image quality of the segmented images.

[0201] <Operation Example> Next, an operation example of the digital communication system 201b will be described with reference to FIG.

[0202] 15 is a flowchart showing an example of the operation of the image receiver 240 according to the second modification of the second embodiment. Note that the same processes as those in FIG. 11A are denoted by the same reference numerals, and the description thereof will be omitted as appropriate.

[0203] First, the image transmitter 210b processes a first divided image in which a plurality of non-overlapping divided regions are formed (S111).

[0204] The image transmitter 210b also processes the second divided image in which a plurality of overlapping divided regions are formed (S112).

[0205] Specifically, the second acquisition unit 221 acquires a second divided image in which a plurality of overlapping divided regions are formed. The second encoding unit 222 encodes the second divided image acquired by the second acquisition unit 221 using a machine learning model. The second noise application unit 223 applies noise to the second divided image encoded by the second encoding unit 222. The second decoding measurement unit 224 decodes the second divided image to which noise has been applied by the second noise application unit 223 using a machine learning model to measure image quality. The second decoding measurement unit 224 outputs the measured image quality of the second image with overlap to the comparison unit 231.

[0206] Furthermore, since the second acquisition unit 221 receives feedback of the result of decoding by the second decoding measurement unit 224 of the second divided image to which noise has been added so as to correspond to the number of pixels, each time the second acquisition unit 221 acquires the number of pixels from the overlap ratio adjustment unit 226, it divides the input image into a plurality of divided areas and acquires the image by taking into account the number of pixels acquired from the overlap ratio adjustment unit 226. Since the second acquisition unit 221 can acquire second divided images with adjusted numbers of pixels multiple times for one image each time the number of pixels is adjusted, it can output a plurality of second divided images in a brute-force manner.

[0207] In addition, the second encoding unit 222 receives feedback of the results of decoding by the second decoding measurement unit 224 of the second divided image to which noise has been added so that the result corresponds to the compression rate, so that each time the second encoding unit 222 receives the result from the compression rate adjustment unit 225, it can take into account the result obtained from the compression rate adjustment unit 225 and compress and encode the second divided image in a brute force manner.

[0208] Next, the compression rate adjustment unit 225 uses the machine learning model to adjust the compression rate of the overlapped second image, the image quality of which has been measured by the second decoding measurement unit 224 (S112b1). As described above, the compression rate adjustment unit 225 feeds back to the second encoding unit 222 the result of compressing the second divided image at the adjusted compression rate.

[0209] The overlap ratio adjustment unit 226 also uses the machine learning model to adjust the number of pixels in the overlapped second image, the image quality of which has been measured by the second decoding measurement unit 224 (S112b2). The overlap ratio adjustment unit 226 feeds back the adjusted number of pixels in the second divided image to the second acquisition unit 221.

[0210] The order of steps S112b1 and S112b2 may be reversed, or steps S112b2 may be performed simultaneously. Therefore, the flowchart in Fig. 15 is merely an example and is not limited to this embodiment.

[0211] Then, after the processes of steps S113 to S117, the flow chart of FIG. 15 ends.

[0212] As a result, a signal representing the divided image is transmitted from the image transmitter 210 b to the image receiver 240 via the actual communication path 3 .

[0213] In this modification, the operation example of the image receiver 240 is the same as the flowchart in FIG. 11B, and therefore the operation example will be omitted.

[0214] (Modification 3) Next, a digital communication system 201c including an image transmitter 210c and the like in this modification will be described with reference to FIG.

[0215] FIG. 16 is a block diagram showing an image transmitter 210c according to the third modification of the second embodiment.

[0216] 9 , a determination unit 231a is used instead of the comparison unit 231 of FIG. 9 , and a compression rate adjustment unit 225 and an overlap ratio adjustment unit 226 are further included. Since the basic configuration of a digital communication system 201c including the image transmitter 210c and the like in this modification is the same as the basic configuration of the above-described embodiment, the basic configuration of the digital communication system 201c including the image transmitter 210c and the like in this modification will be assigned the same reference numerals as those in the above-described embodiment and the modifications included therein, and a description thereof will be omitted where appropriate.

[0217] The image transmitter 210c includes a first acquisition unit 211, a second acquisition unit 221, a compression rate adjustment unit 225, an overlap ratio adjustment unit 226, a determination unit 231a, an image selection unit 232, an encoding unit 233, an image selection result embedding unit 234, and a signal transmission unit 235.

[0218] The first acquisition unit 211 is configured to acquire a first divided image in which a plurality of non-overlapping divided regions are formed, and outputs the acquired plurality of divided regions to the determination unit 231 a.

[0219] The second acquisition unit 221 is configured to acquire a second divided image in which a plurality of overlapping divided regions are formed, and outputs the acquired plurality of divided regions to the determination unit 231 a.

[0220] The compression rate adjustment unit 225 is configured to adjust the compression rate of the second divided image acquired by the second acquisition unit 221 by using a machine learning model.

[0221] The overlap ratio adjustment unit 226 is configured to adjust the number of pixels of the second divided image acquired by the second acquisition unit 221 using a machine learning model.

[0222] These machine learning models are, for example, Deep JSCC, deep learning networks, support vector machines, random forests, linear regression models, and the like.

[0223] When the compression rate adjustment unit 225 adjusts the compression rate of the second image segments and the overlap rate adjustment unit 226 adjusts the number of pixels for the second image segments, the determination unit 231a uses a machine learning model to select the second image segment with the best characteristics from among the second image segments adjusted by the compression rate adjustment unit 225 and the overlap rate adjustment unit 226. The determination unit 231a uses the machine learning model to compare the efficiency of the first image segments with the efficiency of the second image segments acquired by the second acquisition unit 221, whose compression rate and number of pixels have been adjusted, and determines whether the efficiency of the first image segments and the efficiency of the second image segments are each within a threshold value. In other words, the determination unit 231a compares which of the overlapping image segments to be transmitted has higher efficiency and determines whether the image segments satisfy the desired image quality. Efficiency refers to compression efficiency, transmission efficiency, encoding efficiency, etc. The machine learning model may be, for example, Deep JSCC, a deep learning network, a support vector machine, a random forest, a linear regression model, or the like. When configured as a deep learning network, it may be configured as a simple fully connected layer, or a convolutional layer may be used. The output result of the determination unit 231a may be an estimated value using the PSNR method, or may simply indicate whether or not overlap should be performed, such as 0 / 1, using a sigmoid function or a softmax function. The determination unit 231a outputs the determination result to the image selection unit 232.

[0224] Furthermore, the compression rate adjustment unit 225 may adjust the compression rate of the second divided image and output the adjusted compression rate to the second acquisition unit 221. The overlap rate adjustment unit 226 may adjust the number of pixels of the second divided image and output the adjusted number of pixels to the second acquisition unit 221. The second acquisition unit 221 can divide the input image into a plurality of divided regions taking into account the compression rate and the number of pixels and acquire the second divided image, and therefore can output various types of second divided images with adjusted compression rates and adjusted numbers of pixels.

[0225] The compression rate adjustment unit 225, the overlap ratio adjustment unit 226, and the determination unit 231a may be separate, independent components, or may be a single component that cannot be separated without being destroyed.

[0226] The image selection unit 232 is configured to select either the first divided image or the second divided image based on the result output by the determination unit 231 a. For example, the image selection unit 232 selects the second divided image if the result output by the determination unit 231 a shows that the image quality has improved beyond a preset threshold. When selecting the second divided image, the image selection unit 232 outputs the result output by the determination unit 231 a, i.e., the second divided image according to the number of pixels and compression rate, to the encoding unit 233, and outputs information indicating that the second divided image is an overlapped image to the image selection result embedding unit 234.

[0227] When the image selection unit 232 selects the first divided image, the image selection result embedding unit 234 embeds metadata indicating that the image is a non-overlapping image into the header of the first divided image, etc., and outputs the metadata to the signal transmission unit 235.

[0228] Also, when the image selection unit 232 selects the second divided image, the image selection result embedding unit 234 embeds information indicating that it is an overlapped image, the number of pixels, and the compression rate as metadata in the header of the second divided image, etc., and outputs it to the signal transmission unit 235.

[0229] The configuration and functions of the signal transmitting unit 235 are the same as those in the above-described embodiment.

[0230] The image receiver 240 may also have a noise estimation unit that calculates estimated noise in an image reconstructed from the segmented images. In this case, as indicated by the dashed line in FIG. 16 , the image receiver 240 may transmit the calculated estimated noise to the determination unit 231a of the image transmitter 210c. The estimated noise may also be input to the machine learning model of the determination unit 231a. That is, the estimated noise, the first segmented image, and the second segmented image may be input to the determination unit 231a. The determination unit 231a can take the estimated noise into consideration and determine whether overlapping the segmented images to be transmitted will reduce efficiency. This allows the determination unit 231a to more appropriately determine whether the segmented images do not overlap or do overlap.

[0231] <Operation Example> Next, an operation example of the digital communication system 201c will be described with reference to FIG.

[0232] 17 is a flowchart showing an example of the operation of the image transmitter 210c according to the third modification of the second embodiment. Note that the same processes as those in FIG. 13 are denoted by the same reference numerals, and the description thereof will be omitted as appropriate.

[0233] First, after the processes of steps S111a and S112a, the compression rate adjustment unit 225 adjusts the compression rate of the second divided image acquired by the second acquisition unit 221 using the machine learning model (S112b1). The compression rate adjustment unit 225 outputs the adjusted compression rate to the second acquisition unit 221.

[0234] Furthermore, the overlap ratio adjustment unit 226 adjusts the number of pixels in the second divided image acquired by the second acquisition unit 221 using the machine learning model (S112b2). The overlap ratio adjustment unit 226 outputs the adjusted number of pixels to the second acquisition unit 221.

[0235] Then, after the processes of steps S113a to S117, the flow chart of FIG. 17 ends.

[0236] As a result, a signal representing the divided image is transmitted from the image transmitter 210 c to the image receiver 240 via the actual communication path 3 .

[0237] In this modification, the operation example of the image receiver 240 is the same as the flowchart in FIG. 11B, and therefore the operation example will be omitted.

[0238] <Operational Effects> Next, operational effects of the mobile image transmitters 10, 10a, the digital communication systems 1, 1a, and the image transmission method and program will be described.

[0239] As described above, the image transmitter 10, 10a of technology 1 relating to this embodiment includes an image acquisition unit 11 that acquires an image, a compression rate management unit that manages multiple compression rates at which a desired image quality is obtained for the acquired image, a compression rate selection unit 32 that selects an optimal compression rate from the multiple compression rates managed by the compression rate management unit, a first encoding unit (transmission encoding unit 33) that encodes the acquired image using a machine learning model, a compression rate selection result embedding unit 34 that embeds metadata indicating the compression rate selected by the compression rate selection unit 32 into the image to be encoded by the first encoding unit, and a transmission unit (signal transmission unit 35) that transmits the image encoded by the first encoding unit (transmission encoding unit 33) with the metadata embedded by the compression rate selection result embedding unit 34.

[0240] According to this, by managing a plurality of compression rates so as to obtain a desired image quality, it becomes possible to select an optimum compression rate from among a plurality of compression rates that satisfy the desired image quality.

[0241] Therefore, with this image transmitter 10, 10a, it is possible to suppress degradation of image quality and obtain a desired image quality.

[0242] Furthermore, the image transmitter 10 of Technology 2 relating to this embodiment further includes a second encoding unit (adjustment encoding unit 13) that encodes the image acquired by the image acquisition unit 11 using a machine learning model, a noise application unit 14 that applies noise to the image encoded by the second encoding unit (adjustment encoding unit 13), a decoding unit (adjustment decoding unit 15) that decodes the image to which noise has been applied by the noise application unit 14 using a machine learning model, a calculation unit 30 that calculates the image quality of the image decoded by the decoding unit (adjustment decoding unit 15), and a determination unit 31 that determines whether the image quality calculated by the calculation unit 30 is equal to or greater than a threshold and the highest compression rate, wherein the machine learning model used by the second encoding unit (adjustment encoding unit 13) and the machine learning model used by the decoding unit (adjustment decoding unit 15) are both information source channel joint coding, and the compression rate selection unit 32 selects an optimal compression rate from among a plurality of compression rates based on the result of determination by the determination unit 31.

[0243] According to this, the calculation unit 30 calculates the image quality of the image according to each compression ratio, and the determination unit 31 can determine the highest compression ratio from among the multiple image qualities that satisfies the desired image quality. This makes it possible to select the optimal compression ratio from among the multiple compression ratios that satisfy the desired image quality. As a result, degradation of image quality can be further suppressed, and the desired image quality can be obtained.

[0244] The image transmitter 10 of Technology 3 according to this embodiment is the image transmitter 10 described in Technology 2. In this case, the calculation unit 30 outputs the result of calculating the image quality of the image to the compression rate management unit, and the compression rate management unit is a compression rate adjustment unit 12 that adjusts the compression rate based on the result of calculating the image quality of the acquired image.

[0245] According to this, since the calculation unit 30 calculates the image quality of the image each time the compression rate is adjusted, the determination unit 31 can determine the highest compression rate that satisfies the desired image quality from among the multiple image quality levels. This makes it possible to select the optimal compression rate from among the multiple compression rates that satisfy the desired image quality. As a result, degradation of image quality can be further suppressed, and the desired image quality can be obtained.

[0246] The image transmitter 10a according to Technology 4 of this embodiment is the image transmitter 10a described in Technology 1. In this case, the compression rate management unit is a compression rate estimation unit 12a that estimates a compression rate that will provide the image quality of the acquired image, and the compression rate selection unit 32 selects an optimal compression rate from among a plurality of compression rates based on the result estimated by the compression rate estimation unit 12a.

[0247] This allows estimation of multiple compression ratios that will result in a desired image quality, making it possible to select the optimal compression ratio from among multiple compression ratios that satisfy the desired image quality, thereby further suppressing degradation of image quality and achieving the desired image quality.

[0248] Furthermore, the image transmitters 10b and 10c of Technology 5 according to the present embodiment are the image transmitters 10b and 10c described in any one of Technologies 1 to 4. In this case, an image reduction unit 11a that processes the acquired image is further provided, and the compression rate management unit manages a plurality of compression rates that can obtain a desired image quality for the image processed by the image reduction unit 11a.

[0249] According to this, the image reducing section 11a processes the image in advance, and therefore it is possible to suppress an increase in the processing load when the compression rate management section manages the compression rate.

[0250] Furthermore, the digital communication system 1, 1a of Technology 6 relating to this embodiment includes the image transmitter 10, 10a described in any one of Technologies 1 to 5, and an image receiver 40, and the image receiver 40 has a receiving unit (signal receiving unit 41) that receives an image transmitted by the transmitting unit (signal transmitting unit 35) of the image transmitter 10, 10a, a decoding unit (receiving decoding unit 42) that decodes the received image using a machine learning model, and a compression rate selection result reading unit 43 that reads metadata embedded in the image.

[0251] This system also provides the same effects as those described above.

[0252] Furthermore, the image transmission method of technique 7 relating to this embodiment includes the steps of an image acquisition unit 11 acquiring an image, a compression rate management unit managing multiple compression rates that result in a desired image quality for the acquired image, a compression rate selection unit 32 selecting an optimal compression rate from the multiple compression rates managed by the compression rate management unit, a first encoding unit encoding the acquired image using a machine learning model, a compression rate selection result reading unit 43 embedding metadata indicating the compression rate selected by the compression rate selection unit 32 into the image to be encoded by the first encoding unit 33, and a transmitting unit transmitting the image encoded by the first encoding unit with the metadata embedded by the compression rate selection result reading unit 43.

[0253] This method also provides the same effects as those described above.

[0254] As described above, when transmitting high-quality images with a resolution equivalent to 4K or 8K to the image transmitters 210, 210a, 210b, and 210c of Technology 8 according to this embodiment using Deep JSCC, such as that described in Non-Patent Document 2, the parallel computation scale of the convolutional layer within Deep JSCC becomes large. For this reason, it is possible to divide the image into several blocks before transmitting. In this case, noise known as block noise occurs at the boundaries between blocks when the divided images are combined. An effective way to remove this noise is to perform Deep JSCC by overlapping blocks. However, this requires transmitting overlapping pixels, which increases the number of pixels to be transmitted and reduces transmission efficiency. Therefore, further increasing the image compression rate is considered to suppress the decrease in transmission efficiency. However, increasing the compression rate reduces image quality, resulting in a trade-off between the number of overlapping pixels and the compression rate.

[0255] An object of the present disclosure is to provide an image transmitter and the like that can suppress a decrease in transmission efficiency while suppressing a decrease in image quality.

[0256] As described above, the image transmitters 210, 210a, 210b, and 210c of technique 8 relating to this embodiment include a first acquisition unit 211 that acquires a first divided image in which a plurality of non-overlapping divided areas are formed, a second acquisition unit 221 that acquires a second divided image in which a plurality of overlapping divided areas are formed, a processing unit (comparison unit 231 or determination unit 231a) that outputs a result of comparing the first divided image and the second divided image, an image selection unit 232 that selects either the first divided image or the second divided image based on the result output by the processing unit (comparison unit 231 or determination unit 231a), an encoding unit 233 that encodes the divided image selected by the image selection unit 232 using a machine learning model, an image selection result embedding unit 234 that embeds metadata indicating whether or not there is overlap in the divided image selected by the image selection unit 232 and to be encoded by the encoding unit 233, and a transmission unit that transmits the divided image in which the image selection result embedding unit 234 embeds the metadata and which the encoding unit 233 encodes.

[0257] According to this, the comparison unit 231 or the judgment unit 231a compares the first divided image which does not overlap with the second divided image which does overlap, and the image selection unit 232 can select the divided image with better image quality based on the result output by the comparison unit 231 or the judgment unit 231a.

[0258] Furthermore, if only the overlapping second image segments were transmitted uniformly, the transmission efficiency would be reduced. However, in this embodiment, if the image quality satisfies a threshold, the image selection unit 232 can select the non-overlapping first image segments, thereby suppressing an increase in the transmission volume.

[0259] Therefore, with the image transmitters 210, 210a, 210b, and 210c, it is possible to suppress a decrease in image quality (minimize block noise) while suppressing a decrease in transmission efficiency.

[0260] Furthermore, the image transmitter 210 of Technology 9 according to this embodiment includes a first encoding unit 212 that encodes the first divided image acquired by the first acquisition unit 211 using a machine learning model, a second encoding unit 222 that encodes the second divided image acquired by the second acquisition unit 221 using a machine learning model, a first noise applying unit 213 that applies noise to the first divided image encoded by the first encoding unit 212, a second noise applying unit 223 that applies noise to the second divided image encoded by the second encoding unit 222, and a noise applying unit 224 that applies noise to the first divided image to which the first noise applying unit 213 applied noise. The image transmitter 210 is further provided with a first decoding measurement unit 214 that decodes the first divided image using a machine learning model to measure the image quality, and a second decoding measurement unit 224 that decodes the second divided image to which noise has been applied by the second noise application unit 223 using a machine learning model to measure the image quality, and the processing unit (comparison unit 231) compares the image quality of the first image without overlap measured by the first decoding measurement unit 214 with the image quality of the second image with overlap measured by the second decoding measurement unit 224.

[0261] According to this, the image quality of the first image segment can be measured in advance by passing through the first encoding unit 212, the first noise injection unit 213, and the first decoding measurement unit 214. Also, the image quality of the second image segment can be measured in advance by passing through the second encoding unit 222, the second noise injection unit 223, and the second decoding measurement unit 224. This allows the comparison unit 231 to compare the image quality of the first image with the image quality of the second image more frequently. The image selection unit 232 can select the image segment with better image quality based on the comparison result of the comparison unit 231. The image receiver 240 can receive image segments with good image quality from the image transmitter 210.

[0262] Moreover, the image transmitter 210b of Technology 10 according to this embodiment is the image transmitter 210b according to Technology 9, which further includes a compression rate adjustment unit 225 that adjusts the compression rate of the second image, the image quality of which has been measured by the second decoding measurement unit 224, using a machine learning model, and an overlap ratio adjustment unit 226 that adjusts the number of pixels in the second image, the image quality of which has been measured by the second decoding measurement unit 224, using a machine learning model, and the processing unit (comparison unit 231) compares the image quality of the first image measured by the first decoding measurement unit 214 with the image quality of the second image, the second image measured by the second decoding measurement unit 224, the compression rate and the number of pixels of which have been adjusted.

[0263] According to this, the image quality of the first image segment can be measured in advance by passing through the first encoding unit 212, the first noise injection unit 213, and the first decoding measurement unit 214. Furthermore, the image quality of the second image segment can be measured in advance by passing through the second encoding unit 222, the second noise injection unit 223, the second decoding measurement unit 224, the compression rate adjustment unit 225, and the overlap ratio adjustment unit 226. This allows the comparison unit 231 to compare the image quality of the first image with the image quality of the second image more frequently. The image selection unit 232 can select the image segment with better image quality. The image receiver 240 can receive image segments with better image quality from the image transmitter 210b.

[0264] Furthermore, the image transmitter 210b according to Technology 11 of this embodiment is the image transmitter 210b described in Technology 10. In this case, the compression rate adjustment unit 225 outputs the result of decoding the second divided image to which noise has been added by the second decoding measurement unit 224 so as to correspond to the compression rate to the second encoding unit 222, the overlap ratio adjustment unit 226 outputs the result of decoding the second divided image to which noise has been added by the second decoding measurement unit 224 so as to correspond to the number of pixels to the second acquisition unit 221, the second acquisition unit 221 divides the input image taking into account the result acquired from the overlap ratio adjustment unit 226 to acquire the second divided image formed with a plurality of divided regions, the second encoding unit 222 encodes the image taking into account the result acquired from the compression rate adjustment unit 225, and the division of the image performed by the second acquisition unit 221 is formed so that each of the plurality of divided regions has an overlap region in which it partially overlaps with an adjacent divided region.

[0265] According to this, each time the compression rate of the compression rate adjustment unit 225 and the number of pixels of the overlap ratio adjustment unit 226 are adjusted, the comparison unit 231 can obtain the adjusted second image. Each time the compression rate and the number of pixels are adjusted, the comparison unit 231 can compare the first image with the second image. In other words, the comparison unit 231 can compare multiple second images with the first image in a round-robin manner. This allows the comparison unit 231 to compare the image quality of the first image with the image quality of the second image more frequently. The image selection unit 232 can select the divided image with the better image quality.

[0266] The image transmitter 210a according to Technology 12 of the present embodiment is the image transmitter 210a described in Technology 8. In this case, the processing unit (determination unit 231a) uses a machine learning model to compare the efficiency of the first divided image with the efficiency of the second divided image, and determines whether the efficiency of the first divided image and the efficiency of the second divided image are each within a threshold value.

[0267] This allows the determination unit 231a to compare the efficiency of the first divided image with the efficiency of the second divided image, and the image selection unit 232 to select the divided image with the better efficiency, thereby preventing a decrease in efficiency.

[0268] Moreover, the image transmitter 210c of Technology 13 according to this embodiment is the image transmitter 210c described in Technology 12. In this case, the image transmitter 210c further includes a compression rate adjustment unit 225 that adjusts the compression rate of the second divided image acquired by the second acquisition unit 221 using a machine learning model, and an overlap ratio adjustment unit 226 that adjusts the number of pixels of the second divided image acquired by the second acquisition unit 221 using the machine learning model, and the processing unit (determination unit 231a) compares the first divided image acquired by the first acquisition unit 211 with the second divided image acquired by the second acquisition unit 221, the second divided image having the adjusted compression rate and number of pixels.

[0269] This allows the determination unit 231a to compare the efficiency of the first divided image with the efficiency of the second divided image more frequently, and the image selection unit 232 to select the divided image with the better efficiency.

[0270] Furthermore, the image transmitter 210c of Technology 14 according to this embodiment is the image transmitter 210c described in Technology 13. In this case, the compression rate adjustment unit 225 outputs the adjusted compression rate of the second divided image to the second acquisition unit 221, the overlap ratio adjustment unit 226 outputs the adjusted number of pixels of the second divided image to the second acquisition unit 221, the second acquisition unit 221 divides the input image into a plurality of divided areas taking into account the compression rate and the number of pixels, and the division of the image performed by the second acquisition unit 221 is formed so that each of the plurality of divided areas has an overlap area that partially overlaps with an adjacent divided area.

[0271] According to this, each time the compression rate of the compression rate adjustment unit 225 and the number of pixels of the overlap ratio adjustment unit 226 are adjusted, the determination unit 231a can acquire the adjusted second divided image. Therefore, each time the compression rate and the number of pixels are adjusted, the determination unit 231a can compare the efficiency of the first divided image with the efficiency of the second divided image. In other words, the determination unit 231a can compare multiple second divided images with the first divided image in a brute-force manner. Therefore, the determination unit 231a can compare the efficiency of the first divided image with the efficiency of the second divided image more frequently. The image selection unit 232 can select the divided image with the better efficiency.

[0272] In addition, digital communication systems 201, 201a, 201b, and 201c of technology 15 relating to this embodiment include image transmitters 210, 210a, 210b, and 210c described in any one of technologies 8 to 14, and an image receiver 240, and the image receiver 240 has a receiving unit that receives the divided images transmitted by the transmitting units of the image transmitters 210, 210a, 210b, and 210c, a decoding unit 242 that decodes the received divided images using a machine learning model, an image selection result reading unit 243 that reads metadata embedded in the divided images, and an image reconstruction unit 244 that reconstructs the divided images based on the divided images decoded by the decoding unit 242 and the metadata of the divided images read by the image selection result reading unit 243.

[0273] This system also provides the same effects as those described above.

[0274] In addition, the image transmission method of technology 16 relating to this embodiment includes the steps of a first acquisition unit 211 acquiring a first divided image in which a plurality of non-overlapping divided areas are formed, a second acquisition unit 221 acquiring a second divided image in which a plurality of overlapping divided areas are formed, a processing unit outputting a result of comparing the first divided image and the second divided image, an image selection unit 232 selecting either the first divided image or the second divided image based on the result output by the processing unit, an encoding unit 233 encoding the divided image selected by the image selection unit 232 using a machine learning model, an image selection result embedding unit 234 embedding metadata indicating whether or not there is overlap in the divided image selected by the image selection unit 232 and to be encoded by the encoding unit 233, and a transmission unit transmitting the divided image in which the image selection result embedding unit 234 embeds the metadata and which has been encoded by the encoding unit 233.

[0275] This method also provides the same effects as those described above.

[0276] The program of Technique 17 according to this embodiment is a program for causing a computer to execute the image transmission method of Technique 7 or 16.

[0277] This program also provides the same effects as those described above.

[0278] (Other Modifications) While the embodiments and modifications according to the present disclosure have been described above based on the above-described embodiments and modifications, the present disclosure is not limited to these embodiments and modifications. As long as they do not deviate from the spirit of the present disclosure, various modifications that a person skilled in the art may make to the embodiments and modifications may also be included in the scope of the present disclosure.

[0279] For example, the image transmitter, digital communication system, image transmission method, and program according to the present disclosure may be realized by a computer having a program execution unit such as a CPU or processor. In this case, the program for realizing this function may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be read into and executed by a computer system.

[0280] The term "computer system" here includes hardware such as the OS and peripheral devices. Furthermore, "computer-readable recording media" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, as well as storage devices such as hard disks built into computer systems.

[0281] Furthermore, "computer-readable recording medium" may also include something that dynamically stores a program for a short period of time, such as a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line, or something that stores a program for a certain period of time, such as volatile memory inside a computer system that serves as a server or client in that case.

[0282] Furthermore, the program may be one that realizes part of the above-mentioned functions, or may be one that can realize the above-mentioned functions in combination with a program already recorded in a computer system, or may be one that is realized using a microprocessor such as a DSP (Digital Signal Processor) or a programmable logic device such as an FPGA (Field Programmable Gate Array).

[0283] Furthermore, all of the numbers used above are examples for specifically explaining the present disclosure, and the embodiments of the present disclosure are not limited to the numbers shown as examples.

[0284] The division of functional blocks in the block diagram is an example, and multiple functional blocks may be realized as a single functional block, one functional block may be divided into multiple blocks, or some functions may be moved to another functional block.Furthermore, the functions of multiple functional blocks having similar functions may be processed in parallel or in time-sharing by a single piece of hardware or software.

[0285] The order in which the steps in the flowchart are executed is merely an example for specifically explaining the present disclosure, and other orders may be used. Also, some of the steps may be executed simultaneously (in parallel) with other steps.

[0286] In addition, this disclosure also includes forms obtained by making various modifications to the above-mentioned embodiments and modifications that a person skilled in the art would think of, and forms realized by arbitrarily combining the components and functions of the embodiments and modifications within the scope that does not deviate from the intent of this disclosure.

[0287] The present disclosure is applicable to, for example, an image transmitter using a communication path, a digital communication system, an image transmission method, and a program.

[0288] 1, 1a, 201, 201a, 201b, 201c Digital communication system 10, 10a, 210, 210a, 210b, 210c Image transmitter 11 Image acquisition unit 11a Image reduction unit 12 Compression rate adjustment unit (compression rate management unit) 12a Compression rate estimation unit (compression rate management unit) 13 Adjustment encoding unit (second encoding unit) 14 Noise application unit 15 Adjustment decoding unit (decoding unit) 30 Calculation unit 31 Determination unit 32 Compression rate selection unit 33 Transmission encoding unit (first encoding unit) 34 Compression rate selection result embedding unit 35 Signal transmission unit (transmission unit) 40 Image receiver 41 Signal reception unit (reception unit) 42 Reception decoding unit (decoding unit) 43 Compression rate selection result reading unit

Claims

1. An image transmitter comprising: an image acquisition unit that acquires an image; a compression rate management unit that manages multiple compression rates that result in a desired image quality for the acquired image; a compression rate selection unit that selects the optimal compression rate from the multiple compression rates managed by the compression rate management unit; a first encoding unit that encodes the acquired image using a machine learning model; a compression rate selection result embedding unit that embeds metadata indicating the compression rate selected by the compression rate selection unit into the image to be encoded by the first encoding unit; and a transmission unit that transmits the image encoded by the first encoding unit after the compression rate selection result embedding unit has embedded the metadata.

2. The image transmitter of claim 1, further comprising: a second encoding unit that encodes the image acquired by the image acquisition unit using a machine learning model; a noise application unit that applies noise to the image encoded by the second encoding unit; a decoding unit that decodes the image to which noise has been applied by the noise application unit using a machine learning model; a calculation unit that calculates the image quality of the image decoded by the decoding unit; and a determination unit that determines whether the image quality calculated by the calculation unit is equal to or higher than a threshold and has the highest compression rate, wherein the machine learning model used by the second encoding unit and the machine learning model used by the decoding unit are both information source channel joint coding, and the compression rate selection unit selects an optimal compression rate from the plurality of compression rates based on the result of determination by the determination unit.

3. The image transmitter according to claim 2, wherein the calculation unit outputs the result of calculating the image quality of the image to the compression rate management unit, and the compression rate management unit is a compression rate adjustment unit that adjusts the compression rate based on the result of calculating the image quality of the acquired image.

4. An image transmitter as described in claim 1, wherein the compression rate management unit is a compression rate estimation unit that estimates a compression rate that will result in the image quality of the acquired image, and the compression rate selection unit selects an optimal compression rate from among the multiple compression rates based on the result estimated by the compression rate estimation unit.

5. An image transmitter according to any one of claims 1 to 4, further comprising an image reduction unit that processes the acquired image, wherein the compression rate management unit manages a plurality of compression rates that will provide a desired image quality for the image processed by the image reduction unit.

6. A digital communication system comprising: an image transmitter according to any one of claims 1 to 4; and an image receiver, wherein the image receiver has: a receiving unit that receives the image transmitted by the transmitting unit of the image transmitter; a decoding unit that decodes the received image using a machine learning model; and a compression rate selection result reading unit that reads the metadata embedded in the image.

7. An image transmission method comprising: an image acquisition unit acquiring an image; a compression rate management unit managing a plurality of compression rates that will provide a desired image quality for the acquired image; a compression rate selection unit selecting an optimal compression rate from the plurality of compression rates managed by the compression rate management unit; a first encoding unit encoding the acquired image using a machine learning model; a compression rate selection result embedding unit embedding metadata indicating the compression rate selected by the compression rate selection unit into the image to be encoded by the first encoding unit; and a transmitting unit transmitting the image encoded by the first encoding unit, with the metadata embedded by the compression rate selection result embedding unit.

8. A program for causing a computer to execute the image transmission method according to claim 7.

Images