Training device, communication system, and program

Abstract

A training device according to the present invention includes a first control unit that performs training of a mathematical model for executing: a compressed image estimation process that estimates a simulated compressed image, which is an image obtained by capturing an image of a subject that appears in a processing target image that is a hyperspectral image to be processed and is an image to be reconstructed in compressed sensing, on the basis of the processing target image; image encoding, which is encoding of the simulated compressed image and is encoding for obtaining a differentiable image that is an image that is differentiable; a decoding process for decoding the differentiable image; and an image reconstruction process for estimating the processing target image on the basis of the result of the decoding. During the learning, the content of the compressed image estimation process, the content of the image encoding process, the content of the decoding process, and the content of the image reconstruction process are updated so as to reduce differences between the processing target image and a result of the image reconstruction process, and reduce the size of image data of the differentiable image.

Description

Learning device, communication system and program

[0001] The present invention relates to a learning device, a communication system, and a program.

[0002] In recent years, compressed spectral imaging (see Non-Patent Document 1) has been expected to be a remote sensing technology for capturing images of the ground using satellites or High Altitude Platform Stations (HAPS), in which compressed images are obtained and image data of the obtained compressed images is transmitted to the ground. As described in Non-Patent Document 1, compressed spectral imaging is a technology based on the theory of compressed sensing, and compressed images are images that undergo reconstruction processing in compressed sensing. Therefore, in compressed spectral imaging, compressed images are images obtained by optically compressing spectral images through an optical system called an observation model.

[0003] Harumitsu Sogabe, "Compressed Spectral Imaging Using Wavelength-Dependent PSF Metalens," Journal of the Institute of Image Information and Television Engineers, vol. 76, no. 2, pp. 234-239, Mar. 2022.

[0004] However, due to the limited computational resources available on artificial satellites and HAPS, the communication bandwidth between the satellites and the ground is narrow. Therefore, it takes a long time to transmit large amounts of data, such as image data of compressed images obtained by compressed spectral imaging, and there are concerns that it may take a long time to reconstruct a hyperspectral image. In other words, there are concerns that it may take a long time to obtain a hyperspectral image obtained by reconstruction processing on the ground based on the results of imaging in space. Note that this problem occurs not only in communications between space and the ground, but also in communications within outer space.

[0005] In view of the above circumstances, an object of the present invention is to provide a technology that reduces the time required to acquire a hyperspectral image obtained by reconstruction processing based on the results of imaging in space.

[0006] One aspect of the present invention is a learning device that includes a first control unit that learns a mathematical model that executes the following: a compressed image estimation process that estimates a simulated compressed image, which is an image captured of a subject appearing in a processing target image, which is a hyperspectral image to be processed, based on the processing target image, and which is an image to be reconstructed in compressed sensing; image encoding that encodes the simulated compressed image to obtain a differentiable image, which is an image that can be differentiated; a decoding process that decodes the differentiable image; and an image reconstruction process that estimates the processing target image based on the result of the decoding, wherein the learning updates the content of the compressed image estimation process, the content of the image encoding process, the content of the decoding process, and the content of the image reconstruction process so as to reduce the difference between the processing target image and the result of the image reconstruction process and the size of the image data of the differentiable image.

[0007] One aspect of the present invention includes a first control unit that performs learning of a mathematical model that executes: a compressed image estimation process that estimates a simulated compressed image, which is an image captured of a subject appearing in a processing target image that is a hyperspectral image to be processed and is an image to be reconstructed in compressed sensing, based on the processing target image; image encoding that encodes the simulated compressed image to obtain a differentiable image that is a differentiable image; a decoding process that decodes the differentiable image; and an image reconstruction process that estimates the processing target image based on the decoding result. In the learning, the first control unit performs learning of a mathematical model that executes learning of the content of the compressed image estimation process and the A communication system comprising: an imaging system including an optical system configured based on a learned compressed image estimation process obtained by a learning device, in which the content of image encoding, the content of the decoding process, and the content of the image reconstruction process are updated, and an imaging device that images a subject through the optical system; a second control unit that encodes the image obtained by the imaging system; a transmission interface that is a communication interface that transmits the result of the encoding to a destination; a receiving interface that is a communication interface that receives the result transmitted by the transmission interface; and a third control unit that decodes the result received by the receiving interface and performs the learned image reconstruction process obtained by the learning device on the result of the decoding, wherein the transmission interface exists in space.

[0008] One aspect of the present invention is a program for causing a computer to function as the learning device described above.

[0009] The present invention makes it possible to reduce the time required to acquire a hyperspectral image obtained by reconstruction processing based on the results of imaging in space.

[0010] An explanatory diagram illustrating an overview of a learning device in an embodiment. An explanatory diagram illustrating a communication system obtained using learning results of the learning device in an embodiment. A diagram showing an example of the hardware configuration of a learning device in an embodiment. A flowchart showing an example of the flow of processing performed by a learning device in an embodiment. A diagram showing an example of the hardware configuration of a transmitting side device in an embodiment. A diagram showing an example of the hardware configuration of a receiving side device in an embodiment. A flowchart showing an example of the flow of processing performed by a communication system in an embodiment.

[0011] 1 is an explanatory diagram illustrating an overview of a learning device 1 according to an embodiment. The learning device 1 includes a first control unit 11, which is a control unit including a processor 91, such as a central processing unit (CPU), a graphics processing unit (GPU), or a neural network processing unit (NPU), and a memory 92, all connected via a bus. The first control unit 11 executes a learning process to optimize compression and reconstruction used in compressed spectral imaging, which includes image encoding and decoding. Compression optimization refers to optimization of the optical system that performs compression in compressed spectral imaging.

[0012] The first control unit 11 executes a learning process. In the learning process, learning is performed on a mathematical model to be learned (hereinafter referred to as a "learning target model"). The learning target model is a mathematical model that includes a compressed image estimation process, an image encoding process, a decoding process, and an image reconstruction process.

[0013] The compressed image estimation process is a process of estimating, based on the processing target image, a simulated compressed image that is an image resulting from imaging of a subject appearing in the processing target image, which is a hyperspectral image to be processed, and that is an image on which reconstruction processing in compressed sensing is performed. Therefore, when the processing target image is input, the compressed image estimation process is a process of estimating, by simulation, a compressed image of the subject appearing in the processing target image, for example.

[0014] Note that a compressed image is an image that undergoes reconstruction processing in compressed sensing. Therefore, in compressed spectral imaging, for example, a compressed image is an image obtained by optically compressing a spectral image through an optical system called an observation model (see Non-Patent Document 1). The compressed image estimation process for obtaining a simulated compressed image based on a hyperspectral image is a linear transformation, as is well known in the theory of compressed sensing.

[0015] Specifically, the simulation is a process of simulating the optical system 41, which will be described later. Simulating the optical system 41 is, for example, a process of solving equations relating to electromagnetics, such as Maxwell's equations, using given parameter values ​​for optical elements included in the optical system 41. In the learning process, the parameter values ​​for these optical elements are updated. Therefore, in the learned compressed image estimation process, values ​​that are set for the parameters of the optical elements included in the optical system 41 when used in the communication system 100, which will be described later, are used.

[0016] In this way, the user can obtain the optical system information because the learned compressed image estimation process uses information indicating values ​​to be set for the parameters of the optical elements included in the optical system 41 when used in the communication system 100. The optical system information is information indicating values ​​to be set for the parameters of the optical elements included in the optical system 41 from the learned compressed image estimation process when used in the communication system 100. In other words, the optical system information is information indicating the values ​​of the parameters of the optical elements included in the optical system 41.

[0017] Image coding is coding of a simulated compressed image to obtain a differentiable image, which is an image that can be differentiated. Therefore, image coding is a process of differentiable coding of a simulated compressed image. Therefore, image coding is, for example, a process of performing Differential JPEG on a simulated compressed image. Specifically, image coding is performed by a neural network.

[0018] Note that while the compressed image estimation process for obtaining a simulated compressed image based on a hyperspectral image as described above is a linear transformation, the encoding process for obtaining a differentiable image is a nonlinear transformation. In other words, the image encoding is nonlinear encoding of the simulated compressed image to obtain a differentiable image, which is a differentiable image.

[0019] The decoding process is a process of decoding the differentiable image obtained by image encoding. That is, it is a process of decoding the differentiable image obtained by image encoding. Specifically, the signal processing is performed by a neural network.

[0020] The image reconstruction process is a process of estimating a processing target image based on the result of the decoding process, and is therefore a process of reconstructing an image in compressed sensing.

[0021] In the learning of the learning model, the learning model is updated so as to reduce the difference between the processing target image and the result of the image reconstruction process and the size of the image data of the differentiable image. In the updating of the learning model, at least the details of the compressed image estimation process, the details of the image encoding process, the details of the decoding process, and the details of the image reconstruction process are updated.

[0022] Learning is performed until a predetermined condition for terminating learning (hereinafter referred to as a "learning termination condition") is satisfied. The learning termination condition may be any condition for terminating learning, such as a condition that the change in the model to be trained due to an update is smaller than a predetermined change. The learning termination condition may be, for example, a condition that the model to be trained has been updated a predetermined number of times.

[0023] Training continues until the training termination condition is met, thereby optimizing the compression and reconstruction used in compressed spectral imaging, which includes image encoding and decoding.

[0024] The size of the image data of the differentiable image is expressed by, for example, differentiable entropy, and is obtained by, for example, the first control unit 11 based on the differentiable image.

[0025] 2 is an explanatory diagram illustrating a communication system 100 obtained using the learning results of the learning device 1 in this embodiment. The communication system 100 includes a transmitting device 2, a receiving device 3, and an imaging system 4. The imaging system 4 includes an optical system 41 and an imaging device 42.

[0026] The optical system 41 is an optical system configured based on the learned compressed image estimation process obtained by the learning device 1. Here, being configured based on the learned compressed image estimation process means that the parameter values ​​of the optical elements included in the optical system 41 are configured to be the parameter values ​​indicated by the optical system information.

[0027] The optical system 41 includes, for example, a metalens as an optical element. In such a case, the metalens is a metalens created so as to have the values ​​of the parameters of the metalens indicated by the optical system information.

[0028] The imaging device 42 is a device that captures an image of a subject via the optical system 41. Therefore, the imaging device 42 is, for example, a hyperspectral camera that captures an image of a subject via the optical system 41.

[0029] A compressed image in compressed spectrum imaging is acquired using the optical system 41 and the imaging device 42. That is, the imaging system 4 acquires a compressed image in compressed spectrum imaging.

[0030] In this way, the imaging system 4 included in the communication system 100 is a system configured using optical system information, which is an example of the learning result of the learning device 1.

[0031] The imaging system 4 is located in space. The subject may be located on the ground or in space, for example.

[0032] The transmitting device 2 includes a second control unit 21, which is a control unit including a processor 93 such as a CPU, GPU, or NPU, and a memory 94, connected by a bus, and a communication interface 22 (an example of a transmission interface).

[0033] The second control unit 21 encodes the images obtained by the imaging system 4. The encoding here is not the same as the encoding of the images obtained by the learning device 1. It is a well-known lossy encoding method such as JPEG. Therefore, the encoding performed by the second control unit 21 is nonlinear encoding. Note that, for example, if the image encoding performed by the learning device 1 is Differential JPEG, the encoding performed by the second control unit 21 is JPEG.

[0034] The communication interface 22 is a communication interface that transmits the encoding result to the destination. The second control unit 21 controls the operation of the communication interface 22 to cause it to transmit the encoding result for the image obtained by the imaging system 4 to the destination.

[0035] The transmitting device 2 is located in space.

[0036] The receiving device 3 includes a third control unit 31, which is a control unit including a processor 95 such as a CPU, GPU, or NPU, and a memory 96, connected by a bus, and a communication interface 32 (an example of a receiving interface).

[0037] The communication interface 32 is a communication interface that receives the results transmitted by the communication interface 22 (i.e., the results of encoding the images obtained by the imaging system 4).

[0038] The third control unit 31 performs decoding on the result received by the communication interface 32 and performs learned image reconstruction processing obtained by the learning device 1 on the decoding result. Here, the decoding here does not mean decoding of the decoding processing obtained by the learning device 1. It means decoding corresponding to the encoding performed by the second control unit 21. In other words, for example, if the encoding performed by the second control unit 21 is encoding in JPEG, the decoding performed by the third control unit 31 is decoding in JPEG.

[0039] The third control unit 31 controls the operation of the communication interface 32 to obtain the results sent by the communication interface 22 .

[0040] The receiving device 3 may be located in space or on the ground.

[0041] 3 is a diagram showing an example of the hardware configuration of the learning device 1 according to an embodiment. The learning device 1 is equipped with a first control unit 11, which is a control unit including a processor 91 such as a CPU, GPU, or NPU, and a memory 92, which are connected via a bus, and executes a program. By executing the program, the learning device 1 functions as a device including the first control unit 11, an interface unit 12, and a storage unit 13.

[0042] More specifically, the processor 91 reads the program stored in the storage unit 13 and stores the read program in the memory 92. The processor 91 executes the program stored in the memory 92, causing the learning device 1 to function as a device including the first control unit 11, the interface unit 12, and the storage unit 13.

[0043] The first control unit 11 controls the operation of each functional unit included in the learning device 1. The first control unit 11, for example, executes a learning process. The first control unit 11, for example, acquires information stored in the memory unit 13. Specifically, the process of acquiring information stored in the memory unit 13 is reading.

[0044] The first control unit 11 may acquire information stored in the storage unit 13 and output the acquired information to a predetermined output destination via the interface unit 12 .

[0045] The interface unit 12 includes a communication interface for connecting the learning device 1 to an external device. The interface unit 12 communicates with the external device via wired or wireless communication. The external device is, for example, a device that transmits image data of the image to be processed. The interface unit 12 acquires the image data of the image to be processed by communicating with the device that transmits the image data of the image to be processed.

[0046] The external device may be, for example, a predetermined output destination of the information stored in the storage unit 13. In such a case, the interface unit 12 may be controlled by the first control unit 11 and output the information read out from the storage unit 13 by the first control unit 11 to the predetermined output destination.

[0047] The interface unit 12 may be configured to include input devices such as a mouse, keyboard, or touch panel. The interface unit 12 may be configured as an interface that connects these input devices to the study device 1. In this way, the input devices of the interface unit 12 accept input of various information to the study device 1 via wired or wireless connections. Note that information does not necessarily have to be input to the communication interface of the interface unit 12, but may also be input to the input devices of the interface unit 12.

[0048] The interface unit 12 outputs, for example, various types of information. The interface unit 12 includes a display device such as a CRT (Cathode Ray Tube) display, a liquid crystal display, or an organic EL (Electro-Luminescence) display, as well as a speaker. The interface unit 12 may be configured as an interface that connects these display devices or speakers to the learning device 1. Therefore, the interface unit 12 outputs, for example, information input to an input device of the interface unit 12 as an image or sound.

[0049] The storage unit 13 is configured using a computer-readable storage medium device (non-transitory computer-readable recording medium) such as a magnetic hard disk device or a semiconductor storage device. The storage unit 13 stores various information related to the learning device 1. The storage unit 13 stores various information generated by the operation of the first control unit 11, for example. Therefore, the storage unit 13 stores, for example, optical system information. The information that the first control unit 11 reads from the storage unit 13 and outputs to a predetermined output destination may be, for example, optical system information. The storage unit 13 may exist on a cloud, for example.

[0050] 4 is a flowchart showing an example of the flow of processing executed by the learning device 1 according to the embodiment. The first control unit 11 executes the learning processing (step S101). The learning processing is executed until a learning end condition is satisfied.

[0051] 5 is a diagram showing an example of the hardware configuration of the transmitting device 2 in an embodiment. The transmitting device 2 is equipped with a second control unit 21, which is a control unit including a processor 93 such as a CPU, GPU, or NPU, and a memory 94, which are connected via a bus, and executes a program. By executing the program, the transmitting device 2 functions as a device including the second control unit 21, an interface unit 20 including at least a communication interface 22, and a storage unit 23.

[0052] More specifically, the processor 93 reads the program stored in the storage unit 23 and stores the read program in the memory 94. The processor 93 executes the program stored in the memory 94, whereby the transmitting device 2 functions as a device including the second control unit 21, the interface unit 20, and the storage unit 23.

[0053] The second control unit 21 controls the operation of each functional unit included in the transmitting device 2. The second control unit 21, for example, performs encoding on an image obtained by the imaging system 4. The second control unit 21, for example, acquires information stored in the memory unit 23. Specifically, the process of acquiring information stored in the memory unit 23 is reading.

[0054] The interface unit 20 includes at least a communication interface 22. The communication interface 22 communicates with an external device via a wired or wireless connection. The external device is, for example, the imaging system 4. The communication interface 22 acquires image data of a compressed image obtained by the imaging system 4 in compressed spectral imaging through communication with the imaging system 4. More specifically, communication with the imaging system 4 refers to communication with the imaging device 42, and the image data of the compressed image acquired through communication with the imaging system 4 is, more specifically, image data of an image obtained by imaging with the imaging device 42.

[0055] The external device is, for example, the receiving device 3. The communication interface 22 communicates with the receiving device 3 to transmit the results of encoding the image obtained by the imaging system 4.

[0056] The interface unit 20 may be configured to include input devices such as a mouse, a keyboard, a touch panel, etc. The interface unit 20 may be configured as an interface that connects these input devices to the transmitting device 2. In this way, the input devices of the interface unit 20 accept input of various information to the transmitting device 2 via wired or wireless connections.

[0057] The interface unit 20 outputs, for example, various types of information. Therefore, the interface unit 20 includes, for example, a display device such as a CRT display, a liquid crystal display, or an organic EL display, and a speaker. The interface unit 20 may be configured as an interface that connects these display devices or speakers to the transmitting device 2. Therefore, the interface unit 20 outputs, for example, information input to an input device of the interface unit 20 as an image or sound.

[0058] The storage unit 23 is configured using a computer-readable storage medium (non-transitory computer-readable recording medium) such as a magnetic hard disk drive or a semiconductor storage device. The storage unit 23 stores various information related to the transmitting device 2. The storage unit 23 stores various information generated by the operation of the second control unit 21, for example. The storage unit 23 may exist on a cloud, for example.

[0059] 6 is a diagram showing an example of the hardware configuration of the receiving device 3 in an embodiment. The receiving device 3 is equipped with a third control unit 31, which is a control unit including a processor 95 such as a CPU, GPU, or NPU, and a memory 96, which are connected via a bus, and executes a program. By executing the program, the receiving device 3 functions as a device including the third control unit 31, an interface unit 30 including at least a communication interface 32, and a storage unit 33.

[0060] More specifically, the processor 95 reads the program stored in the storage unit 33 and stores the read program in the memory 96. The processor 95 executes the program stored in the memory 96, whereby the receiving-side device 3 functions as a device including a third control unit 31, an interface unit 30, and a storage unit 33.

[0061] The third control unit 31 controls the operation of each functional unit included in the receiving device 3. The third control unit 31, for example, decodes the result of the encoding performed by the second control unit 21. The third control unit 31, for example, performs learned image reconstruction processing on the result of the decoding. This learned image reconstruction processing is the learned image reconstruction processing obtained by the learning device 1. The third control unit 31, for example, acquires information stored in the memory unit 33. Specifically, the process of acquiring the information stored in the memory unit 33 is reading.

[0062] The interface unit 30 includes at least a communication interface 32. The communication interface 32 communicates with an external device via a wired or wireless connection. The external device is, for example, the transmitting device 2. The communication interface 32 receives the result of the encoding performed by the second control unit 21 through communication with the transmitting device 2.

[0063] The interface unit 30 may be configured to include input devices such as a mouse, a keyboard, a touch panel, etc. The interface unit 30 may be configured as an interface that connects these input devices to the receiving-side device 3. In this way, the input devices of the interface unit 30 accept input of various information to the receiving-side device 3 via wired or wireless connections.

[0064] The interface unit 30 outputs, for example, various types of information. Therefore, the interface unit 30 includes, for example, a display device such as a CRT display, a liquid crystal display, or an organic EL display, and a speaker. The interface unit 30 may be configured as an interface that connects these display devices or speakers to the receiving-side device 3. Therefore, the interface unit 30 outputs, for example, information input to an input device of the interface unit 30 as an image or sound.

[0065] The storage unit 33 is configured using a computer-readable storage medium device (non-transitory computer-readable recording medium) such as a magnetic hard disk device or a semiconductor storage device. The storage unit 33 stores various information related to the receiving-side device 3. The storage unit 33 stores various information generated by the operation of the third control unit 31, for example. The storage unit 33 may exist on a cloud, for example.

[0066] 7 is a flowchart showing an example of the flow of processing executed by the communication system 100 in this embodiment. The imaging system 4 captures an image of a subject (step S201). Next, the second control unit 21 encodes the image captured by the imaging system 4 (step S202). Next, the second control unit 21 controls the operation of the communication interface 22 to transmit the result of encoding in step S202 to the receiving-side device 3, which is the destination (step S203).

[0067] Next, the communication interface 32 receives the encoding result transmitted in step S203 (step S204). Next, the third control unit 31 decodes the result received by the communication interface 32 in step S204 (step S205). Next, the third control unit 31 executes the learned image reconstruction process for the decoding result of step S205 (step S206).

[0068] <Technical Significance of the Learning Process> The technical significance of the learning process will now be explained. When compressed spectral imaging is used only on the ground, the communication bandwidth is wide, so there is no need to consider encoding when transferring compressed images to a device that performs reconstruction. Therefore, compressed images in RGB format or a format corresponding to the Bayer array are sent and received as is.

[0069] However, satellites and High Altitude Platform Stations (HAPS) have limited computing resources. Therefore, communication bandwidth is limited when creating compressed images in space and reconstructing them on Earth, or when performing compressed spectral imaging using two different devices in space. As a result, coding becomes increasingly important. However, because coding is a nonlinear process, applying the parameters of compressed spectral imaging without encoding and decoding directly can result in reduced image reconstruction accuracy.

[0070] Therefore, the learning device 1 optimizes, through learning processing, the optimal compression and reconstruction parameters when encoding and decoding are included. In this case, simply using an encoding method that generates a non-differentiable image, such as JPEG, would prevent learning from progressing. Therefore, in the learning processing, encoding that generates a differentiable image is performed during encoding. This makes it possible to optimize compression and reconstruction when encoding is included. Learning does not progress if the image is non-differentiable because backpropagation learning cannot be performed.

[0071] Furthermore, as can be seen from the loss function described above, learning is performed not only to increase the accuracy of image estimation (i.e., reconstruction accuracy), but also to reduce the data size of the encoded image. This is because, as described above, it is desirable for the image data size to be smaller so that it can be used in communications in narrow communication bandwidth environments. If learning were performed without the constraint of reducing the image data size, it is estimated that both encoding and decoding would be processes that are very close to the identity mapping. This is because, as described above, it is the encoding and decoding that cause poor estimation accuracy.

[0072] The system learns to reduce the data size of the encoded images, which reduces the time required for narrow-bandwidth communications and therefore reduces the time required to acquire hyperspectral images reconstructed from images captured in space.

[0073] <About solving the problem> The learning device 1 in this embodiment configured as described above performs a learning process that optimizes the compression and reconstruction used in compressed spectral imaging, which includes image encoding and decoding. Therefore, as described in <About the technical significance of the learning process>, it is possible to reduce the time required to acquire a hyperspectral image obtained by a reconstruction process based on the results of imaging in space. Note that the learning may also be performed on Earth. Because the parameters of the optical system 41 are established based on the results of the learning, the learning itself does not need to be performed in space.

[0074] Furthermore, the communication system 100 in this embodiment configured as described above performs compression and reconstruction optimized by the learning process executed by the learning device 1. Therefore, compressed spectral imaging can be performed in which reconstruction can be performed with high accuracy even when encoding and decoding is performed, while reducing the time required for reconstruction due to narrow communication bandwidth. Therefore, the communication system 100 configured as described above can reduce the time required to acquire a hyperspectral image obtained by reconstruction processing based on the results of imaging in space.

[0075] (Variation) The model to be trained may perform spatial filtering, color conversion, or tone mapping. The spatial filtering, color conversion, or tone mapping may be performed, for example, before image encoding. In this case, the spatial filtering, color conversion, or tone mapping processes to be performed and the order in which they are performed if there are multiple processes to be performed are determined in advance before the training process is performed. In the training process, the contents of these processes to be performed are updated through training.

[0076] The spatial filtering, color conversion, or tone mapping may be performed, for example, before the image reconstruction process. In this case, the spatial filtering, color conversion, or tone mapping process to be performed and the order in which the processes are performed if there are multiple processes to be performed are determined in advance before the learning process is performed. In the learning process, the contents of the processes to be performed are updated through learning.

[0077] The spatial filter, color conversion, or tone mapping executed in the learning process is applied to the communication system 100. That is, the spatial filter, color conversion, or tone mapping executed before image encoding in the learning process is executed by the second control unit 21 in the order in the learning process when the learning termination condition is satisfied, before encoding by the second control unit 21. Furthermore, the spatial filter, color conversion, or tone mapping executed before image reconstruction processing in the learning process is executed by the third control unit 31 in the order in the learning process when the learning termination condition is satisfied, before execution of the trained image reconstruction processing by the third control unit 31.

[0078] Spatial filtering is a process that removes spatial noise from an image. Removing spatial noise makes encoding easier. Easier encoding means that the quality of the resulting image data can be improved while reducing its size.

[0079] Note that color conversion refers to the conversion of color space. Tone mapping is a process that determines the strength of amplitude. Amplitude here is defined as the magnitude of the signal value.

[0080] <Others> The learning device 1 may be implemented using multiple information processing devices connected to each other via a network. In this case, the functional units of the learning device 1 may be distributed and implemented across the multiple information processing devices.

[0081] The transmitting device 2 may be implemented using a plurality of information processing devices connected to each other via a network, in which case the respective functional units of the transmitting device 2 may be distributed and implemented among the plurality of information processing devices.

[0082] The receiving device 3 may be implemented using a plurality of information processing devices communicably connected via a network, in which case the respective functional units of the receiving device 3 may be distributed and implemented among the plurality of information processing devices.

[0083] Note that all or part of the functions of the learning device 1 and communication system 100 may be realized using hardware such as an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array). The program may be recorded on a computer-readable recording medium. Examples of computer-readable recording media include portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, and storage devices such as hard disks built into computer systems. The program may also be transmitted via a telecommunications line.

[0084] Although an embodiment of the present invention has been described above in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and includes designs within the scope of the gist of the present invention.

[0085] 100...communication system, 1...learning device, 2...transmitting side device, 3...receiving side device, 4...imaging system, 11...first control unit, 12...interface unit, 13...storage unit, 20...interface unit, 21...second control unit, 22...communication interface, 23...storage unit, 30...interface unit, 31...third control unit, 32...communication interface, 33...storage unit, 41...optical system, 42...imaging device, 91...processor, 92...memory, 93...processor, 94...memory, 95...processor, 96...memory

Claims

1. A learning device comprising: a first control unit that performs learning of a mathematical model that performs the following: a compressed image estimation process that estimates a simulated compressed image, which is an image obtained by imaging a subject appearing in a processing target image, which is a hyperspectral image to be processed, based on the processing target image, and which is an image to be reconstructed in compressed sensing; image encoding that encodes the simulated compressed image to obtain a differentiable image, which is an image that can be differentiated; a decoding process that decodes the differentiable image; and an image reconstruction process that estimates the processing target image based on the result of the decoding; wherein during the learning, the content of the compressed image estimation process, the content of the image encoding process, the content of the decoding process, and the content of the image reconstruction process are updated so as to reduce the difference between the processing target image and the result of the image reconstruction process and the size of the image data of the differentiable image.

2. The learning device according to claim 1, wherein the image encoding is a process of performing Differential JPEG on the simulated compressed image.

3. An imaging system comprising: a first control unit that learns a mathematical model that executes: a compressed image estimation process that estimates a simulated compressed image, which is an imaging result of a subject appearing in a processing target image that is a hyperspectral image to be processed, based on the processing target image, the simulated compressed image being an image to be reconstructed in compressed sensing; image encoding that is encoding of the simulated compressed image to obtain a differentiable image that is a differentiable image; a decoding process that decodes the differentiable image; and image reconstruction process that estimates the processing target image based on the decoding result, wherein in the learning, the content of the compressed image estimation process, the content of the image encoding, the content of the decoding, and the content of the image reconstruction process are updated so as to reduce the difference between the processing target image and the result of the image reconstruction process and the size of the image data of the differentiable image; and a second control unit that encodes the image obtained by the imaging system; a transmission interface that is a communication interface that transmits the encoding result to a destination; and a receiving interface that is a communication interface that receives the result transmitted by the transmission interface. a third control unit that performs decoding of the result received by the receiving interface and a learned image reconstruction process obtained by the learning device for the result of the decoding, wherein the transmitting interface is located in space.

4. A program for causing a computer to function as the learning device according to claim 1 or 2.

Images