Image processing device, method, and program

Abstract

An image processing device according to one embodiment of the present invention comprises: a generation unit that generates an image obtained by converting, into an image of a visible region, a multi-dimensional captured image expressed using wavelength information and recorded in a first recording device, and records the generated image in a second recording device; and a transmission unit that, when the image converted in response to an image transmission request from a device which is the source of a distribution request is recorded in the second recording device, transmits the recorded image to the device which is the source of the distribution request. When the converted image according to a condition of the transmission request is not recorded in the second recording device, the generation unit generates an image in which the captured image recorded in the first recording device is visualized in accordance with the condition of the transmission request, and the transmission unit transmits the generated image to the device which is the source of the distribution request.

Description

Image processing device, method and program

[0001] FIELD Embodiments of the present invention relate to an image processing device, method, and program.

[0002] Non-Patent Document 1 describes a method for mapping a hyperspectral image, which is a multidimensional photographic image expressed using wavelength information acquired by a hyperspectral camera, into a color space in the human visible range, thereby generating a realistic photographic image, i.e., a two-dimensional photographic image.

[0003] M. Magnusson, J. Sigurdsson, SE Armansson, MO Ulfarsson, H. Deborah and JR Sveinsson, “Creating RGB Images from Hyperspectral Images Using a Color Matching Function,” IGARSS 2020 -2020 IEEE International Geoscience and Remote Sensing Symposium, 2020, pp. 2045-2048, doi: 10.1109 / IGARSS39084.2020.9323397.https: / / ntnuopen.ntnu.no / ntnu-xmlui / bitstream / handle / 11250 / 2732828 / hsi2rgb_s3.pdf

[0004] The above-mentioned Non-Patent Document 1 describes a method for generating an RGB image by mapping a hyperspectral image, which is an image obtained by photographing with a hyperspectral camera, into a color space in the human visible range by using the color matching function of the CIE 1931 colorimetric standard observer.

[0005] The hyperspectral camera and computer are connected with an Ethernet (registered trademark) cable or a USB cable, and the connected computer, which is the computer to which the hyperspectral camera is connected, can generate RGB images using the method described in non-patent document 1, which can then be viewed by the user.

[0006] However, because the data volume of hyperspectral images is relatively large, the connected computer required for viewing them must have relatively high performance and take a relatively long processing time. In addition, the connected computer may be equipped with a relatively expensive graphics processing unit (GPU) for graphics processing and this may be used to shorten the processing speed.

[0007] Furthermore, since the connected computer does not have the function of distributing images or videos, images or videos cannot be easily viewed even from another computer that can communicate with the connected computer.

[0008] Even if the connected computer has existing image or video distribution capabilities, when transmitting a hyperspectral image from the connected computer to another computer, the amount of communication traffic increases in proportion to the data volume, and the communication takes a long time.

[0009] In addition, in order to transmit the hyperspectral image via communication and have another computer that can communicate with the connected computer process it to generate an RGB image so that the user can view it, the other computer that can communicate with the connected computer will be required to have relatively high-performance hardware.

[0010] Furthermore, even if the connected computer has the existing image and video distribution function, it cannot allow another computer to instruct the conversion method of the hyperspectral image and view it.

[0011] This invention has been made in light of the above circumstances, and its purpose is to provide an image processing device, method, and program that allows captured images to be viewed appropriately on the device that requested the distribution.

[0012] An image processing device according to one aspect of the present invention includes a generation unit that generates an image in which a multidimensional captured image represented using wavelength information and recorded in a first recording device is converted into an image in the visible range and records the image in a second recording device, and a transmission unit that transmits the recorded image to the device that originated the distribution request when the converted image in response to an image transmission request from the device that originated the distribution request is recorded in the second recording device, wherein when the converted image in response to the conditions of the transmission request is not recorded in the second recording device, the generation unit generates an image in which the captured image recorded in the first recording device is visualized in response to the conditions of the transmission request, and the transmission unit transmits the generated image to the device that originated the distribution request.

[0013] An image processing method according to one aspect of the present invention is a method performed by an image processing device, and includes the steps of: generating an image in which a multidimensional captured image represented using wavelength information and recorded in a first recording device is converted into an image in the visible range by a generation unit of the image processing device; and recording the image in a second recording device; and when the converted image in response to an image transmission request from a device that has requested distribution is recorded in the second recording device, transmitting the recorded image to the device that has requested distribution by a transmission unit of the image processing device.When the converted image in response to the conditions of the transmission request is not recorded in the second recording device, the generation unit generates an image in which the captured image recorded in the first recording device is visualized in response to the conditions of the transmission request, and the transmission unit transmits the generated image to the device that has requested distribution.

[0014] According to the present invention, a captured image can be appropriately viewed on the device that has made the distribution request.

[0015] FIG. 1 is a diagram illustrating an application example of a distribution system according to a first embodiment of the present invention. FIG. 2A is a diagram illustrating an example of a procedure of a processing operation of the distribution system according to the first embodiment of the present invention. FIG. 2B is a diagram illustrating an example of a procedure of a processing operation of the distribution system according to the first embodiment of the present invention. FIG. 3 is a diagram illustrating an application example of a distribution system according to a second embodiment of the present invention. FIG. 4A is a diagram illustrating an example of a procedure of a processing operation of the distribution system according to the second embodiment of the present invention. FIG. 4B is a diagram illustrating an example of a procedure of a processing operation of the distribution system according to the second embodiment of the present invention. FIG. 5 is a diagram illustrating an application example of a distribution system according to a third embodiment of the present invention. FIG. 6A is a diagram illustrating an example of a procedure of a first processing operation of the distribution system according to the third embodiment of the present invention. FIG. 6B is a diagram illustrating an example of a procedure of a first processing operation of the distribution system according to the third embodiment of the present invention. FIG. 7 is a diagram illustrating an example of a procedure of a second processing operation of the distribution system according to the third embodiment of the present invention. FIG. 8 is a diagram illustrating an application example of a distribution system according to a fourth embodiment of the present invention. FIG. 9A is a diagram illustrating an example of a procedure of a first processing operation of the distribution system according to the fourth embodiment of the present invention. FIG. 9B is a diagram illustrating an example of a procedure of the first processing operation of the distribution system according to the fourth embodiment of the present invention. FIG. 10 is a diagram illustrating an example of a video frame and a shooting time. FIG. 11 is a diagram showing an example of a procedure of a second processing operation of the distribution system according to the fourth embodiment of the present invention. FIG. 12 is a diagram showing an example of a procedure of a third processing operation of the distribution system according to the fourth embodiment of the present invention. FIG. 13 is a diagram showing an application example of the distribution system according to the fourth embodiment of the present invention. FIG. 14A is a diagram showing an example of a procedure of a first processing operation of the distribution system according to the fifth embodiment of the present invention. FIG. 14B is a diagram showing an example of a procedure of the first processing operation of the distribution system according to the fifth embodiment of the present invention. FIG. 15A is a diagram showing an example of a procedure of a second processing operation of the distribution system according to the fifth embodiment of the present invention. FIG. 15B is a diagram showing an example of a procedure of the second processing operation of the distribution system according to the fifth embodiment of the present invention. FIG. 16 is a diagram showing an example of a procedure of a third processing operation of the distribution system according to the fifth embodiment of the present invention.FIG. 17 is a block diagram showing an example of the hardware configuration of a hyperspectral image distribution server device according to one embodiment of the present invention.

[0016] An embodiment of the present invention will be described below with reference to the drawings. According to the distribution system of the present invention, a server device generates a two-dimensional image such as an RGB image for a hyperspectral image and distributes the image to a client device used by a user, thereby enabling the user to easily view the two-dimensional image.

[0017] Furthermore, according to a distribution system according to one embodiment of the present invention, a function used to convert a hyperspectral image into a two-dimensional image can be specified, and the two-dimensional image obtained according to this function can be easily viewed by the user.

[0018] First Embodiment Next, a first embodiment will be described. Fig. 1 is a diagram showing an application example of a distribution system according to a first embodiment of the present invention. As shown in Fig. 1, the distribution system according to the first embodiment includes a hyperspectral image distribution server device (sometimes simply referred to as a server device) 100a, a hyperspectral image distribution client device (sometimes simply referred to as a client device) 200a, and a hyperspectral camera (sometimes simply referred to as a camera) 300a.

[0019] The hyperspectral image distribution server device 100a includes a hyperspectral data recording unit 101a, a rendering unit 102a, a rendered image recording unit 103a, a session management unit 104a, a view image request receiving unit 105a, and an RGB image transmitting unit 106a.

[0020] The hyperspectral image distribution client device 200a includes a browse image designation unit 201a, a browse image request transmission unit 202a, an RGB image reception unit 203a, an RGB image drawing unit 204a, and a screen unit 205a.

[0021] Next, we will explain the flow of processing performed by the distribution system according to this embodiment, in which data captured by the hyperspectral camera 300a is visualized and displayed on the hyperspectral image distribution client device 200a. FIGS. 2A and 2B are diagrams showing an example of the procedure for processing operations of the distribution system according to the first embodiment of the present invention. (1-1) The hyperspectral camera 300a is a camera that captures images by dispersing light into wavelengths, and commercially available products based on existing technology are available. The hyperspectral camera 300a can record the intensity of light that passes through the lens and reaches the sensor for each arbitrary wavelength range. The wavelength range varies depending on the camera design.

[0022] (1-2) The hyperspectral image distribution server device 100a is a computer having a CPU (Central Processing Unit) and memory. The hyperspectral camera 300a transmits hyperspectral data (sometimes referred to as HS data) obtained by image capture to the hyperspectral data recording unit 101a of the hyperspectral image distribution server device 100a (S101). Here, the hyperspectral data will be described.

[0023] First, RGB image data is expressed as a color by mixing the brightness values ​​of R (red), G (green), and B (blue) for each pixel, which can be expressed by two-dimensional coordinates (x, y). If each brightness is expressed as a value in 256 levels, for example, each pixel will have 96 bits (= 32 bits x 3).

[0024] Hyperspectral data is expressed using the reflection intensity values ​​of each wavelength for each pixel, which can be expressed using two-dimensional coordinates (x, y). The wavelength range and wavelength resolution that a sensor can detect vary depending on the specifications of the hyperspectral camera. For example, if the detectable wavelength range is 350 to 1,000 nm and the wavelength resolution is 4 nm, the number of bands will be 164. If the reflection intensity of each band is expressed using 128 levels of values, the number of bits per pixel is 2624 bits (= 16 bits x 164). Therefore, when comparing hyperspectral data with an RGB image of the same resolution, the number of bits per pixel in hyperspectral data is approximately 27 times that of the RGB image.

[0025] Since various compression formats are popular for RGB images, when these are used, the relative ratios become larger and the volume of hyperspectral data becomes larger.

[0026] The hyperspectral data recording unit 101a receives the hyperspectral data transmitted from the hyperspectral camera 300a, generates a file of the hyperspectral data (S102), and stores the file in association with the time of shooting (S103).

[0027] The connection between the hyperspectral camera 300a and the hyperspectral image distribution server device 100a can be established using existing technology. The hyperspectral camera 300a and the hyperspectral image distribution server device 100a are connected, for example, by an Ethernet cable, and data captured by the hyperspectral camera 300a is transferred from the hyperspectral camera 300a to the server device 100a via communication in accordance with the GigE Vision standard and stored therein. The hyperspectral data recording unit 101a associates the hyperspectral data with the capture time and stores the data.

[0028] (1-3) The hyperspectral image distribution client device 200a is a computer having a CPU and memory. The hyperspectral image distribution client device 200a is connected to the hyperspectral image distribution server device 100a. The hyperspectral image distribution client device 200a and the hyperspectral image distribution server device 100a are connected, for example, via a network and communicate with each other using IP (Internet Protocol). In this embodiment, it is assumed that multiple hyperspectral image distribution client devices 200a are connected to one hyperspectral image distribution server device 100a.

[0029] Although not described in this embodiment, it is also possible to imagine a case where a plurality of hyperspectral image distribution server devices 100a are connected to one hyperspectral image distribution client device 200a.

[0030] (1-4) The hyperspectral image distribution client device 200a has a viewing image designation unit 201a. The viewing image designation unit 201a allows the operator of the hyperspectral image distribution client device 200a to designate an image to be viewed. The operator may be a person or a program. Possible methods for designating an image to be viewed include designating the most recent image or the time of capture. The viewing image designation unit 201a passes the capture time (sometimes referred to as the desired capture time or the desired HS data time) of the image the user wishes to view to the viewing image request transmission unit 202a as a viewing image request (S111). If the most recent image is designated, the capture time of the image to be viewed may be information indicating that the most recent capture time is the desired capture time.

[0031] (1-5) The browse image request sending unit 202a receives a browse image request from the browse image specifying unit 201a. The browse image request sending unit 202a sends a new browse image request that combines the received request with the client ID to the browse image request receiving unit 105a of the hyperspectral image distribution server device 100a (S112).

[0032] The client ID is used to identify the hyperspectral image distribution client device 200a. The client ID includes (1) destination information of the hyperspectral image distribution client device 200a, or (2) information that enables the session management unit 104a of the hyperspectral image distribution server device 100a to extract source information as destination information. This source information is, for example, an IP address or a fully qualified domain name (FQDN) of the hyperspectral image distribution client device 200a that is the sender of the viewing image request.

[0033] The client ID may be changed for each transmission from the same client device 200 a, or may be fixed as long as it is unique enough to identify the hyperspectral image distribution client device 200 a. The browsing image request transmission unit 202 a has a function to save the client ID or a function to generate the client ID.

[0034] (1-6) The view image request receiving unit 105a receives the view image request transmitted from the view image request transmitting unit 202a of the hyperspectral image distribution client device 200a. The view image request receiving unit 105a passes the received view image request to the session management unit 104a (S113).

[0035] (1-7) The session management unit 104a receives the request for an image to be viewed from the request receiving unit 105a for requesting an image to be viewed, and stores the request in association with the desired shooting time and the client ID included in the request for an image to be viewed.

[0036] The session management unit 104a passes a rendering image acquisition request including the desired shooting time included in the browsing image request to the rendering image recording unit 103a (S114).

[0037] (1-8) The rendering image recording unit 103a receives a rendering image acquisition request including the desired shooting time from the session management unit 104a.

[0038] The rendering image recording unit 103a checks whether a rendering image for the desired shooting time included in the rendering image acquisition request is recorded. If a rendering image that is an RGB image for the desired shooting time is recorded in the rendering image recording unit 103a, the rendering image recording unit 103a associates the RGB image with the desired shooting time and passes it to the RGB image sending unit 106a.

[0039] If the corresponding image is not recorded in the rendering image recording unit 103a, the rendering image recording unit 103a associates a conversion function, which is a function used for conversion into an RGB image, with the desired shooting time and passes it to the rendering unit 102a as rendering information (S121). In this embodiment, it is assumed that the conversion function is stored in advance in the rendering image recording unit 103a.

[0040] (1-9) The rendering unit 102a receives rendering information including a conversion function and a desired shooting time from the rendering image recording unit 103a.

[0041] The rendering unit 102a passes a hyperspectral data transmission request including the desired photographing time to the hyperspectral data recording unit 101a (S122).

[0042] (1-10) The hyperspectral data recording unit 101a receives the desired image capturing time included in the hyperspectral data transmission request from the rendering unit 102a. The hyperspectral data recording unit 101a extracts the hyperspectral data corresponding to the desired image capturing time and passes it to the rendering unit 102a (S123).

[0043] (1-11) The rendering unit 102a receives the hyperspectral data from the hyperspectral data recording unit 101a. The rendering unit 102a converts the hyperspectral data into data in the RGB space, i.e., an RGB image, using a conversion function, thereby generating an RGB image (S124).

[0044] Although conversion to data in RGB space is described here, depending on the output method of the hyperspectral image distribution client device 200a, conversion to data in, for example, YUV color space, sRGB color space, or Adobe RGB (registered trademark) color space may also be used.

[0045] (1-12) When performing visualization based on human vision, color matching functions are used as an example of conversion functions used to convert to RGB images. Color matching functions include x(λ), y(λ), and z(λ), and numerically represent the color vision response of a standard observer for each wavelength. For example, the color matching functions of the CIE 1931 colorimetric standard observer or the CIE 1964 colorimetric supplementary standard observer, as specified in ISO 11664-1 and JIS Z8781-1, are used.

[0046] When performing visualization other than visualization according to human vision, this can be achieved by using a function other than the color matching function and performing processing on the hyperspectral data similar to the processing described in Non-Patent Document 1 above.

[0047] (1-13) The rendering unit 102a associates the generated RGB image with the shooting time of the image (sometimes simply referred to as the shooting time) and passes it to the rendering image recording unit 103a (S125). (1-14) The rendering image recording unit 103a receives the RGB image and the shooting time of the image from the rendering unit 102a, associates them, and stores them (S126). The rendering image recording unit 103a associates the RGB image with the shooting time of the image and passes it to the session management unit 104a (S131).

[0048] (1-15) The session management unit 104a receives the RGB image and the shooting time of the image from the rendering image recording unit 103a. The session management unit 104a reads out the client ID saved in (1-7) based on the received shooting time. If necessary, the session management unit 104a extracts sender information from the client ID saved in (1-7). The following description will be given assuming that sender information is handled. The session management unit 104a passes the sender information as destination information to the RGB image and shooting time, linking them together (S132).

[0049] (1-16) The RGB image transmitting unit 106a receives the RGB image, the shooting time, and the destination information from the session managing unit 104a. The RGB image transmitting unit 106a associates the RGB image with the shooting time and transmits it to the RGB image receiving unit 203a of the hyperspectral image distribution client device 200a corresponding to the destination information (S133).

[0050] (1-17) The RGB image receiving unit 203a receives the RGB image and the shooting time from the RGB image sending unit 106a of the hyperspectral image distribution server device 100a. The RGB image receiving unit 203a associates the RGB image with the shooting time and passes it to the RGB image drawing unit 204a (S134).

[0051] (1-18) The RGB image drawing unit 204a receives the RGB image and the shooting time from the RGB image receiving unit 203a. The RGB image drawing unit 204a passes at least the RGB image to the screen unit 205a (S135). The data passed to the screen unit 205a may include the shooting time and may also include other elements.

[0052] (1-19) The screen unit 205a receives at least the RGB image, which is the data passed from the RGB image drawing unit 204a, and displays at least the RGB image, which is the received data, on the screen.

[0053] (1-20) In this embodiment, the display destination is the screen unit 205a, but it may also be another device to which the output of the hyperspectral image distribution client device 200a is connected. In this case, the RGB image rendering unit 204a passes at least the received RGB image to a screen device, which is another device connected to the hyperspectral image distribution client device 200a, and displays it on the screen.

[0054] (1-21) It is also possible that the output method of the hyperspectral image distribution client device 200a is to output an image file. In this case, the RGB image rendering unit 204a has a file output function. It is also possible that the output method of the hyperspectral image distribution client device 200a is to print. In this case, the RGB image rendering unit 204a has a printer output function.

[0055] Second Embodiment Next, a second embodiment will be described. In this second embodiment, the above-described conversion function can be controlled from a client device.

[0056] 3 is a diagram showing an application example of a distribution system according to a second embodiment of the present invention. As shown in Fig. 3, the distribution system according to the second embodiment includes a hyperspectral image distribution server device 100b, a hyperspectral image distribution client device 200b, and a hyperspectral camera 300b. While Fig. 3 shows an example in which multiple hyperspectral image distribution client devices 200b are provided, the number of devices is not particularly limited.

[0057] The hyperspectral image distribution server device 100b includes a hyperspectral data recording unit 101b, a rendering unit 102b, a rendering image recording unit 103b, a session management unit 104b, an image / transformation function information receiving unit 105b, and an RGB image transmitting unit 106b.

[0058] The hyperspectral image distribution client device 200b includes an image and transformation function specification unit 201b, an image and transformation function information transmission unit 202b, an RGB image reception unit 203b, an RGB image drawing unit 204b, and a screen unit 205b.

[0059] Next, a description will be given of the processing flow of the system in the distribution system according to the present embodiment, in which data captured by the hyperspectral camera 300 b is visualized based on a conversion function determined by the hyperspectral image distribution client device 200 b and displayed on the hyperspectral image distribution client device 200 b. Figures 4A and 4B are diagrams showing an example of the procedure of the processing operation of the distribution system according to the second embodiment of the present invention.

[0060] (2-1) The hyperspectral camera 300b is a camera that separates light into wavelengths and captures images, similar to the hyperspectral camera 300a.

[0061] (2-2) The hyperspectral image distribution server device 100b is a computer having a CPU and memory, similar to the hyperspectral image distribution server device 100a. The hyperspectral camera 300b transmits hyperspectral data acquired by image capture to the hyperspectral data recording unit 101b of the hyperspectral image distribution server device 100b (S201). The hyperspectral data recording unit 101b receives the hyperspectral data transmitted from the hyperspectral camera 300b, generates a hyperspectral data file (S202), and stores the file in association with the image capture time (S203).

[0062] The connection between the hyperspectral camera 300b and the hyperspectral image distribution server device 100b can be established using existing technology. The hyperspectral camera 300b and the hyperspectral image distribution server device 100b are connected, for example, by an Ethernet cable, and data captured by the hyperspectral camera 300b is transferred from the hyperspectral camera 300b to the server device 100b via communication in accordance with the GigE Vision standard and stored therein. The hyperspectral data recording unit 101b associates the hyperspectral data with the time of capture and stores the data.

[0063] (2-3) The hyperspectral image distribution client device 200b is a computer having a CPU and memory. The hyperspectral image distribution client device 200b is connected to the hyperspectral image distribution server device 100b. The hyperspectral image distribution client device 200b and the hyperspectral image distribution server device 100b are connected, for example, via a network and communicate with each other using IP.

[0064] In this embodiment, it is assumed that a plurality of hyperspectral image distribution client devices 200b are connected to one hyperspectral image distribution server device 100b.

[0065] Although not described in this embodiment, it is also possible to imagine a case where a plurality of hyperspectral image distribution server devices 100b are connected to one hyperspectral image distribution client device 200b.

[0066] (2-4-1) The hyperspectral image distribution client device 200b has an image and conversion function designation unit 201b, which allows the operator of the hyperspectral image distribution client device 200b to designate an image to be viewed and a conversion function to be used for rendering.

[0067] As a method for specifying an image to be viewed, it is conceivable to specify the most recent image or the image by specifying the time of photographing.

[0068] Methods for specifying a conversion function to be used in rendering include (1) a method in which the operator selects from a plurality of functions prepared in advance, (2) a method in which the operator specifies a function by inputting numerical values, (3) a method in which the operator specifies a function by inputting a mathematical expression, (4) a method in which the operator specifies a function by drawing a graph, (5) a method in which the operator specifies a function by modifying a mathematical expression or a graph based on a base function, or (6) a method in which the operator specifies a function by selecting a desired function from a plurality of base functions and making changes to the selected function. It is also possible to provide a default value for the conversion function to be used when no conversion function is specified.

[0069] The data of the conversion function consists of, for example, wavelength, red value, green value, and blue value, and is expressed as an array of values ​​such as the following (examples of data of the conversion function): 380, 0.0014, 0.0000, 0.0065 385, 0.0022, 0.0001, 0.0105 ...

[0070] (2-4-2) When the image / transformation function specifying unit 201b generates a transformation function, the image / transformation function specifying unit 201b generates the transformation function specified by the method shown in (2-4-1). In this case, the image / transformation function specifying unit 201b associates the shooting time of the image to be viewed (sometimes referred to as the desired shooting time or the desired HS data time) with the generated transformation function and passes it to the image / transformation function information transmitting unit 202b (S211). If the most recent image is specified, the shooting time of the image to be viewed may be information indicating that the most recent shooting time is the desired shooting time.

[0071] As another example, if the image / transformation function specification unit 201b does not generate a transformation function, the image / transformation function specification unit 201b passes the desired shooting time, a transformation function option, difference information from a base function, or the base function option and its difference information, etc., to the image / transformation function information transmission unit 202b. These transformation function options, difference information from a base function, or the base function option and its difference information, etc., may be referred to as transformation function information. Furthermore, "transformation function or transformation function information" may be referred to as "transformation function, etc." or "transformation function / information."

[0072] (2-5) The image and conversion function information transmitting unit 202b receives the desired photographing time and conversion function passed from the image and conversion function specifying unit 201b.

[0073] The image and transformation function information sending unit 202b associates the received desired shooting time, transformation function, etc. with the client ID and sends them to the image and transformation function information receiving unit 105b of the hyperspectral image distribution server device 100b (S212).

[0074] The client ID is used to identify the hyperspectral image distribution client device 200b. The client ID includes (1) destination information of the hyperspectral image distribution client device 200b, or (2) information that enables the session management unit 104b of the hyperspectral image distribution server device 100b to extract source information as destination information.

[0075] The client ID may be changed for each transmission from the same client device 200b, or may be fixed, as long as it is unique enough to identify the hyperspectral image distribution client device 200b. The image and transformation function information transmission unit 202b has a function to save the client ID or a function to generate the client ID.

[0076] (2-6) The image and transformation function information receiving unit 105b receives the desired photographing time, the transformation function, etc., and the client ID transmitted from the image and transformation function information transmitting unit 202b of the hyperspectral image distribution server device 100b.

[0077] The image and conversion function information receiving unit 105b associates the received desired shooting time, conversion function, etc. with the client ID and passes them to the session management unit 104b.

[0078] (2-7) The session management unit 104b receives the desired shooting time, the conversion function, etc., and the client ID from the image and conversion function information receiving unit 105b, and stores these as session information in association with each other (S213).

[0079] As another example, in the above (2-4-2), if the image / conversion function specification unit 201b does not generate a conversion function, and the session management unit 104b receives, not the conversion function itself, but a conversion function option, or difference information from a base function, or the base function option and its difference information, i.e., conversion function information, together with the desired shooting time and client ID, from the image / conversion function information receiving unit 105b, the session management unit 104b generates a conversion function based on this conversion function information and saves it in association with the desired shooting time and client ID.

[0080] The session management unit 104b associates the desired shooting time and the conversion function from the stored information and passes it to the rendering image recording unit 103b as a rendering image acquisition request (S214).

[0081] (2-8) The rendering image recording unit 103b receives a rendering image acquisition request including the desired shooting time and the conversion function from the session management unit 104b.

[0082] The rendering image recording unit 103b checks whether or not there is a recording of a rendering image whose photographing time matches the received desired photographing time and whose conversion function used for rendering matches the received conversion function.

[0083] If the matching RGB image, i.e., a rendering image, is recorded in the rendering image recording unit 103b, the rendering image recording unit 103b links the RGB image with the desired shooting time and passes it to the RGB image transmitting unit 106b.

[0084] If the corresponding image is not recorded in the rendering image recording unit 103b, the rendering image recording unit 103b associates the received conversion function with the desired shooting time and passes it to the rendering unit 102b as rendering information (S221).

[0085] (2-9) The rendering unit 102b receives rendering information including the conversion function and the desired shooting time from the rendering image recording unit 103b.

[0086] The rendering unit 102b passes a hyperspectral data transmission request including the desired photographing time to the hyperspectral data recording unit 101b (S222).

[0087] (2-10) The hyperspectral data recording unit 101b receives a hyperspectral data transmission request including the desired photographing time from the rendering unit 102b.

[0088] The hyperspectral data recording unit 101b extracts the hyperspectral data corresponding to the desired photographing time and passes it to the rendering unit 102b (S223).

[0089] (2-11) The rendering unit 102b receives the hyperspectral data from the hyperspectral data recording unit 101b.

[0090] The rendering unit 102b converts the hyperspectral data into data in the RGB space using a conversion function, thereby generating an RGB image (S224).

[0091] (2-12) As in the first embodiment, when visualization is performed based on human vision, a color matching function is used as an example of a conversion function. (2-13) The rendering unit 102b associates the generated RGB image, the capture time of the image, and the conversion function, and passes them to the rendering image recording unit 103b (S225). (2-14) The rendering image recording unit 103b receives the RGB image, the capture time of the image, and the conversion function from the rendering unit 102b, associates them, and saves them (S226).

[0092] The rendering image recording unit 103b associates the RGB image with the capture time and conversion function of the image, and passes the image to the session management unit 104b (S231).

[0093] (2-15) The session management unit 104b receives the RGB image, the shooting time of the image, and the conversion function from the rendering image recording unit 103b. The session management unit 104b reads out the client ID saved in (2-7) based on the received shooting time and conversion function. If necessary, the session management unit 104b extracts sender information from the client ID saved in (2-7). The following description will be given assuming that sender information is handled.

[0094] If necessary, based on the received shooting time and the client ID saved in (2-7), the session management unit 104b reads out conversion function information, such as conversion function options, difference information from the base function, or the base function options and their difference information, as information on the source of the conversion function linked to the above shooting time and client ID.

[0095] The session management unit 104b associates the sender information as destination information with the RGB image, the shooting time, the conversion function, and the like, and passes it to the RGB image sending unit 106b (S232).

[0096] (2-16) The RGB image transmitting unit 106b receives the RGB image, the shooting time, the conversion function, etc., and the destination information from the session managing unit 104b.

[0097] The RGB image transmitting unit 106b associates the RGB image with the shooting time, the conversion function, and the like, and transmits the image to the RGB image receiving unit 203b of the hyperspectral image distribution client device 200b corresponding to the destination information (S233).

[0098] (2-17) The RGB image receiving unit 203b receives the RGB image, the shooting time, the conversion function, and the like from the RGB image transmitting unit 106b of the hyperspectral image distribution server device 100b.

[0099] The RGB image receiving unit 203b associates the RGB image with the shooting time, conversion function, etc., and passes them to the RGB image drawing unit 204b (S234).

[0100] (2-18) The RGB image drawing unit 204b receives the RGB image, the shooting time, the conversion function, and the like from the RGB image receiving unit 203b.

[0101] The RGB image drawing unit 204b passes at least the RGB image to the screen unit 205b (S235).

[0102] This passed data may include the time of shooting, the conversion function or conversion function information, and other elements.

[0103] (2-19) The screen unit 205b receives at least the RGB image, which is the data passed from the RGB image drawing unit 204b, and displays at least the RGB image, which is the received data, on the screen.

[0104] (2-20) In this embodiment, the display destination is the screen unit 205b, but it is also possible that the display destination is another device to which the output of the hyperspectral image distribution client device 200b is connected. In this case, the RGB image rendering unit 204b passes at least the received RGB image to a screen device, which is another device connected to the hyperspectral image distribution client device 200b, and displays it on the screen.

[0105] (2-21) It is also possible that the hyperspectral image distribution client device 200b outputs an image file as an output method. In this case, the RGB image drawing unit 204b has a file output function.

[0106] It is also possible that the hyperspectral image distribution client device 200b outputs the image by printing, in which case the RGB image rendering unit 204b has a function of outputting the image to a printer.

[0107] Third Embodiment Next, a third embodiment will be described. In this third embodiment, a communication path for an image stream is established. FIG. 5 is a diagram showing an application example of a distribution system according to the third embodiment of the present invention. As shown in FIG. 5, the distribution system according to the third embodiment includes a hyperspectral image distribution server device 100c, a hyperspectral image distribution client device 200c, and a hyperspectral camera 300c.

[0108] The hyperspectral image distribution server device 100c includes a hyperspectral data recording unit 101c, a rendering unit 102c, a rendering image recording unit 103c, a session management unit 104c, an image / transformation function information receiving unit 105c, and an RGB image transmitting unit 106c.

[0109] The hyperspectral image distribution client device 200c includes an image / transformation function designation unit 201c, an image / transformation function information transmission unit 202c, an RGB image reception unit 203c, an RGB image drawing unit 204c, a screen unit 205c, and a communication path control unit 206c.

[0110] Next, a description will be given of the processing flow of a system in which data captured by a hyperspectral camera 300 c is visualized based on a transformation function determined by a hyperspectral image distribution client device 200 c, and the data can be push-distributed from a hyperspectral image distribution server device 100 c and displayed on the hyperspectral image distribution client device 200 c in a distribution system according to the third embodiment of the present invention. Figures 6A and 6B are diagrams showing an example of the procedure of a first processing operation of a distribution system according to the third embodiment of the present invention.

[0111] (3-1) The hyperspectral camera 300c, like the hyperspectral camera 300a, is a camera that separates light into wavelengths and captures images.

[0112] (3-2) The hyperspectral image distribution server device 100c is a computer having a CPU and a memory, similar to the hyperspectral image distribution server device 100a.

[0113] The hyperspectral camera 300c transmits the hyperspectral data obtained by capturing an image to the hyperspectral data recording unit 101c of the hyperspectral image distribution server device 100c (S301). The hyperspectral data recording unit 101c receives the hyperspectral data transmitted from the hyperspectral camera 300c, generates a hyperspectral data file (S302), and stores the file in association with the capture time (S303).

[0114] The connection between the hyperspectral camera 300c and the hyperspectral image distribution server device 100c can be established using existing technology.

[0115] The hyperspectral camera 300c and the hyperspectral image distribution server device 100c are connected by, for example, an Ethernet cable, and data captured by the hyperspectral camera 300c is transferred from the hyperspectral camera 300b to the server device 100c via communication in accordance with the GigE Vision standard, where it is stored. The hyperspectral data recording unit 101c stores the hyperspectral data in association with the capture time.

[0116] (3-3) The hyperspectral image distribution client device 200c is a computer having a CPU and a memory.

[0117] The hyperspectral image distribution client device 200c is connected to the hyperspectral image distribution server device 100c.

[0118] The hyperspectral image distribution client device 200c and the hyperspectral image distribution server device 100c are connected, for example, via a network and communicate with each other using IP.

[0119] In this embodiment, it is assumed that a plurality of hyperspectral image distribution client devices 200c are connected to one hyperspectral image distribution server device 100c.

[0120] Although not described in this embodiment, it is also possible to imagine a case where a plurality of hyperspectral image distribution server devices 100c are connected to one hyperspectral image distribution client device 200c.

[0121] (3-4-1) The hyperspectral image distribution client device 200c includes a communication path control unit 206c, which holds a client ID as an identifier of the hyperspectral image distribution client device 200c itself.

[0122] In the communication path control unit 206c, the operator of the hyperspectral image distribution client device 200c can request the establishment of a communication path with the hyperspectral image distribution server device 100c by operating the communication path control unit 206c.

[0123] When the communication path control unit 206c receives a request from the operator to establish a communication path with the hyperspectral image distribution server device 100c, it sends the client ID held above and a request to establish a communication path to the RGB image receiving unit 203c (S311).

[0124] (3-4-2) The client ID is used to identify the hyperspectral image distribution client device 200c.

[0125] The client ID may be changed for each transmission from the same client device 200c, or may remain unchanged, as long as it is unique enough to identify the hyperspectral image distribution client device 200c.

[0126] The client ID includes (1) destination information of the hyperspectral image distribution client device 200c, or (2) information that enables the session management unit 104c of the hyperspectral image distribution server device 100c to extract source information as destination information.

[0127] (3-4-3) The RGB image receiving unit 203c transmits the client ID and a request for establishing a communication path to the RGB image transmitting unit 106c of the hyperspectral image distribution server device 100c (S312).

[0128] (3-4-4) The RGB image transmission unit 106c accepts the request to establish a communication path, and establishes a communication path that allows two-way communication (S313).

[0129] When accepting a request to establish a communication path, the RGB image transmitting unit 106c may perform a process of checking whether the client ID is valid, and accept the request to establish a communication path only if the client ID is valid.

[0130] (3-4-5) When the communication path is established, the RGB image transmission unit 106c passes the client ID and a notification that the communication path has been established to the session management unit 104c (S314).

[0131] (3-4-6) The session management unit 104c stores the client ID as identification information of the client with which the communication path is currently established.

[0132] (3-4-7) The hyperspectral image distribution client device 200c has an image and conversion function designation unit 201c, which allows the operator of the hyperspectral image distribution client device 200c to designate an image to be viewed and a conversion function to be used for rendering.

[0133] The method of specifying the image to be viewed may be to specify the most recent image, or to specify the image by specifying the time of shooting, etc. If no particular specification is made, the most recent image may be treated as the image to be viewed.

[0134] The method of specifying the conversion function used for rendering includes, but is not limited to, (1) to (6) described in the second embodiment. It is also possible to prepare a default value for the conversion function to be used when no conversion function is specified.

[0135] (3-4-8) When the image and transformation function specifying unit 201c generates a transformation function, the image and transformation function specifying unit 201c generates the transformation function specified by the method shown in (3-4-7). In this case, the image and transformation function specifying unit 201c associates the shooting time of the image to be viewed with the generated transformation function and passes it to the image and transformation function information transmitting unit 202c (S321). If the most recent image is specified, the shooting time of the image to be viewed may be information indicating that the most recent shooting time is the desired shooting time.

[0136] As another example, if the image / conversion function specification unit 201c does not generate a conversion function, the image / conversion function specification unit 201c passes the desired shooting time, a conversion function option, or difference information from the base function, or the base function option and its difference information, i.e., conversion function information, to the image / conversion function information transmission unit 202c.

[0137] (3-5) The image and conversion function information transmitting unit 202c receives the desired photographing time and conversion function passed from the image and conversion function specifying unit 201c.

[0138] The image / transformation function information sending unit 202c links the received desired shooting time, transformation function, etc. with the client ID stored above and sends them to the image / transformation function information receiving unit 105c of the hyperspectral image distribution server device 100c (S322).

[0139] The client ID is used to identify the hyperspectral image distribution client device 200c. The client ID includes (1) destination information of the hyperspectral image distribution client device 200c, or (2) information that enables the session management unit 104c of the hyperspectral image distribution server device 100c to extract source information as destination information.

[0140] The client ID may be changed each time a transmission is made from the same client device 200 c, or may be fixed, as long as it is unique enough to identify the hyperspectral image distribution client device 200 c. The image and transformation function information transmission unit 202 c has a function to save the client ID or a function to generate the client ID.

[0141] (3-6) The image and transformation function information receiving unit 105c receives the desired shooting time, the transformation function, etc., and the client ID transmitted from the image and transformation function information transmitting unit 202c of the hyperspectral image distribution server device 100c. The image and transformation function information receiving unit 105c associates the received desired shooting time, the transformation function, etc., with the client ID and passes them to the session management unit 104c.

[0142] (3-7) The session management unit 104c receives the desired shooting time, the conversion function, etc., and the client ID from the image and conversion function information receiving unit 105c, and stores these as session information in association with each other (S323).

[0143] As another example, in the case where the image / conversion function specification unit 201c does not generate a conversion function in (3-4-8) above, if the session management unit 104c receives, not the conversion function itself, but conversion function options, or difference information from a base function, or the base function options and their difference information, i.e., conversion function information, from the image / conversion function information receiving unit 105c, the session management unit 104c generates a conversion function based on this conversion function information and stores it in association with the desired shooting time and client ID. The session management unit 104c associates the desired shooting time and conversion function from the stored information and passes it to the rendering image recording unit 103c as a rendering image acquisition request (S324).

[0144] (3-8) The rendering image recording unit 103c receives a rendering image acquisition request including the desired shooting time and the conversion function from the session management unit 104c. The rendering image recording unit 103c checks whether a rendering image whose shooting time matches the received desired shooting time and whose conversion function used for rendering matches the received conversion function is recorded. If the matching RGB image, i.e., a rendering image, is recorded in the rendering image recording unit 103c, the rendering image recording unit 103c links the RGB image with the desired shooting time and passes it to the RGB image sending unit 106c.

[0145] If the corresponding image is not recorded in the rendering image recording unit 103c, the rendering image recording unit 103c associates the received conversion function with the desired shooting time and passes it to the rendering unit 102c as rendering information (S331).

[0146] (3-9) The rendering unit 102c receives rendering information including the conversion function and the desired shooting time from the rendering image recording unit 103c. If the rendering unit 102c has not received hyperspectral data from the hyperspectral data recording unit 101c, the rendering unit 102c sends a hyperspectral data transmission request to the hyperspectral data recording unit 101c (S332).

[0147] On the other hand, when hyperspectral data is received from the hyperspectral data recording unit 101c, the rendering unit 102c generates an RGB image by converting the hyperspectral data into data in the RGB space using a conversion function.

[0148] (3-10) The hyperspectral data recording unit 101c receives the hyperspectral data transmission request from the rendering unit 102c, and passes the latest hyperspectral data to the rendering unit 102c (S341).

[0149] (3-11) The rendering unit 102c receives the hyperspectral data from the hyperspectral data recording unit 101c. The rendering unit 102c converts the hyperspectral data into data in the RGB space using a conversion function, thereby generating an RGB image (S342).

[0150] (3-12) As in the first embodiment, when visualization is performed based on human vision, a color matching function is used as an example of a conversion function.

[0151] (3-13) The rendering unit 102c associates the generated RGB image with the image capture time and the conversion function, and passes them to the rendering image recording unit 103c (S343). (3-14) The rendering image recording unit 103c receives the RGB image, the image capture time, and the conversion function from the rendering unit 102c, associates them, and stores them (S344).

[0152] The rendering image recording unit 103c associates the RGB image with the capture time and conversion function of the image, and passes the image to the session management unit 104c (S345).

[0153] (3-15) The session management unit 104c receives the RGB image, the shooting time of the image, and the conversion function from the rendering image recording unit 103c. The session management unit 104c reads out the client ID saved in (3-7) based on the received shooting time and conversion function. If necessary, the session management unit 104c extracts sender information from the client ID. The following description will be given assuming that sender information is handled.

[0154] If necessary, based on the received shooting time and the client ID saved in (3-7), the session management unit 104c reads out conversion function information, such as conversion function options, difference information from the base function, or the base function options and their difference information, as information on the source of the conversion function linked to the above shooting time and client ID.

[0155] The session management unit 104c associates the sender information as destination information with the RGB image, the shooting time, the conversion function, and the like, and passes it to the RGB image sending unit 106c (S346).

[0156] (3-16) The RGB image sending unit 106c receives the RGB image, the shooting time, the conversion function, etc., and the destination information from the session management unit 104c.

[0157] The RGB image transmitting unit 106c associates the RGB image with the shooting time, the conversion function, and the like, and transmits the image to the RGB image receiving unit 203c of the hyperspectral image distribution client device 200c corresponding to the destination information (S347).

[0158] (3-17) The RGB image receiving unit 203c receives the RGB image, the shooting time, the conversion function, and the like from the RGB image transmitting unit 106c of the hyperspectral image distribution server device 100c.

[0159] The RGB image receiving unit 203c associates the RGB image with the shooting time, the conversion function, and the like, and passes them to the RGB image drawing unit 204c (S348).

[0160] (3-18) The RGB image drawing unit 204c receives the RGB image, the shooting time, the conversion function, etc. from the RGB image receiving unit 203c. The RGB image drawing unit 204c passes at least the RGB image to the screen unit 205c (S349).

[0161] This passed data may include the time of shooting, the conversion function or conversion function information, and other elements.

[0162] (3-19) The screen unit 205c receives at least the RGB image, which is the data passed from the RGB image drawing unit 204c, and displays at least the RGB image, which is the received data, on the screen.

[0163] (3-20) In this embodiment, the display destination is the screen unit 205c, but it is also possible that the display destination is another device to which the output of the hyperspectral image distribution client device 200c is connected.

[0164] In this case, the RGB image drawing unit 204c passes at least the received RGB image to a screen device, which is a separate device connected to the hyperspectral image distribution client device 200c, and displays it on the screen.

[0165] (3-21) It is also possible that the hyperspectral image distribution client device 200c outputs an image file as an output method. In this case, the RGB image drawing unit 204c has a file output function.

[0166] It is also possible that the output method of the hyperspectral image distribution client device 200c is printing. In this case, the RGB image drawing unit 204c has a function of outputting to a printer.

[0167] (3-22) The following describes the processing flow for stopping image stream reception in the server device. Fig. 7 is a diagram showing an example of the procedure of the second processing operation of the distribution system according to the third embodiment of the present invention. (3-23) The communication path control unit 206c of the hyperspectral image distribution client device 200c passes a request to stop stream reception to the RGB image receiving unit 203c (S351).

[0168] (3-24) When the RGB image receiving unit 203c receives a request to stop receiving the stream from the communication path control unit 206c, it releases the communication path established between the RGB image receiving unit 203c and the RGB image transmitting unit 106c of the hyperspectral image distribution server device 100c (S352).

[0169] (3-25) When the RGB image transmission unit 106c of the hyperspectral image distribution server device 100c detects that the communication path with the hyperspectral image distribution client device 200c has been released (S353), it passes a stop notification including the client ID of the hyperspectral image distribution client device 200c related to this communication path to the session management unit 104c (S354).

[0170] (3-26) The session management unit 104c extracts the conversion function used by the client device associated with the client ID from the client ID included in the stop notification received from the RGB image transmission unit 106c. The session management unit 104c checks whether there is a client ID of a client device that uses the same conversion function as the client ID. After checking, the session management unit 104c deletes the data storing the combination of the client ID.

[0171] (3-27) If there is a client ID of a client device using the same conversion function, the process related to the stop request is terminated. (3-28) If there is no client ID of a client device using the same conversion function, the session management unit 104c passes the extracted conversion function and a rendering stop request to the rendering image recording unit 103c (S361).

[0172] (3-29) When the rendering image recording unit 103c receives the conversion function and the request to stop rendering, it passes the conversion function and the request to stop rendering to the rendering unit 102c (S362).

[0173] (3-30) When the rendering unit 102c receives the conversion function and the request to stop rendering, it stops rendering using the conversion function, that is, stops converting the hyperspectral data into an RGB image (S363).

[0174] (3-31) When the rendering unit 102c detects that all rendering using any conversion function has stopped, it sends a request to stop transmitting hyperspectral data to the hyperspectral data recording unit 101c (S371).

[0175] (3-32) When the hyperspectral data recording unit 101c receives the request to stop transmitting hyperspectral data, it stops transmitting the hyperspectral data to the rendering unit 102c (S372).

[0176] (3-33) Of the above descriptions, the contents described in (3-4-7) to (3-9) show the flow of processing that starts when the conversion function is set.

[0177] (3-34) The contents described in (3-10) to (3-21) show the flow of the process of receiving images, etc. from the hyperspectral image distribution server device 100c and drawing the images on the hyperspectral image distribution client device 200c, triggered by the generation of a hyperspectral data file from the image captured by the hyperspectral camera 300c.

[0178] (3-35) In the processing described in this order, before the hyperspectral image distribution client device 200c receives an image from the hyperspectral image distribution server device 100c, the hyperspectral image distribution client device 200c transmits information about the conversion function to the hyperspectral image distribution server device 100c.

[0179] (3-36) However, when a hyperspectral data file is generated, the hyperspectral image distribution client device 200c can receive an image or the like from the hyperspectral image distribution server device 100c and transmit information about the conversion function to the hyperspectral image distribution server device 100c while the hyperspectral image distribution client device 200c is performing the process of drawing the image.

[0180] (3-37) In this case, the process flow in which the hyperspectral image distribution client device 200c transmits information about the transformation function to the hyperspectral image distribution server device 100c is the same as that described in (3-4-7) to (3-9).

[0181] (3-38) In addition, when a hyperspectral data file is generated, an image or the like is received once from the hyperspectral image distribution server device 100c, and while the hyperspectral image distribution client device 200c is performing the process of drawing the image, the hyperspectral image distribution client device 200c can transmit information about the conversion function to the hyperspectral image distribution server device 100c multiple times.

[0182] (Fourth Embodiment) Next, a fourth embodiment will be described. In this fourth embodiment, video is distributed. FIG. 8 is a diagram showing an application example of a distribution system according to the fourth embodiment of the present invention. As shown in FIG. 8, the distribution system according to the fourth embodiment includes a hyperspectral video distribution server device 100d, a hyperspectral video distribution client device 200d, and a hyperspectral camera 300d.

[0183] The hyperspectral video distribution server device 100d includes a hyperspectral data recording unit 101d, a rendering unit 102d, a video storage unit 103d, a session management unit 104d, a conversion function information receiving unit 105d, and an RGB video transmitting unit 106d.

[0184] The hyperspectral video distribution client device 200d includes a conversion function designation unit 201d, a conversion function information transmission unit 202d, an RGB video reception unit 203d, an RGB video drawing unit 204d, a screen unit 205d, and a video control unit 206d.

[0185] Next, a description will be given of the processing flow of the system in the distribution system according to this embodiment, in which video data captured by the hyperspectral camera 300 d is visualized based on a conversion function determined by the hyperspectral video distribution client device 200 d, and the video is displayed on the hyperspectral video distribution client device 200 d. Figures 9A and 9B are diagrams showing an example of the procedure of the first processing operation of the distribution system according to the fourth embodiment of the present invention.

[0186] There are publicly available methods for distributing video from the hyperspectral video distribution server device 100d over a network and receiving and playing the video at the hyperspectral video distribution client device 200d, such as standards such as HLS (HTTP Live Streaming) and MPEG-DASH (MPEG Dynamic Adaptive Streaming over HTTP) (ISO / IEC 23009-1).

[0187] (4-1) The hyperspectral camera 300d is a camera that separates light into wavelengths and captures images, similar to the hyperspectral camera 300a.

[0188] (4-2) The hyperspectral video distribution server device 100d is a computer having a CPU and a memory, similar to the hyperspectral image distribution server device 100a.

[0189] The hyperspectral camera 300d transmits the hyperspectral data obtained by capturing the image to the hyperspectral data recording unit 101d of the hyperspectral video distribution server device 100d (S401). The hyperspectral data recording unit 101d of the hyperspectral video distribution server device 100d receives the hyperspectral data transmitted from the hyperspectral camera 300d, generates a hyperspectral data file (S402), and stores the file in association with the capture time (S403).

[0190] The connection between the hyperspectral camera 300d and the hyperspectral video distribution server device 100d can be established using existing technology. The hyperspectral camera 300d and the hyperspectral video distribution server device 100d are connected, for example, by an Ethernet cable, and hyperspectral data captured by the hyperspectral camera 300d is transferred from the hyperspectral camera 300d to the server device 100d via communication in accordance with the GigE Vision standard and stored therein. The hyperspectral data recording unit 101d associates the hyperspectral data with the capture time and stores the data.

[0191] (4-3) The hyperspectral video distribution client device 200d is a computer having a CPU and a memory, and is connected to the hyperspectral video distribution server device 100d.

[0192] The hyperspectral video distribution client device 200d and the hyperspectral video distribution server device 100d are connected, for example, via a network and communicate with each other using IP. In this embodiment, it is assumed that multiple hyperspectral video distribution client devices 200d are connected to one hyperspectral video distribution server device 100d.

[0193] (4-4-1) The hyperspectral video distribution client device 200d includes a conversion function designation unit 201d, which allows the operator of the hyperspectral video distribution client device 200d to designate a conversion function to be used for rendering.

[0194] The method of specifying the conversion function used for rendering includes, but is not limited to, (1) to (6) described in the second embodiment. It is also possible to prepare a default value for the conversion function to be used when no conversion function is specified.

[0195] The conversion function designation unit 201d may have a function for designating a video. Possible methods for designating a video playback position include (1) designating the latest video, or (2) designating a playback start time to designate a video playback position. It is also possible to prepare a default video value to be used when no video is designated. In this embodiment, an example is shown in which, when no video is designated, the latest video is assumed to be designated and the device operates accordingly.

[0196] (4-4-2) When the conversion function designation unit 201d generates a conversion function, the conversion function designation unit 201d generates the conversion function designated by the method shown in (4-4-1).

[0197] In this case, the conversion function designation unit 201d passes the generated conversion function to the conversion function information transmission unit 202d (S411).

[0198] As another example, if the conversion function specification unit 201d does not generate a conversion function, the conversion function specification unit 201d passes conversion function information, such as options for the conversion function, or difference information from the base function, or options for the base function and their difference information, to the conversion function information transmission unit 202d.

[0199] (4-5) The conversion function information transmitting unit 202d receives the conversion functions and the like passed from the conversion function specifying unit 201d.

[0200] The conversion function information transmitting unit 202d associates the received conversion function and the like with its own device's client ID and transmits them to the conversion function information receiving unit 105d of the hyperspectral video distribution server device 100d (S412).

[0201] The client ID is used to identify the hyperspectral video distribution client device 200d. The client ID includes (1) destination information of the hyperspectral video distribution client device 200d, or (2) information that enables the session management unit 104d of the hyperspectral video distribution server device 100d to extract source information as destination information.

[0202] The client ID may be changed for each transmission from the same client device 200d, or may be fixed as long as it is unique enough to identify the hyperspectral video distribution client device 200d.

[0203] The conversion function information transmitting unit 202d has a function of saving a client ID or a function of generating a client ID.

[0204] (4-6) The conversion function information receiving unit 105d receives the conversion function and the client ID transmitted from the conversion function information transmitting unit 202d of the hyperspectral video distribution server device 100d.

[0205] The conversion function information receiving unit 105d associates the received conversion function and the like with the client ID and passes them to the session management unit 104d.

[0206] (4-7) The session management unit 104d receives the conversion function and the client ID from the conversion function information receiving unit 105d, associates them, and stores them as session information (S413).

[0207] As another example, when the session management unit 104d receives, from the conversion function information receiving unit 105d, not the conversion function itself but an option for the conversion function, or difference information from a base function, or an option for the base function and the difference information, i.e., conversion function information, the session management unit 104d generates a conversion function based on this conversion function information and uses it for subsequent processing. The session management unit 104d holds the received or generated conversion function, associates this conversion function with the client ID, and passes it to the video storage unit 103d as a video generation request (S414).

[0208] (4-8) The video storage unit 103d receives a video generation request including a conversion function and a client ID from the session management unit 104d.

[0209] The video storage unit 103d checks whether there is video in which the received conversion function matches the conversion function used for rendering. If the video storage unit 103d finds the matching video, it passes the video to the RGB video transmission unit 106d. If the video storage unit 103d does not find the matching video, it passes the received conversion function to the rendering unit 102d as rendering information (S421).

[0210] (4-9) The rendering unit 102d receives rendering information including a conversion function from the video storage unit 103d. If no hyperspectral data has been received from the video storage unit 103d, the rendering unit 102d passes a request to transmit the hyperspectral data to the hyperspectral data recording unit 101d (S422).

[0211] (4-10) On the other hand, when the hyperspectral data recording unit 101d receives a request to send hyperspectral data from the video storage unit 103d, it links the latest hyperspectral data at that time with the time of shooting of that data and passes it to the rendering unit 102d (S431), triggered by the generation of a hyperspectral data file thereafter until it receives a request to stop sending the hyperspectral data from the video storage unit 103d.

[0212] (4-11) The rendering unit 102d receives the hyperspectral data from the hyperspectral data recording unit 101d. The rendering unit 102d converts the hyperspectral data into data in the RGB space using a conversion function, thereby generating an RGB image (S432).

[0213] Although conversion to data in the RGB space is described here, depending on the output method of the hyperspectral video distribution client device 200d, conversion to data in, for example, the YUV color space, the sRGB color space, or the Adobe RGB color space may also be used. The type of color space of the data that can be displayed by the hyperspectral video distribution client device 200d may be preset in the rendering unit 102d.

[0214] (4-12) As in the first embodiment, when visualization is performed based on human vision, a color matching function is used as an example of a conversion function.

[0215] (4-13) The rendering unit 102d associates the generated RGB image with the shooting time of the image and the conversion function used, and passes them to the video storage unit 103d (S433). (4-14-1) The video storage unit 103d receives the RGB image, shooting time, and conversion function from the rendering unit 102d, associates them, and stores them. The video storage unit 103d generates a video (moving image) using a combination of the received RGB image (still image) and information on the shooting time of the image (S434).

[0216] (4-14-2) There are various possible methods for generating a video, but here is one example: The video storage unit 103d stores a combination of a video, information on the shooting time of the image used in the final frame of the video, and information on the transformation function used to generate the video.

[0217] The video storage unit 103d checks, based on the conversion function and the shooting time received from the rendering unit 102d, whether there is any video to be added that has the same conversion function and consecutive shooting times.

[0218] As a query method, if the conversion function is the same as the stored one and the latest shooting time is within the shooting interval of the hyperspectral camera 300d from the stored shooting time (for example, within one second if the shooting interval is once per second), it is considered that there is video to add. If there is video to add, the video accumulation unit 103d frames the image after the final frame of the stored video and adds the same frames for the shooting interval.

[0219] The video storage unit 103d overwrites and saves the video, and updates the shooting time information of the image used for the final frame of the video. If there is no video to add, the video storage unit 103d frames the image and creates a new video consisting of the same frames for the shooting interval. The video storage unit 103d saves the video, and saves the shooting time information of the image used for the final frame of the video, linked to the saved video.

[0220] Fig. 10 is a diagram showing an example of video frames and shooting times. The example shown in Fig. 10 is an example when the video frame rate is 30 fps, and the video corresponding to code a is video stored in the video storage unit 103d, while the video corresponding to code b is video added based on this stored video.

[0221] (4-14-3) When the HLS or MPEG-DASH standard is used, the video storage unit 103d generates and stores segment files of the stored video as needed (S435).

[0222] (4-14-4) The video storage unit 103d updates the playlist of the stored videos, associates this playlist with the conversion function received from the rendering unit 102d, and passes it to the session management unit 104d (S436).

[0223] (4-15) The session management unit 104d receives the playlist and the conversion function from the video storage unit 103d. The session management unit 104d associates the received playlist and conversion function with each other and stores them.

[0224] (4-16) The video control unit 206d of the hyperspectral video distribution client device 200d has an interface that allows an operator to operate the video control unit 206d. The interface provided in the video control unit 206d has at least a function of accepting play and stop operations.

[0225] 11 is a diagram showing an example of the procedure of the second processing operation of the distribution system according to the fourth embodiment of the present invention. When the video control unit 206d of the hyperspectral video distribution client device 200d receives a playback instruction, the video control unit 206d passes a client ID and an instruction to obtain a playlist to the RGB video receiving unit 203d (S441).

[0226] (4-17) The RGB video receiving unit 203d receives the client ID and an instruction to obtain a playlist from the video control unit 206d.

[0227] (4-18) The RGB video receiving unit 203d sends a request to acquire the client ID and the playlist to the session management unit 104d of the hyperspectral video distribution server device 100d (S442).

[0228] (4-19) The session management unit 104d of the hyperspectral video distribution server device 100d receives the client ID and the request to acquire the playlist from the RGB video receiving unit 203d of the hyperspectral video distribution client device 200d.

[0229] As described in (4-7), the session management unit 104d associates the conversion function received from the conversion function information receiving unit 105d with the client ID and stores them. The session management unit 104d extracts source information from the client ID received from the RGB video receiving unit 203d, extracts the corresponding conversion function from the client ID, and extracts a playlist corresponding to this conversion function. The session management unit 104d sends the extracted playlist as a playlist acquisition response to the RGB video receiving unit 203d of the hyperspectral video distribution client device 200d associated with the extracted source information (S443).

[0230] (4-20) The RGB video receiving unit 203d of the hyperspectral video distribution client device 200d receives the playlist from the session management unit 104d of the hyperspectral video distribution server device 100d.

[0231] In order to acquire a video segment file, which is video data, according to the URL of the video segment described in the playlist received from the session management unit 104d, the RGB video receiving unit 203d transmits a request to acquire the video segment file to the RGB video sending unit 106d, linking the request to a video segment file ID, which is an identifier of the video segment file (S451). This video segment ID may be the URL of the video segment.

[0232] (4-21) The RGB video transmitter 106d of the hyperspectral video distribution server device 100d receives the video segment file ID and a request to acquire the video segment file from the RGB video receiver 203d of the hyperspectral video distribution client device 200d. The RGB video transmitter 106d associates the video segment file ID with the request to acquire the video data and passes it to the video storage unit 103d (S452).

[0233] (4-22) The video storage unit 103d receives the video segment file ID and a request to obtain video data from the RGB video transmission unit 106d. The video storage unit 103d extracts video data according to the video segment file ID received from the RGB video transmission unit 106d. If the video storage unit 103d finds matching video data, it transmits the video data to the RGB video transmission unit 106d as a video segment file (S453).

[0234] (4-23) The RGB video transmission unit 106d receives the video segment file from the video storage unit 103d. The RGB video transmission unit 106d of the hyperspectral video distribution server device 100d sends the video segment file received from the video storage unit 103d to the RGB video reception unit 203d of the hyperspectral video distribution client device 200d (S454).

[0235] (4-31) The RGB video receiving unit 203d of the hyperspectral video distribution client device 200d receives the video segment file from the RGB video sending unit 106d of the hyperspectral video distribution server device 100d.

[0236] The RGB video receiving unit 203d passes the received video segment file to the RGB video drawing unit 204d (S455).

[0237] (4-32) The RGB video rendering unit 204d plays the video segment file received from the RGB video receiving unit 203d, and passes the video data to the screen unit 205d (S456).

[0238] (4-33) The screen unit 205d displays the video data passed from the RGB video drawing unit 204d. (4-34) When the RGB video receiving unit 203d requests acquisition of the last segment file described in the playlist, it requests the session management unit 104d to acquire a new playlist (S457).

[0239] The session management unit 104d sends the new playlist to the RGB video receiving unit 203d of the hyperspectral video distribution client device 200d as a playlist acquisition response (S458).

[0240] (4-35) The following describes the process flow for stopping the reception of a video stream on the server device. Fig. 12 is a diagram showing an example of the procedure of the third processing operation of the distribution system according to the fourth embodiment of the present invention. (4-36) The video control unit 206d of the hyperspectral video distribution client device 200d passes a request to stop receiving the stream to the RGB video receiving unit 203d (S461).

[0241] (4-37) When the RGB video receiving unit 203d receives a request to stop receiving the stream from the video control unit 206d, it passes a stop notification including the client ID of its own device to the session management unit 104d (S462).

[0242] (4-38) The session management unit 104d extracts the conversion function used by the client device associated with the client ID from the client ID included in the stop notification received from the RGB video transmission unit 106d. The session management unit 104d checks whether there is a client ID of a client device that is playing video using the same conversion function as the relevant conversion function.

[0243] After the inquiry, the session management unit 104d deletes the data that stores the combination of the client ID.

[0244] (4-39) If there is a client ID of a client device that is playing video using the same conversion function, the process related to the stop request is terminated.

[0245] (4-40) If there is no client ID of a client device using the same conversion function, the session management unit 104d associates the extracted conversion function with a request to stop rendering and passes it to the video storage unit 103d (S471).

[0246] (4-41) When the video storage unit 103d receives the conversion function and the request to stop rendering, it associates the conversion function with the request to stop rendering and passes them to the rendering unit 102d (S472).

[0247] (4-42) When the rendering unit 102d receives the conversion function and the request to stop rendering, it stops rendering using the conversion function (S473).

[0248] (4-43) When the rendering unit 102d detects that all rendering using any conversion function has stopped, it sends a request to stop transmitting hyperspectral data to the hyperspectral data recording unit 101d (S474).

[0249] (4-44) When the hyperspectral data recording unit 101d receives the request to stop transmitting hyperspectral data, it stops transmitting the hyperspectral data to the rendering unit 102d (S475).

[0250] Fifth Embodiment Next, a fifth embodiment will be described. In this fifth embodiment, the conversion function is managed by a separate device.

[0251] 13 is a diagram showing an application example of a distribution system according to the fifth embodiment of the present invention. As shown in Fig. 13, the distribution system according to the fifth embodiment includes a hyperspectral image distribution server device 100e, a hyperspectral image distribution client device 200e, a hyperspectral camera 300e, and a conversion function storage device 400.

[0252] The hyperspectral image distribution server device 100e includes a hyperspectral data recording unit 101e, a rendering unit 102e, a rendering image recording unit 103e, a session management unit 104e, a conversion function ID receiving unit 105e, and an RGB image transmitting unit 106e.

[0253] The hyperspectral image distribution client device 200c includes a conversion function specifying unit 201e, a conversion function ID transmitting unit 202e, an RGB image receiving unit 203e, an RGB image drawing unit 204e, a screen unit 205e, and a communication path control unit 206e. The conversion function storage device 400 includes a conversion function storage unit 401.

[0254] Next, a description will be given of the processing flow of a system in which data captured by a hyperspectral camera 300e is visualized based on a conversion function determined by a hyperspectral image distribution client device 200e, the conversion function is stored and managed in a conversion function storage device 400, and the data can be push-distributed from a hyperspectral image distribution server device 100e and displayed on the hyperspectral image distribution client device 200e. Figures 14A and 14B are diagrams showing an example of the procedure of a first processing operation of a distribution system according to a fifth embodiment of the present invention. Here, an embodiment based on the third embodiment will be described, but this embodiment can also be configured based on the first, second, or fourth embodiment.

[0255] (5-1) The hyperspectral camera 300e, like the hyperspectral camera 300a, is a camera that captures images by separating light into wavelengths.

[0256] (5-2) The hyperspectral image distribution server device 100e is a computer having a CPU and memory, similar to the hyperspectral image distribution server device 100a. The hyperspectral camera 300e transmits hyperspectral data acquired by image capture to the hyperspectral data recording unit 101e of the hyperspectral image distribution server device 100e (S501). The hyperspectral data recording unit 101e receives the hyperspectral data transmitted from the hyperspectral camera 300e, generates a hyperspectral data file (S502), and stores the file in association with the image capture time (S503).

[0257] The connection between the hyperspectral camera 300e and the hyperspectral image distribution server device 100e can be established using existing technology.

[0258] The hyperspectral camera 300e and the hyperspectral image distribution server device 100e are connected by, for example, an Ethernet cable, and data captured by the hyperspectral camera 300e is transferred from the hyperspectral camera 300e to the server device 100e via communication in accordance with the GigE Vision standard, where it is stored. The hyperspectral data recording unit 101e stores the hyperspectral data in association with the capture time.

[0259] (5-3) The hyperspectral image distribution client device 200e and the conversion function storage device 400 are computers having a CPU and a memory. The hyperspectral image distribution client device 200e, the hyperspectral image distribution server device 100e, and the conversion function storage device 400 are connected to each other.

[0260] The hyperspectral image distribution client device 200e, the hyperspectral image distribution server device 100e, and the conversion function storage device 400 are connected, for example, via a network and communicate with each other using IP.

[0261] In this embodiment, it is assumed that one hyperspectral image distribution server device 100e, one transformation function storage device 400, and multiple hyperspectral image distribution client devices 200e are connected, but this is not limited to this.

[0262] (5-4-1) The hyperspectral image distribution client device 200e includes a communication path control unit 206e, which holds a client ID as an identifier of the hyperspectral image distribution client device 200e itself.

[0263] In the communication path control unit 206e, the operator of the hyperspectral image distribution client device 200e can request the establishment of a communication path with the hyperspectral image distribution server device 100e by operating the communication path control unit 206e.

[0264] When the communication path control unit 206e receives a request from the operator to establish a communication path with the hyperspectral image distribution server device 100e, it sends the client ID held above and a request to establish a communication path to the RGB image receiving unit 203e (S511).

[0265] (5-4-2) The client ID is used to identify the hyperspectral image distribution client device 200 e. As long as the client ID is unique enough to identify the hyperspectral image distribution client device 200 e, the client ID may change with each transmission from the same client device 200 e, or may remain unchanged.

[0266] The client ID includes (1) destination information of the hyperspectral image distribution client device 200e, or (2) information that enables the session management unit 104e of the hyperspectral image distribution server device 100e to extract source information as destination information.

[0267] (5-4-3) The RGB image receiving unit 203e transmits the client ID and a request for establishing a communication path to the RGB image transmitting unit 106e of the hyperspectral image distribution server device 100e (S512).

[0268] (5-4-4) The RGB image transmission unit 106e accepts the request for establishing a communication path and establishes a communication path capable of two-way communication (S513). When accepting the request for establishing a communication path, the RGB image transmission unit 106e may perform a process of inquiring whether the client ID is valid, and may accept the request for establishing a communication path only if the client ID is valid.

[0269] (5-4-5) When the communication path is established, the RGB image transmission unit 106e passes the client ID and a notification that the communication path has been established to the session management unit 104e (S514). (5-4-6) The session management unit 104e stores the client ID as identification information for the client with whom the communication path is currently established. (5-4-7) The hyperspectral image distribution client device 200e has a conversion function designation unit 201e. The conversion function designation unit 201e allows the operator of the hyperspectral image distribution client device 200e to designate a conversion function to be used for rendering.

[0270] (5-4-8) Methods for specifying the conversion function used in rendering include, but are not limited to, (1) to (6) described in the second embodiment. In addition, it is also possible to prepare default values ​​for the conversion function to be used when no conversion function is specified.

[0271] (5-4-9) The conversion function designation unit 201e can also store information in order to prepare multiple functions in advance. In order to prepare multiple functions in advance, the conversion function designation unit 201e can also acquire information from the conversion function storage unit 401 of the conversion function storage device 400. In this case, the conversion function designation unit 201e stores the conversion function identifiers (sometimes referred to as conversion function identifiers or conversion function IDs) of the functions to be prepared in advance. The conversion function designation unit 201e transmits the conversion function identifiers and a conversion function acquisition request to the conversion function storage unit 401 of the conversion function storage device 400 (S521).

[0272] (5-4-10) The conversion function storage unit 401 of the conversion function storage device 400 receives the conversion function identifier and the conversion function acquisition request from the conversion function designation unit 201e of the hyperspectral image distribution client device 200e.

[0273] The conversion function storage unit 401 extracts a conversion function using the conversion function identifier. The conversion function storage unit 401 of the conversion function storage device 400 associates the extracted conversion function with the conversion function acquisition response and transmits them to the conversion function designation unit 201e of the hyperspectral image distribution client device 200e (S522).

[0274] (5-4-11) The conversion function designation unit 201e transmits the designated conversion function to the conversion function storage unit 401 based on the method shown in (5-4-8) and stores it (S523). Alternatively, the conversion function designation unit 201e transmits the designated conversion function identifier to the conversion function storage unit 401 based on the method shown in (5-4-8). Alternatively, the conversion function designation unit 201e associates the designated conversion function with the designated conversion function identifier and transmits them to the conversion function storage unit 401 based on the method shown in (5-4-8). Alternatively, the conversion function designation unit 201e transmits the designated base conversion function, or the conversion function, the designated conversion function identifier, and any change differences, etc., to the conversion function storage unit 401 based on the method shown in (5-4-8).

[0275] (5-4-12) The conversion function storage unit 401 of the conversion function storage device 400 receives the information transmitted in (5-4-11) from the conversion function designation unit 201e of the hyperspectral image distribution client device 200e.

[0276] When only a conversion function is received, the conversion function storage unit 401 saves the conversion function, generates a conversion function identifier for the conversion function, and stores the conversion function in association with the conversion function identifier. When only a conversion function identifier is received, the conversion function storage unit 401 saves the conversion function identifier. When a conversion function and its identifier are received, the conversion function storage unit 401 saves the conversion function and the conversion function identifier in association with the conversion function. When a specified base conversion function, its identifier, and a change difference are received, the conversion function storage unit 401 changes the conversion function in accordance with the change difference, and saves the changed conversion function in association with the conversion function identifier. The conversion function storage unit 401 transmits the saved conversion function identifier to the conversion function designation unit 201e of the hyperspectral image distribution client device 200e (S524).

[0277] (5-4-13) The conversion function specifying unit 201 e of the hyperspectral image distribution client device 200 e receives the conversion function identifier from the conversion function storage unit 401 of the conversion function storage device 400 .

[0278] The conversion function designation unit 201e passes the conversion function identifier to the conversion function ID transmission unit 202e (S525).

[0279] (5-5) The conversion function ID transmission unit 202e receives the conversion function identifier. The conversion function ID transmission unit 202e reads the stored client ID. The conversion function ID transmission unit 202e associates the received conversion function identifier with the client ID and transmits them to the conversion function ID reception unit 105e of the hyperspectral image distribution server device 100e (S526).

[0280] (5-6) The conversion function ID receiving unit 105e of the hyperspectral image distribution server device 100e receives the conversion function identifier and the client ID from the conversion function ID sending unit 202e. The conversion function ID receiving unit 105e associates the received conversion function identifier with the client ID and passes them to the session management unit 104e.

[0281] (5-7) The session management unit 104e receives the conversion function identifier and the client ID from the conversion function ID transmission unit 202e. The session management unit 104e associates the received conversion function identifier and the client ID and stores them as session information (S527). The session management unit 104e passes a rendering image acquisition request including the received conversion function identifier to the rendering image recording unit 103e (S528).

[0282] (5-8) The rendering image recording unit 103e receives a rendering image acquisition request including a conversion function identifier from the session management unit 104e. The rendering image recording unit 103e checks whether a rendering image that has been rendered using the received conversion function identifier exists. If the rendering image recording unit 103e has recorded the latest RGB image using the conversion function, i.e., a rendering image, the rendering image recording unit 103e associates the RGB image with the capture time of the image and passes it to the RGB image transmission unit 106e.

[0283] If the rendering image recording unit 103e does not record the latest RGB image using the conversion function, the rendering image recording unit 103e passes the conversion function identifier to the rendering unit 102e as rendering information (S531).

[0284] (5-9) The rendering unit 102e receives the conversion function identifier from the rendering image recording unit 103e. The rendering unit 102e of the hyperspectral image distribution server device 100e associates the conversion function identifier with the conversion function identifier and transmits a conversion function acquisition request to the conversion function storage unit 401 of the conversion function storage device 400 (S532).

[0285] The conversion function storage unit 401 of the conversion function storage device 400 receives the conversion function identifier and the conversion function acquisition request from the rendering unit 102e of the hyperspectral image distribution server device 100e. The conversion function storage unit 401 extracts the corresponding conversion function from the received conversion function identifier.

[0286] The conversion function storage unit 401 transmits a conversion function acquisition response including the extracted conversion function to the rendering unit 102e of the hyperspectral image distribution server device 100e (S533). The rendering unit 102e of the hyperspectral image distribution server device 100e receives the conversion function from the conversion function storage unit 401 of the conversion function storage device 400.

[0287] The rendering unit 102e stores the received conversion function. When the rendering unit 102e receives hyperspectral data from the hyperspectral data recording unit 101e, the rendering unit 102e converts the hyperspectral data into data in the RGB space using the stored conversion function, thereby generating an RGB image.

[0288] On the other hand, if the rendering unit 102e has not received the hyperspectral data from the hyperspectral data recording unit 101e, the rendering unit 102e passes a hyperspectral data transmission request to the hyperspectral data recording unit 101e (S534).

[0289] (5-10) The hyperspectral data recording unit 101e receives the hyperspectral data transmission request from the rendering unit 102e. The hyperspectral data recording unit 101e associates the latest hyperspectral data with the capture time of this data and passes it to the rendering unit 102e (S541).

[0290] From then on, until a request to stop transmitting hyperspectral data is received from the session management unit 104e, the hyperspectral data recording unit 101e will link the latest hyperspectral data at that time to the time of shooting of this data and pass it to the rendering unit 102e when a hyperspectral data file is generated.

[0291] 15A and 15B are diagrams showing an example of the procedure of the second processing operation of the distribution system according to the fifth embodiment of the present invention. (5-11) The rendering unit 102e receives the hyperspectral data from the hyperspectral data recording unit 101e. The rendering unit 102e converts the hyperspectral data into data in the RGB space using a conversion function to generate an RGB image (S542).

[0292] (5-12) As in the first embodiment, when visualization is performed based on human vision, a color matching function is used as an example of a conversion function. (5-13) The rendering unit 102e associates the generated RGB image with the image's capture time and the conversion function identifier, and passes the associated RGB image to the rendering image recording unit 103e (S543). (5-14) The rendering image recording unit 103e receives the RGB image, the image's capture time, and the conversion function identifier from the rendering unit 102e, associates them, and stores them (S544). The rendering image recording unit 103e associates the RGB image with the image's capture time and the conversion function identifier, and passes the associated RGB image to the session management unit 104e (S545).

[0293] (5-15) The session management unit 104e receives the RGB image, the shooting time of the image, and the conversion function identifier from the rendering image recording unit 103e. The session management unit 104e reads out the client ID saved in (5-7) based on the received shooting time and conversion function identifier. If necessary, the session management unit 104e extracts sender information from the client ID. The following description will be given assuming that sender information is handled.

[0294] If necessary, the session management unit 104e reads out a conversion function identifier based on the received image capture time and the client ID saved in (5-7).The session management unit 104e associates the RGB image, image capture time, and conversion function identifier with the sender information as destination information, and passes them to the RGB image sending unit 106e (S546).

[0295] (5-16) The RGB image transmitting unit 106e receives the RGB image, the shooting time, the conversion function identifier, and the destination information from the session managing unit 104e. The RGB image transmitting unit 106e associates the RGB image, the shooting time, and the conversion function identifier, and transmits them to the RGB image receiving unit 203e of the hyperspectral image distribution client device 200e corresponding to the destination information (S547).

[0296] (5-17) The RGB image receiving unit 203e receives the RGB image, the shooting time, and the conversion function identifier from the RGB image sending unit 106e of the hyperspectral image distribution server device 100e. The RGB image receiving unit 203e associates the RGB image, the shooting time, and the conversion function identifier, and passes them to the RGB image drawing unit 204e (S548).

[0297] (5-18) The RGB image drawing unit 204e receives the RGB image, the shooting time, and the conversion function identifier from the RGB image receiving unit 203e. The RGB image drawing unit 204e passes at least the RGB image to the screen unit 205e (S549).

[0298] This passed data may include the time of shooting, the conversion function or conversion function information, and other elements.

[0299] (5-19) The screen unit 205e receives at least the RGB image, which is the data passed from the RGB image drawing unit 204e, and displays at least the RGB image, which is the received data, on the screen.

[0300] (5-20) In this embodiment, the display destination is the screen unit 205e, but it is also possible that the display destination is another device to which the output of the hyperspectral image distribution client device 200e is connected. In this case, the RGB image rendering unit 204e passes at least the received RGB image to a screen device, which is another device connected to the hyperspectral image distribution client device 200e, and displays it on the screen.

[0301] (5-21) It is also possible that the output method of the hyperspectral image distribution client device 200e is to output an image file. In this case, the RGB image rendering unit 204e has a file output function. It is also possible that the output method of the hyperspectral image distribution client device 200e is to print. In this case, the RGB image rendering unit 204e has a printer output function.

[0302] (5-22) The following describes the process flow for stopping the image stream reception in the server device. (5-23) The communication path control unit 206e of the hyperspectral image distribution client device 200e sends a request to stop the stream reception to the RGB image receiving unit 203e (S551).

[0303] (5-24) When the RGB image receiving unit 203e receives a request to stop receiving the stream from the communication path control unit 206e, the RGB image receiving unit 203e releases the communication path established between the RGB image receiving unit 203e and the RGB image transmitting unit 106e of the hyperspectral image distribution server device 100e (S552).

[0304] (5-25) When the RGB image transmission unit 106e of the hyperspectral image distribution server device 100e detects that the communication path with the hyperspectral image distribution client device 200e has been released (S553), the RGB image transmission unit 106e passes a stop notification including the client ID of the hyperspectral image distribution client device 200e related to this communication path to the session management unit 104e (S554).

[0305] (5-26) The session management unit 104e extracts the conversion function identifier used by the client device associated with the client ID from the client ID included in the stop notification received from the RGB image transmission unit 106e. The session management unit 104e checks whether there is a client ID of a client device using the same conversion function identifier as the relevant conversion function identifier. (5-27) If there is a client ID of a client device using the same conversion function identifier, the process related to the request to stop receiving the stream is terminated. (5-28) If there is no client ID of a client device using the same conversion function identifier, the session management unit 104e passes the extracted conversion function identifier and a request to stop rendering to the rendering image recording unit 103e (S561).

[0306] After the query processing, the session management unit 104e deletes the data storing the combination of client IDs received from the RGB image transmission unit 106e, and then sends a rendering stop notification together with the conversion function identifier to the conversion function storage unit 401 (S555).

[0307] When the conversion function storage unit 401 receives the rendering stop notification together with the conversion function identifier from the session management unit 104e, it deletes the received conversion function identifier and the information on the associated conversion function.

[0308] (5-29) When the rendering image recording unit 103e receives the conversion function identifier and the request to stop rendering, it passes the conversion function identifier and the request to stop rendering to the rendering unit 102e (S562). (5-30) When the rendering unit 102e receives the conversion function identifier and the request to stop rendering, it stops rendering using the conversion function identifier, i.e., the conversion from hyperspectral data to an RGB image (S563).

[0309] 16 is a diagram showing an example of the procedure of the third processing operation of the distribution system according to the fifth embodiment of the present invention. When the rendering unit 102e detects that all rendering using any conversion function identifier has stopped, it sends a request to stop transmitting hyperspectral data to the hyperspectral data recording unit 101e (S564).

[0310] (5-32) When the hyperspectral data recording unit 101e receives the request to stop transmitting hyperspectral data, it stops transmitting the hyperspectral data to the rendering unit 102e (S565).

[0311] (5-33) Of the above descriptions, the contents from (5-4-7) to (5-9) show the flow of processing that operates in response to the setting of a conversion function. (5-34) The contents from (5-10) to (5-21) show the flow of processing that receives images, etc. from the hyperspectral image distribution server device 100e and draws the images in the hyperspectral image distribution client device 200e, in response to the generation of a hyperspectral data file from the content captured by the hyperspectral camera 300e.

[0312] (5-35) In the processing in the order described above, before the hyperspectral image distribution client device 200e receives an image from the hyperspectral image distribution server device 100e, the hyperspectral image distribution client device 200e transmits information about the conversion function to the hyperspectral image distribution server device 100e.

[0313] (5-36) However, when a hyperspectral data file is generated, the hyperspectral image distribution client device 200e can receive an image or the like from the hyperspectral image distribution server device 100e and transmit information about the conversion function to the hyperspectral image distribution server device 100e while the hyperspectral image distribution client device 200e is performing the process of drawing the image.

[0314] (5-37) In this case, the process flow in which the hyperspectral image distribution client device 200e transmits information about the transformation function to the hyperspectral image distribution server device 100e is the same as that described in (5-4-7) to (5-9).

[0315] (5-38) In addition, when a hyperspectral data file is generated, an image or the like is received once from the hyperspectral image distribution server device 100e, and while the hyperspectral image distribution client device 200e is performing the process of drawing the image, the hyperspectral image distribution client device 200e can transmit information about the conversion function to the hyperspectral image distribution server device 100e multiple times.

[0316] Furthermore, in order to obtain an image of an arbitrary wavelength in the client device in each of the above embodiments, a functional unit for selecting a wavelength may be provided, and image data of the arbitrary wavelength selected from a server device connected to this client device may be transmitted together with image data of a nearby wavelength, thereby speeding up delivery.

[0317] In addition, depending on the weight of each element of the color matching function, which is an example of the conversion function, the amount of information in the converted image may be relatively large for elements with large weight values, and relatively small for elements with low weight values, thereby reducing the amount of processing by the server device.

[0318] Fig. 17 is a block diagram showing an example of the hardware configuration of a hyperspectral image distribution server device according to one embodiment of the present invention. In the example shown in Fig. 17, the hyperspectral image distribution server device 100a of the distribution system according to the first embodiment is configured, for example, by a server computer or a personal computer and has a hardware processor 611A such as a CPU. A program memory 611B, a data memory 612, an input / output interface 613, and a communication interface 614 are connected to this hardware processor 611A via a bus 615. The same applies to the hyperspectral image distribution client device 200a according to the first embodiment and each device, such as each server device and each client device, of the distribution system in each of the second and subsequent embodiments.

[0319] The communication interface 614 includes, for example, one or more wireless communication interface units, and enables transmission and reception of information to and from a communication network NW. As the wireless interface, for example, an interface that adopts a low-power wireless data communication standard such as a wireless LAN (Local Area Network) is used.

[0320] An input device 700 and an output device 800 used by a user or the like are connected to the input / output interface 613. The input / output interface 613 receives operation data input by a user or the like through the input device 700, such as a keyboard, touch panel, touchpad, or mouse, and outputs output data to the output device 800, which may include a display device using a liquid crystal display or an organic electroluminescence (EL) display, for display. The input device 700 and the output device 800 may be devices built into the hyperspectral image distribution server device 100a, or may be input devices and output devices of other information terminals that can communicate with the hyperspectral image distribution server device 100a via a network (NW).

[0321] The program memory 611B is a non-transitory tangible storage medium that is a combination of a non-volatile memory that can be written to and read from at any time, such as a hard disk drive (HDD) or a solid state drive (SSD), and a non-volatile memory such as a read only memory (ROM), and stores programs necessary to execute various control processes, etc., according to one embodiment.

[0322] The data memory 612 is a tangible storage medium that is, for example, a combination of the above-mentioned nonvolatile memory and a volatile memory such as RAM (Random Access Memory), and is used to store various data acquired and created during various processes performed by the hyperspectral image distribution server device 100a.

[0323] The hyperspectral image distribution server device 100a according to one embodiment of the present invention may be configured as an information processing device having software-based processing function units. Storage areas used as work memory or the like by each unit of the hyperspectral image distribution server device 100a may be configured using the data memory 612 shown in Fig. 17. However, these configured storage areas are not essential components of the hyperspectral image distribution server device 100a and may be areas provided in, for example, an external storage medium such as a USB (Universal Serial Bus) memory, or a storage device such as a database server located in the cloud.

[0324] The processing function unit can be realized by having the hardware processor 611A read and execute a program stored in the program memory 611B. However, the processing function unit may also be realized in various other forms, including an integrated circuit such as an application specific integrated circuit (ASIC) or a field-programmable gate array (FPGA).

[0325] The methods described in each embodiment may be stored as a program (software means) that can be executed by a computer on a recording medium such as a magnetic disk (e.g., a floppy disk, a hard disk, etc.), an optical disk (e.g., a CD-ROM, a DVD, an MO, etc.), or a semiconductor memory (e.g., a ROM, a RAM, a flash memory, etc.), or may be transmitted and distributed via a communication medium. The program stored on the medium also includes a configuration program that configures the software means (including not only executable programs but also tables and data structures) that the computer executes. The computer that realizes this device reads the program stored on the recording medium and, in some cases, configures the software means using the configuration program, and executes the above-described processing by having the operation controlled by this software means. The term "recording medium" as used herein is not limited to a storage medium for distribution, but also includes a storage medium such as a magnetic disk or semiconductor memory installed inside the computer or in a device connected via a network.

[0326] The present invention is not limited to the above-described embodiments, and various modifications can be made in the implementation stage without departing from the spirit of the invention. Furthermore, the embodiments may be implemented in appropriate combinations, in which case the combined effects can be obtained. Furthermore, the above-described embodiments include various inventions, and various inventions can be extracted by combining selected elements from the disclosed elements. For example, if the problem can be solved and the desired effect can be obtained even if some elements are deleted from all elements shown in the embodiments, the configuration from which these elements are deleted can be extracted as an invention.

[0327] DESCRIPTION OF SYMBOLS 100a, 100b, 100c, 100e... Hyperspectral image distribution server device 100d... Hyperspectral video distribution server device 101a, 101b, 101c, 101d, 101e... Hyperspectral data recording unit 102a, 102b, 102c, 102d, 102e... Rendering unit 103a, 103b, 103c, 103e... Rendered image recording unit 103d... Video accumulation unit 104a, 104b, 104c, 104d, 104e... Session management unit 105a... Viewing image request receiving unit 105b, 105c... Image / conversion function information receiving unit 105d... Conversion function information receiving unit 105e... Conversion function ID receiving unit 106a, 106b, 106c, 106e... RGB image transmitting unit 106d...RGB image transmission unit 200a, 200b, 200c, 200e...Hyperspectral image distribution client device 200d...Hyperspectral image distribution client device 201a...Viewing image designation unit 201b, 201c...Image and conversion function designation unit 201d, 201e...Conversion function designation unit 202a...Viewing image request transmission unit 202b, 202c...Image and conversion function information transmission unit 202d...Conversion function information transmission unit 202e...Conversion function ID transmission unit 203a, 203b, 203c, 203e...RGB image receiving unit 203d...RGB image receiving unit 204a, 204b, 204c, 204e...RGB image drawing unit 204d...RGB image drawing unit 205a, 205b, 205c, 205d, 205e...screen unit 206c, 206e...communication path control unit 206d...image control unit 300a, 300b, 300c, 300d, 300e...hyperspectral camera 400...conversion function storage device 401...conversion function storage unit

Claims

1. A generation unit that generates an image in which a multi-dimensional captured image expressed using wavelength information recorded in a first recording device is converted into an image in the visible region, and records the generated image in a second recording device; and a transmission unit that transmits the converted image recorded in the second recording device to a device that is a delivery request source in response to a transmission request for the image. When the converted image corresponding to the conditions of the transmission request is not recorded in the second recording device, the generation unit generates an image in which the captured image recorded in the first recording device is visualized according to the conditions of the transmission request. The transmission unit transmits the generated image to the device that is the delivery request source. An image processing apparatus.

2. The image processing apparatus according to claim 1, further comprising a specifying unit that receives a specification of a function used for converting the captured image recorded in the first recording device into an image in the visible region. The generation unit generates an image in which the captured image is converted into an image in the visible region using the function specified by the specifying unit.

3. A method performed by an image processing apparatus, including: generating, by a generation unit of the image processing apparatus, an image in which a multi-dimensional captured image expressed using wavelength information recorded in a first recording device is converted into an image in the visible region, and recording the generated image in a second recording device; and transmitting, by a transmission unit of the image processing apparatus, the converted image recorded in the second recording device to a device that is a delivery request source in response to a transmission request for the image. When the converted image corresponding to the conditions of the transmission request is not recorded in the second recording device, the generation unit generates an image in which the captured image recorded in the first recording device is visualized according to the conditions of the transmission request. The transmission unit transmits the generated image to the device that is the delivery request source. An image processing method.

4. An image processing program that causes a processor to function as each unit of the image processing apparatus according to claim 1 or 2.

Images