Remote standard image generation system, remote standard image generation method and program

Abstract

To provide a remote standard image generation system which generates an image with information on a color excluding the influence of a light source environment from an observation image (affected portion image) being a photographed image of a subject (a patient, an affected portion of the patient or skin and the like of an advisee) imaged at an imaging place different from an observation place where an observer (a doctor or beauty advisor) performs observation, integrates the characteristics of visual recognition of colors of the observation image, and improves accuracy of inspection in remote medical care and on-line counseling.SOLUTION: A remote standard image generation system for generating a remote standard image including information about the spectral reflectance of a subject as an image used for observation of a subject by an observer from an observation image of the subject captured at an imaging place different from an observation place where the observer performs observation comprises: a subject spectral reflectance estimation unit which estimates the spectral reflectance of the subject from at least the observation image, the spectral distribution of a light source in an environment of the imaging place and the spectral sensitivity of the imaging device that captures the observation image; and a standardized image generation unit which generates a remote standard image including the information about the estimated spectral reflectance.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a remote standard image generating system, a remote standard image generating method, and a program. [Background technology]

[0002] In recent years, there has been an increasing demand for remote medical consultations (online medical consultations) and online cosmetic counseling. For example, the main method of treatment for general outpatients is for the patient to visit a medical institution on a regular basis, undergo physical examinations using the medical and health equipment available at the institution, and have a doctor or other specialist assess the patient's condition based on the results of the examinations. In recent years, remote medical treatment in remote islands and rural areas has been increasing due to a shortage of doctors. Since patients cannot easily visit medical institutions in person, doctors at medical institutions examine patients based on images and videos taken of the patient (see, for example, Patent Documents 1 and 2).

[0003] Similarly, the same is true for patients receiving care at home, for whom the doctors are unable to visit frequently, and so the remote medical care mentioned above is being carried out. In addition, in the case of sudden illness or injury due to an accident, emergency transportation is required, but since every second counts, patients can be transported to the hospital at home, at the accident scene, or in the ambulance, and remote medical treatment similar to that described above is performed. If patients can be examined at home, at the scene of an accident, or during transport, it will be possible to anticipate tests to be performed after admission, which will help with early treatment tailored to the patient's condition. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 3417235 [Patent Document 2] Patent No. 6001245 Summary of the Invention [Problem to be solved by the invention]

[0005] However, whether an image is captured with a normal digital camera or a hyperspectral camera, it is not possible to observe the accurate color of the affected area of ​​a patient due to ambient light such as lighting. For example, doctors usually examine patients in examination rooms at medical institutions, so if the environment is different, the color of the affected area may not match the color observed in the medical institution, and there is a risk that an accurate examination may not be possible.

[0006] Furthermore, even if an image is captured using an expensive hyperspectral camera, the color of the patient's affected area will be affected by the above-mentioned ambient light, just as with a digital camera. For this reason, even if it is possible to extract the color for each wavelength with a high resolution corresponding to the performance, the accuracy of the analysis of the condition from the extracted color will be low due to the influence of ambient light, and the accuracy of the visual inspection cannot be improved. Furthermore, since electronic medical records are essential patient data for remote treatment, the color data of the affected area on the electronic medical record needs to be consistent with specified characteristics, even if the imaging device that captures the image of the affected area is different, or even if the display device that displays the image is different.

[0007] The present invention has been made in consideration of the above-mentioned circumstances, and provides a remote standard image generation system, a remote standard image generation method, and a program that generate an image based on color information that removes the influence of the light source environment from an observation image (affected area image) that is an image of a subject (a patient, or the patient's affected area, or the client's skin, etc.) captured at an imaging location different from the observation location where an observer (doctor or beauty advisor) observes, thereby unifying the visual characteristics of the color of the observation image and improving the accuracy of visual examinations in remote medical care and online counseling. [Means for solving the problem]

[0008] In order to solve the above-mentioned problems, the remote standard image generating system of the present invention comprises: From an observation image (image of affected area) which is an image of a subject (such as a patient, a patient's affected area, or a client's skin) captured at an imaging location different from the observation location observed by an observer (doctor or beauty advisor), A remote standard image generating system for generating a remote standard image including information on a spectral reflectance of the subject as an image used by the observer to observe the subject, comprising: an imaging environment estimation unit that estimates an imaging location where the observation image was captured; an environment light source spectral distribution acquisition unit that acquires a spectral distribution of a light source corresponding to the environment of the image capture location estimated by the image capture environment estimation unit; At least the observed image; The imaging environment estimation unit estimates The environment of the shooting location In response, the environment light source spectral distribution acquisition unit acquires a subject spectral reflectance estimating unit that estimates a spectral reflectance of the subject from a spectral distribution of a light source and a spectral sensitivity of an imaging device that captured the observation image; a standardized image generating unit for generating the remote standard image including information on the estimated spectral reflectance; The present invention is characterized by comprising:

[0009] The remote standard image generating system of the present invention is characterized in that the subject spectral reflectance estimation unit estimates the spectral reflectance of the subject consisting of the number of color channels of the remote standard image from the observed image having a number of color channels fewer than the number of color channels of the remote standard image.

[0011] The remote standard image generating system of the present invention comprises: The above The imaging environment estimation unit uses position information acquired by a GPS (global positioning system) receiver carried by a person who takes an image of the subject at the imaging location, and estimates the imaging location based on the position indicated by the position information in the map data. Whether it is indoors or outdoors Estimate If the image capturing location is outdoors, the environment light source spectral distribution acquisition unit acquires the Weather at the shooting location The spectral distribution of the light source corresponding to The present invention is characterized by obtaining: In the remote standard image generating system of the present invention, the imaging environment estimation unit estimates whether the imaging location is inside a vehicle or not from a moving speed based on a time-series change in the position information; The environmental light source spectral distribution acquisition unit acquires a spectral distribution of a light source corresponding to whether the image capturing location is inside a vehicle or not.

[0012] The remote standard image generating system of the present invention is characterized in that the imaging environment estimating section classifies the type of light source at the imaging location based on the color tone (hue) of the imaging location.

[0013] The remote standard image generating system of the present invention is characterized in that the imaging environment estimation section estimates the imaging location where the observational image was captured from a background image of the observational image.

[0014] The remote standard image generating system of the present invention is characterized in that, when estimating and acquiring the spectral distribution of the light source, the spectral distribution is judged according to predetermined conditions, and if it is judged that the estimation accuracy is poor, it is corrected using a reference spectral distribution, which is a past spectral distribution corresponding to the spectral distribution.

[0015] The remote standard image generating system of the present invention is characterized in that, when there is a spectrum of a wavelength that should not be included in the estimated spectral distribution, the data correction unit corrects the spectrum of that wavelength.

[0016] The remote standard image generating system of the present invention is characterized in that, when a spectrum of a wavelength that should be included in the estimated spectral sensitivity does not exist, the data correction unit corrects the spectrum of that wavelength.

[0017] The remote standard image generating system of the present invention is characterized in that the data correction unit compares each spectrum at each wavelength of the estimated spectral distribution with each spectrum at each wavelength of a reference spectral distribution that is similar to the estimated spectral distribution, and corrects any spectrum that has a predetermined difference or more with respect to the reference spectral distribution.

[0018] The remote standard image generating method of the present invention is a method for generating a remote standard image including information on a spectral reflectance of a subject from an observed image, which is an image of the subject captured at an imaging location different from an observation location where an observer observes the subject, as an image used by the observer to observe the subject, comprising: an imaging environment estimating step in which an imaging environment estimating unit estimates an imaging location where the observation image was captured; and an environment light source spectral distribution acquiring step in which an environment light source spectral distribution acquiring unit acquires a spectral distribution of a light source corresponding to the environment of the imaging location estimated by the imaging environment estimating unit; a subject spectral reflectance estimating unit that estimates at least the observed image, The imaging environment estimation unit estimates The environment of the shooting location In response, the environment light source spectral distribution acquisition unit acquiresThe method includes a subject spectral reflectance estimation process for estimating the spectral reflectance of the subject from the spectral distribution of a light source and the spectral sensitivity of an imaging device that captured the observed image, and a standardized image generation process in which a standardized image generation unit generates the remote standard image including information on the estimated spectral reflectance.

[0019] The program of the present invention is a program that causes a computer to function as a remote standard image generating system that generates, from a subject image that is an image of a subject captured at an imaging location different from an observation location where an observer observes the subject, a remote standard image including information on a spectral reflectance of the subject as an image used by the observer to observe the subject, the program comprising: an imaging environment estimation means for estimating an imaging location where the observed image was captured; and an environment light source spectral distribution acquisition means for acquiring a spectral distribution of a light source corresponding to the environment of the imaging location estimated by the imaging environment estimation means; At least the observed image; The imaging environment estimation means estimates The environment of the shooting location In response, the environmental light source spectral distribution acquisition means acquires The program functions as a subject spectral reflectance estimation means for estimating the spectral reflectance of the subject from the spectral distribution of a light source and the spectral sensitivity of the imaging device that captured the observation image, and as a standardized image generation means for generating the remote standard image including information on the estimated spectral reflectance. Effect of the Invention

[0020] As described above, according to the present invention, by generating an image based on color information that removes the influence of the light source environment from an observation image (affected area image), which is an image of a subject (a patient, or the patient's affected area, or the client's skin, etc.) captured at an imaging location different from the observation location observed by an observer (doctor or beauty advisor), it is possible to provide a remote standard image generation system, a remote standard image generation method and a program that can unify the visual characteristics of the color of the observation image and improve the accuracy of visual examinations in remote medical care and online counseling. [Brief description of the drawings]

[0021] [Figure 1] 1 is a block diagram showing an example of the configuration of a remote standard image generation system according to a first embodiment of the present invention. [Diagram 2]13 is a conceptual diagram showing an example of a display of a remote standard image on the display screen of the display device of the display unit 109. FIG. [Diagram 3] 10 is a flowchart showing an example of the operation of a process for generating a standard image for remote medical use from an affected area image by the remote standard image generating system 100 of this embodiment. [Figure 4] 1 is a conceptual diagram showing a concept of a process for generating a standard image for remote medical use from an affected area image by the remote standard image generating system 100 of this embodiment. [Diagram 5] FIG. 11 is a block diagram showing an example of the configuration of a remote standard image generating system according to a second embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] <First embodiment> Hereinafter, a first embodiment of the present invention will be described with reference to the drawings. Fig. 1 is a block diagram showing an example of the configuration of a remote standard image generation system according to a first embodiment of the present invention. In this embodiment, a remote standard image generation system 100 that generates a standard image used in remote medical examinations will be described as an example. The remote standard image generation system 100 may be used for cosmetic counseling. The remote standard image generation system 100 includes an image data input unit 101, an observed image correction unit 102, an image capture device spectral sensitivity acquisition unit 103, an image capture location estimation unit 104, an environment light source spectral distribution acquisition unit 105, a subject spectral reflectance estimation unit 106, a standard image generation unit 107, an output image generation unit 108, a display unit 109, an image data output unit 110, an observed image data storage unit 111, an image capture device spectral sensitivity storage unit 112, an environment light source spectral distribution storage unit 113, a remote standard image storage unit 114, and an output image storage unit 115.

[0023] The image data input unit 101 inputs an affected area image (observed image) that is an image of an affected area captured by an imaging device, which is an external device, from the imaging device. Moreover, the image data input unit 101 writes the data into the observed image data storage unit 111 for storage.

[0024] The observation image correction unit 102 reads the data of the affected area image from the observation image data memory unit 111, performs trimming to remove unnecessary image areas that include surrounding people, such as family members, emergency personnel, and doctors, as well as instruments and other parts of the image other than the patient's affected area, and generates a corrected affected area image (corrected observation image) that contains only the area in which the patient's affected area (subject) is captured. Moreover, the observation image correction section 102 writes and stores the corrected affected area image in the observation image data storage section 111 in association with the affected area image before trimming from which the corrected affected area image was generated.

[0025] The imaging device spectral sensitivity acquisition unit 103 searches for an imaging device in the imaging device spectral sensitivity storage unit 112, for example, using identification information that identifies the imaging device, such as the product number of the imaging device that captured the affected area image, which is added to the affected area image, as a search key. Then, when an imaging device corresponding to the search key is found, the imaging device spectral sensitivity acquisition unit 103 reads out data on the spectral sensitivity of this imaging device from the imaging device spectral sensitivity storage unit 112 . Furthermore, if the imaging device spectral sensitivity acquisition unit 103 is unable to find an imaging device corresponding to the search key in the imaging device spectral sensitivity storage unit 112, it searches for the spectral sensitivity of the imaging device via the Internet, thereby acquiring the spectral sensitivity from outside.

[0026] Furthermore, when the spectral sensitivity of the imaging device is unknown, the imaging device spectral sensitivity acquisition unit 103 estimates the spectral sensitivity of the imaging device using the relationship that the result of multiplying the spectral distribution, the spectral sensitivity, and the spectral reflectance becomes the color components R, G, and B of each pixel of the affected area image captured by the imaging device. For example, in this embodiment, the imaging device spectral sensitivity acquisition unit 103 estimates the spectral sensitivity of the imaging device by, for example, calculating a weighted sum of the spectral sensitivity basis vectors acquired in advance from a principal component analysis of the spectral sensitivities of a plurality of imaging devices (similar to the estimation of the light source spectral distribution and the spectral reflectance described later).

[0027] Then, the imaging device spectral sensitivity acquisition unit 103 obtains the color components R, G, and B of each pixel in the corrected affected area image, and estimates the spectral sensitivity of the imaging device that captured the corrected affected area image for each pixel based on the spectral reflectance of the sampled portion (the estimated spectral reflectance described later) and the spectral distribution of the light source in the imaged environment (the light source spectral distribution described later) (for example, see JP 2021-12514 A). Here, the imaging device spectral sensitivity acquisition unit 103 samples, for example, wavelength units from 400 nm to 700 nm in 10 nm increments in the wavelength of the estimated spectral distribution, and obtains the spectral sensitivity as a 31-dimensional vector.

[0028] The imaging location estimation unit 104 associates location information (latitude and longitude information) obtained by radio waves received by a GPS (global positioning system) receiver with latitude and longitude in map data, and estimates the type of location at that latitude and longitude. For example, if a building stands at the latitude and longitude in the map data, it is presumed that the location is inside the building, and if the latitude and longitude is a road, it is presumed that the location is outdoors on the road. This makes it possible to determine whether the image was captured indoors or outdoors, and if indoors, what type of facility it is. The image capture location estimation unit 104 then outputs the estimated image capture location to the environment light source spectral distribution acquisition unit 105 .

[0029] If it can be estimated that the image capture location is outdoors, the image capture location estimation unit 104 refers to a weather forecast website for the area at that latitude and longitude via the Internet, and acquires weather data (sunny, cloudy, rainy, snowy, etc.). Then, the image capture location estimation unit 104 outputs weather data of the weather forecast for the image capture location to the environment light source spectral distribution acquisition unit 105.

[0030] In addition, when the subject is moving using GPS, the image capture location estimation unit 104 can estimate the image capture location based on the speed, such as whether the subject is moving in an ambulance (inside the ambulance) or on a transport platform being pushed by a paramedic or the like (outdoors or inside a building). Then, the image capture location estimation unit 104 outputs to the environment light source spectral distribution acquisition unit 105 the location estimated based on the moving speed.

[0031] The imaging location estimating unit 104 also performs image recognition of background images other than the patient captured in the affected area image, such as indoors or outdoors. That is, the imaging location estimation unit 104 estimates (may use a machine learning model) the imaging location where the affected area was imaged by determining whether it was inside a home, inside an ambulance, outdoors (such as on the street), in a hospital hallway, an examination room, etc. Here, the imaging location estimation unit 104 estimates the type of light source at the imaging location from the color tone (hue) of the skin other than the affected area in the affected area image, and outputs the estimated type to the environment light source spectral distribution acquisition unit 105.

[0032] The environment light source spectral distribution acquisition unit 105 searches for a light source in the environment light source spectral distribution storage unit 113, for example, using identification information (such as the manufacturing number or product number of the light source) that is added to the supplied affected area image and identifies the light source of the environment in which the affected area image was captured as a search key. In addition, this identification information may be the product number and serial number of the light source that is added when the person who took the image confirms the light source and sends the image of the affected area captured by the imaging device, or the type of light source, such as an LED (light emitting diode), fluorescent lamp, or incandescent lamp.

[0033] Then, when a light source corresponding to the search key is found, the environment light source spectral distribution acquisition unit 105 reads out data on the spectral distribution of this light source from the environment light source spectral distribution storage unit 113 . In addition, in the case of light source types such as LED, fluorescent lamp, and incandescent lamp, the environment light source spectral distribution acquisition unit 105 reads out, from the environment light source spectral distribution storage unit 113, data on the spectral distribution of a light source type that is generally used in the image capturing location estimated by the image capturing location estimation unit 104, which is supplied from the image capturing location estimation unit 104.

[0034] Furthermore, even if the type of light source, such as LED, fluorescent lamp, or incandescent lamp, is not supplied from the person who captured the image, the environment light source spectral distribution acquisition unit 105 reads out data on the spectral distribution of a commonly used type of light source that corresponds to the image capture location estimated by the image capture location estimation unit 104 and is supplied from the image capture location estimation unit 104, from the environment light source spectral distribution storage unit 113. For this reason, a correspondence table between locations and general light sources at the locations is written and stored in advance in the environment light source spectral distribution storage unit 113. Furthermore, if the environment light source spectral distribution acquisition unit 105 is unable to find in the environment light source spectral distribution storage unit 113 the spectral distribution of a light source corresponding to the search key or of a light source commonly used in the imaging location (including the type of light source estimated by the imaging location estimation unit 104 from the color tone of the affected area image), the environment light source spectral distribution acquisition unit 105 externally acquires this spectral distribution by searching for the spectral distribution of the light source via the Internet.

[0035] Furthermore, when the environmental light source is unknown, the environmental light source spectral distribution acquisition unit 105 estimates the spectral distribution of the light source using the relationship where the result of multiplying the spectral distribution, the spectral sensitivity, and the spectral reflectance becomes the R, G, and B color components of the pixels of the affected area image captured by the imaging device, in the same manner as in the case of the spectral sensitivity. For example, in this embodiment, the environment light source spectral distribution acquisition unit 105 estimates the environment light source spectral distribution by a weighted sum of light source spectral distribution definition vectors acquired in advance from a principal component analysis of the spectral distributions of a plurality of light sources.

[0036] Then, the environmental light source spectral distribution acquisition unit 105 estimates the spectral distribution from which the color components R, G, and B of each pixel in the corrected affected area image are obtained, corresponding to each pixel, based on the spectral reflectance of the sampled portion (the estimated spectral reflectance described below) and the spectral sensitivity of the imaging device (see, for example, JP 2021-12514 A). Here, the environment light source spectral distribution acquisition unit 105 samples, for example, wavelength units from 400 nm to 700 nm in increments of 10 nm at the wavelength of the estimated spectral distribution, and obtains information on the spectral distribution as a 31-dimensional vector.

[0037] At this time, the environment light source spectral distribution acquisition unit 105 uses information about a general light source corresponding to the image capture location estimated by the image capture location estimation unit 104, thereby enabling the search range for estimating the wavelength spectrum when estimating the spectral distribution to be narrowed. That is, if the estimated location is indoors, the environment light source spectral distribution acquisition unit 105 can determine that the light source is, for example, a fluorescent lamp or an LED. The environment light source spectral distribution acquisition unit 105 can estimate the spectral distribution of the light source quickly and accurately by starting estimation from the spectrum for each wavelength of a light source such as a general fluorescent lamp or LED.

[0038] Furthermore, when the imaging location estimated by the imaging location estimation unit 104 is outdoors, the environment light source spectral distribution acquisition unit 105 acquires, via the Internet, the spectral distribution of sunlight corresponding to the weather data in the weather forecast (including information on the time of imaging), as an estimate of the light source.

[0039] The subject spectral reflectance estimation unit 106 estimates the spectral reflectance of the patient's affected area based on the corrected affected area image, the spectral sensitivity of the imaging device acquired by the imaging device spectral sensitivity acquisition unit 103, and the spectral distribution of the light source at the imaging location acquired by the environment light source spectral distribution acquisition unit 105. In this embodiment, the subject spectral reflectance estimating unit 106 estimates the spectral reflectance of the affected area by, for example, a weighted sum of a plurality of spectral reflectance basis vectors previously obtained by principal component analysis of the spectral reflectances of a plurality of affected areas.

[0040] Then, the subject spectral reflectance estimation unit 106 estimates the spectral reflectance from which the color components R, G, and B of each pixel in the corrected affected area image are obtained for each pixel (see, for example, JP 2021-12514 A). Here, the subject spectral reflectance estimating unit 106 samples, for example, wavelength units from 400 nm to 700 nm at 10 nm intervals at the wavelength of the estimated spectral reflectance, and obtains information on the spectral reflectance as a 31-dimensional vector.

[0041] As a result, when the imaging device is a digital camera, the color expressed in three dimensions (three channels, where a channel is a color channel) of color components R, G, and B can be increased in color dimension (channel) to obtain information on the spectral reflectance as a standard characteristic expressed in 31 dimensions (31 channels) as color components and not influenced by the light source at the time of imaging. That is, the subject spectral reflectance estimation unit 106 estimates the spectral reflectance (spectral reflectance information) of the affected area consisting of the number of color channels of the spectral image (remote standard image) from the affected area image (three channels of color components R, G, and B) with a number of color channels less than the number of spectral images (the number of color channels of the remote standard image).

[0042] The standard image generating unit 107 generates 31 spectral images including information on spectral reflectance, which show the spectrum in wavelength units of each of the 31 dimensions scaled as a gradient. Here, the standard image generation unit 107 generates 31 spectroscopic images so that each of the 31 dimensions becomes one spectroscopic image, and writes and stores this group of 31 spectroscopic images in the standard image memory unit 114 for remote medical care as a remote standard image.

[0043] When this remote target standard image generating system 100 is located in the examination room (observation location) of a doctor (observer) who is examining (observing), the output image generating unit 108 reads the remote standard image from the remote target image storage unit 114. Then, the output image generation unit 108 uses the read remote standard image and the spectral distribution of the light source in the environment of the examination room where the doctor is present to generate an image of the affected area (observation image) in the color observed in the examination room (display color represented by color components R, G, and B), and displays it on the display screen.

[0044] In addition, the output image generating unit 108 may be provided with a display form in which any one of the 31 dimensions, or each of multiple dimensions, is converted into a color represented by the color components R, G, and B (the spectral reflectance of the spectral image is converted into a color scaled color), and displayed on the display screen together with the wavelength of the dimension. In this case, since the spectral images correspond to each of the 31 dimensions of the spectral reflectance, by scaling the spectral images with the gradation of the display color represented by the above-mentioned color components R, G, and B, it is possible to observe the spectrum of the affected area image for each wavelength using each of the 31 spectral images. Therefore, a doctor can diagnose the condition of an affected area by observing a display image generated from a spectroscopic image and observing changes in the spectral intensity of a specific wavelength corresponding to the condition of the affected area.

[0045] In addition, if the remote standard image generating system 100 is not located in the doctor's examination room, but exists on a server such as a hospital management center, the output image generating unit 108 reads the remote standard image from the remote standard image storage unit 114. Then, the output image generation unit 108 uses the remote standard image and the spectral distribution of the light source in the environment of the examination room where the doctor is present to generate a display image of the affected area in accordance with the display characteristics of the display device in the terminal in the doctor's examination room as the color observed in the examination room (display color represented by color components R, G, and B), and displays the image on the display screen of the display device in the doctor's examination room.

[0046] The display unit 109 is, for example, a liquid crystal display or an organic EL (electroluminescent) display. When the remote standard image generating system 100 is accessed from an external device, the image data output unit 110 reads out the requested spectroscopic image of the patient from the remote standard image storage unit 114 and outputs it to the external device.

[0047] The affected area image data supplied from an external imaging device is written together with the patient's identification information and stored in the observed image data storage unit 111. The corrected affected area image, which has been trimmed and corrected by the observed image correction unit 102, is also written together with the affected area image data and the patient's identification information and stored in the observed image data storage unit 111. The imaging device spectral sensitivity storage unit 112 has written therein and stores in advance a pair of identification information of each imaging device and the spectral sensitivity of the imaging device indicated by the identification information.

[0048] In the environment light source spectral distribution storage unit 113, identification information of each light source and the spectral distribution of the light source (light source spectral distribution) indicated by the identification information are written and stored in advance as a pair. In the remote standard image storage unit 114, each of the spectroscopic images generated by the standard image generating unit 107 is written and stored together with the patient's identification information. In the output image storage unit 115, each of the output images (display images) generated by the output image generating unit 108 is written and stored together with the patient's identification information.

[0049] 2 is a conceptual diagram showing an example of a display of a remote standard image on the display screen of the display device of the display unit 109. The following processing in FIG. In FIG. 2, the "Image 1" selection field 201 is a field for selecting whether to display a displayed affected area image created from the estimated spectral reflectance after estimating the spectral reflectance from a captured affected area image, or a spectral image. A field 202 is a field for inputting the minimum wavelength of a spectral image to be displayed when a spectral image is selected in the selection field 201 .

[0050] A field 203 is a field for inputting the maximum wavelength of a spectral image to be displayed when a spectral image is selected in the selection field 201 . Here, if only the minimum wavelength is set in the field 202 and no setting is made in the field 203, a color-scaled image of the spectral image of the wavelengths in the field 202 is displayed. On the other hand, when a minimum wavelength and a maximum wavelength are set in each of columns 202 and 203, respectively, the integrated values ​​of the spectroscopic images of each wavelength from column 202 to column 203, i.e., the integrated values ​​of the spectra of each pixel, are color-scaled to be displayed.

[0051] Similarly, the "Image 2" selection field 204 is a field for selecting whether to display a displayed affected area image created from the estimated spectral reflectance after estimating the spectral reflectance from the captured affected area image, or a spectral image. A field 205 is a field for inputting the minimum wavelength of a spectral image to be displayed when the spectral image is selected in the selection field 204 . A field 206 is a field for inputting the maximum wavelength of a spectral image to be displayed when a spectral image is selected in the selection field 204 .

[0052] Here, if only the minimum wavelength is set in the field 205 and no setting is made in the field 206, a color-scaled image of the spectral image of the wavelengths in the field 205 is displayed. On the other hand, when a minimum wavelength and a maximum wavelength are set in each of columns 205 and 206, respectively, the integrated values ​​of the spectroscopic images of each wavelength from column 205 to column 206, that is, the integrated values ​​of the spectra of each pixel, are color-scaled to be displayed.

[0053] The "ID" column 207 is an input column for identification information that identifies the patient to be displayed, and is information that identifies the patient with the affected area, such as the patient card number or the patient number given in the case of emergency medical care. Furthermore, when multiple affected areas exist for the same patient, a sub-number of the affected area number may be added under the patient number. Each time the display switching button 208 is pressed with a mouse or the like, the image of the affected area of ​​the patient selected in field 207 is switched in sequence between the display image 210A selected in the selection field 201 and the display image 201B selected in the selection field 204, and displayed in the display frame 209.

[0054] As an example of the display in FIG. 2 described above, two display images are switched and displayed, but this is not limited to this, and display images 210A and 210B may be displayed in each of the two display frames. Also, in order to compare the histories of the same patient at different imaging dates and times, a configuration may be adopted in which corrected diseased area images or spectral images captured at two different imaging timings are displayed. In addition, an image of the affected area of ​​the patient being examined and an image of the affected area of ​​a different patient may be displayed side by side in two display frames for comparison, for example, to compare the patient being examined with a normal person, or to compare with a patient whose condition has already been confirmed by examination.

[0055] FIG. 3 is a flowchart showing an example of the operation of the process of generating a remote standard image from an affected area image by the remote standard image generating system 100 of this embodiment. FIG. 4 is a conceptual diagram showing the concept of a process for generating a remote standard image from an affected area image by the remote standard image generating system 100 of this embodiment.

[0056] Step S101: The image data input unit 101 writes and stores images of the affected area supplied from an external imaging device in the observed image data storage unit 111 . For example, as shown in a scene 501 (FIG. 4), when the imaging location is the patient's 200's home, the imaging device is a Web camera 222 provided on a personal computer 221 operated by the patient 200. At this time, the personal computer 221 outputs the image of the affected area captured by the Web camera 222 to the remote standard image generating system 100.

[0057] On the other hand, as shown in scene 501 (FIG. 4), when the imaging location is an emergency scene where the patient 200 is transported by the ambulance staff 223, the imaging device is a digital camera 225 carried by the ambulance staff 223 or installed in the ambulance. At this time, the ambulance crew member 223 outputs the image of the affected area taken by the digital camera 225 to the remote standard image generation system 100 via a portable terminal (not shown) that he or she carries.

[0058] Step S102: Every time an affected area image is written to the observed image data storage unit 111, the observed image correction unit 102 reads out the written affected area image. Then, the observation image correction unit 102 performs a trimming process on the read affected area image, generates a corrected affected area image from the affected area image, and writes the generated corrected affected area image in the observation image data storage unit 111 for storage.

[0059] Step S103: The imaging device spectral sensitivity acquisition unit 103 searches the imaging device spectral sensitivity storage unit 112, for example, by identification information of the imaging device used for imaging, which is added to the affected area image, and reads out the spectral sensitivity of the imaging device corresponding to the identification information. Then, the image capture device spectral sensitivity acquisition unit 103 outputs the read-out spectral sensitivity to the subject spectral reflectance estimation unit 106 .

[0060] Step S104: The environment light source spectral distribution acquisition unit 105 searches the environment light source spectral distribution storage unit 113 using, for example, identification information of the light source of the captured environment that is added to the affected area image, and reads out the spectral distribution of the light source corresponding to the identification information. Then, the environment light source spectral distribution acquisition unit 105 outputs the read-out spectral distribution to the subject spectral reflectance estimation unit 106 . At this time, if the identification information of the light source is not added, the environment light source spectral distribution acquisition unit 105 may search and acquire the light source corresponding to the image capture location estimated by the image capture location estimation unit 104 from the environment light source spectral distribution storage unit 113 or the Internet.

[0061] Step S105: The subject spectral reflectance estimating unit 106 reads out the corrected affected area image corresponding to the identification information of the patient of the predetermined subject from the observed image data storage unit 111 . Then, the subject spectral reflectance estimation unit 106 estimates the spectral reflectance of each pixel in the corrected affected area image from the read corrected affected area image, the spectral sensitivity supplied from the imaging device spectral sensitivity acquisition unit 103, and the spectral distribution supplied from the environment light source spectral distribution acquisition unit 105 (spectral reflectance 503 in FIG. 4).

[0062] Step S106: The standard image generation unit 107 generates an image set consisting of 31 spectral images (spectral images 504 in FIG. 4) corresponding to multiple dimensions of wavelengths extracted at predetermined wavelength intervals, for example, 31 dimensions extracted every 10 nm, from the spectral reflectance estimated by the subject spectral reflectance estimation unit 106, as a remote standard image (image set 505 of spectral images in FIG. 4). Then, the standard image generating unit 107 writes and stores each of the generated spectroscopic images in the remote standard image storage unit 114 together with identification information for identifying the patient.

[0063] Step S107: The output image generating unit 108 generates an image in which the spectral intensity at each wavelength of the spectral image is scaled by the gradation of each of the R, G, and B color components as an output image. In addition, the output image generation unit 108 generates a color image of the primary colors of color components R, G, and B of the display unit 109 as an output image from the spectrum (spectral reflectance) of each wavelength of the spectral image, using the spectral distribution of the light source in the environment where the doctor or other person observes.

[0064] Step S108: Then, the output image generating unit 108 causes the display unit 109 to display the output image scaled with the gradation of each of the R, G, and B color components (FIG. 2). As a result, as shown in scene 506 in FIG. 4, the doctor 210 or the like can detect subtle conditions that are difficult to see in a color image of color components R, G, and B from the output image 211 displayed on the display unit 109, based on changes in the spectral intensity of wavelengths in 31 dimensions, thereby improving the accuracy of the examination.

[0065] Moreover, the output image generating unit 108 displays an output image based on the generated primary colors of the R, G, and B color components on the display unit 109 . This allows a doctor or the like to observe and examine the affected area of ​​a patient under the same conditions as under a light source in a normal examination room, using the output image displayed on the display unit 109.

[0066] As described above, according to this embodiment, from each of images of a patient's affected area (affected area images) taken in locations with different light source environments, color information that has been removed from the effects of the light source environment, i.e., the spectral intensity of the wavelength in the spectral reflectance, is scaled by the gradation of each of the color components R, G, and B to generate a spectral image (remote standard image). This unifies the characteristics when visually recognizing the color of the affected area image, thereby improving the accuracy of visual examination in remote medical care.

[0067] Furthermore, according to this embodiment, it is possible to observe the spectral intensity of 31-dimensional wavelengths in each spectroscopic image, and subtle color changes that are difficult to see in a color image in which the color components R, G, and B are combined can be detected from the changes in the spectral intensity of 31-dimensional wavelengths, thereby improving the accuracy of examination of the affected area. Furthermore, according to this embodiment, by using each of the spectral images (i.e., the spectral reflectance) and the spectral distribution of the light source in the examination room used by the doctor, it is possible to display the image of the affected area in the color state in which it will be observed in the examination room. This allows the doctor to observe the patient's affected area in the same way as during an examination in a normal environment, and shortens the examination time and enables prompt response without having to worry about color changes due to differences in the spectral distribution of the light source.

[0068] In the present embodiment, the imaging device spectral sensitivity acquisition unit 103, the environment light source spectral distribution acquisition unit 105, and the subject spectral reflectance estimation unit 106 estimate the spectral sensitivity, spectral distribution, and spectral reflectance, respectively. However, in a case where the spectral sensitivity is known and the spectral distribution and the spectral reflectance are to be estimated, the subject spectral reflectance estimation unit 106 may be configured to use the known spectral sensitivity to sequentially change the spectral distribution and the spectral reflectance to obtain estimates that provide the R, G, and B color components of each pixel of the corrected affected area image.

[0069] Similarly, in the case where the spectral distribution is known and the spectral sensitivity and the spectral reflectance are to be estimated, the subject spectral reflectance estimation unit 106 may be configured to use the known spectral distribution to sequentially change the spectral sensitivity and the spectral reflectance to obtain estimates that provide the R, G, and B color components of each pixel of the corrected affected area image. Furthermore, when the spectral sensitivity, the spectral distribution, and the spectral reflectance are unknown, the subject spectral reflectance estimation unit 106 may be configured to sequentially change the spectral sensitivity, the spectral distribution, and the spectral reflectance, respectively, to obtain estimated values ​​that provide the R, G, and B color components of each pixel of the corrected affected area image.

[0070] Furthermore, in the present embodiment, a digital camera has been described as an example of an imaging device, but a hyperspectral camera may also be used as the imaging device. In the case of a hyperspectral camera, spectral images of multiple wavelengths are obtained by sampling each of the multiple wavelengths. Therefore, unlike a digital camera, the spectral sensitivity of the imaging device is not acquired (estimated) in the case of a hyperspectral camera. The image of the affected area captured by the hyperspectral camera is information obtained by multiplying the spectral reflectance of the affected area by the spectral distribution in the imaging environment, and in order to estimate the spectral reflectance, it is necessary to acquire (estimate) the spectral distribution. The estimation of the spectral reflectance and the acquisition of the spectral distribution are similar to the estimation and acquisition methods already described.

[0071] In addition, each of the imaging device spectral sensitivity acquisition unit 103, the environment light source spectral distribution acquisition unit 105, and the subject spectral reflectance estimation unit 106 may be configured to estimate the spectral sensitivity, the spectral distribution, and the spectral reflectance using a machine learning model that estimates each of the spectral sensitivity, the spectral distribution, and the spectral reflectance from a corrected affected area image consisting of three color channels of color components R, G, and B of the digital camera.

[0072] <Second embodiment> A second embodiment of the present invention will now be described with reference to the drawings. FIG. 5 is a block diagram showing an example of the configuration of a remote standard image generating system according to the second embodiment of the present invention. The remote standard image generation system 100A includes an image data input unit 101, an observed image correction unit 102, an image capture device spectral sensitivity acquisition unit 103, an image capture location estimation unit 104, an environment light source spectral distribution acquisition unit 105, a subject spectral reflectance estimation unit 106, a standard image generation unit 107, an output image generation unit 108, a display unit 109, an image data output unit 110, an observed image data storage unit 111, an image capture device spectral sensitivity storage unit 112, an environment light source spectral distribution storage unit 113, a remote standard image storage unit 114, an output image storage unit 115, a data correction unit 116, and a past data history storage unit 117.

[0073] In the second embodiment, only the data correction unit 116 and the past data history storage unit 117 that are different from the first embodiment will be described below. The data correction unit 116 corrects the spectral intensity at each wavelength of the estimated spectral sensitivity, light source spectral distribution, and spectral reflectance. The past data history storage unit 117 stores the estimation results of the spectral intensity for each wavelength of the spectral sensitivity, the light source spectral distribution, and the spectral reflectance that have been estimated in the past.

[0074] Here, when correcting the estimated spectral sensitivity, the data correction unit 116 performs template matching with each of the spectral sensitivity stored in the imaging device spectral sensitivity storage unit 112, and extracts the most similar data of the spectral sensitivity as the reference spectral sensitivity. At this time, the data correction unit 116 may perform template matching between the estimated spectral sensitivity and the spectral sensitivity that has been estimated and corrected in the past and is stored in the past data history storage unit 117, and extract the spectral sensitivity in the past data history storage unit 117 that is similar to the estimated spectral sensitivity as the reference spectral sensitivity. Then, the data correcting unit 116 compares the spectral intensity for each wavelength between the reference spectral sensitivity and the estimated spectral sensitivity, and determines whether or not the difference between the spectral intensities exceeds a preset intensity difference.

[0075] At this time, when the data correction unit 116 detects a wavelength whose difference in spectral intensity exceeds the above-mentioned intensity difference, it multiplies the reference spectral sensitivity and the estimated spectral sensitivity by a weighting coefficient for the spectral intensity of each of them, and then adds them together, and calculates the average value (1 / 2) of the addition results. The data correction unit 116 performs correction to change the calculated average value to the spectral intensity of the estimated spectral sensitivity at the processed wavelength. The weighting coefficients by which the reference spectral sensitivity and the estimated spectral sensitivity are multiplied can be set arbitrarily.

[0076] Furthermore, as described above, a configuration may be adopted in which the spectral sensitivity of a general image capture device stored in the image capture device spectral sensitivity storage unit 112 is used as the reference spectral sensitivity without performing the process of extracting a reference spectral sensitivity by matching with a plurality of spectral sensitivities in the image capture device spectral sensitivity storage unit 112 and the past data history storage unit 117. Then, the data correction unit 116 determines whether there are wavelengths whose spectral intensities differ greatly in the reference spectral sensitivity, that is, whether there are wavelengths that are not included in a preset range of spectral intensities, and extracts them as wavelengths that should or should not be included.

[0077] In this case, the data correcting unit 116 multiplies the estimated spectral sensitivity and the reference spectral sensitivity by a weighting coefficient for the spectral intensity at this wavelength, then adds them together, and calculates the average value (1 / 2) of the addition results. The data correction unit 116 performs correction to change the calculated average value at the processed wavelength to the spectral intensity of the estimated spectral sensitivity.

[0078] Furthermore, when correcting the estimated spectral distribution, data correction unit 116 performs template matching with each of the spectral distributions stored in environment light source spectral distribution storage unit 113, and extracts the most similar data of the spectral distribution as a reference spectral distribution. At this time, data correction unit 116 may perform template matching between the estimated spectral distribution and a spectral distribution previously estimated and corrected that is stored in past data history storage unit 117, and extract the spectral distribution in past data history storage unit 117 that is similar to the estimated spectral distribution as the reference spectral distribution. Then, the data corrector 116 compares the spectral intensity for each wavelength between the reference spectral distribution and the estimated spectral distribution, and determines whether or not the difference between the spectral intensities exceeds a preset intensity difference.

[0079] At this time, when the data correction unit 116 detects a wavelength whose difference in spectral intensity exceeds the above-mentioned intensity difference, it multiplies the reference spectral distribution and the estimated spectral distribution by weighting coefficients for their respective spectral intensities, adds them together, and calculates the average value (1 / 2) of the addition results. The data correction unit 116 performs correction to change the calculated average value to the spectral intensity of the estimated spectral distribution at the processed wavelength. The weighting coefficients by which the reference spectral distribution and the estimated spectral distribution are multiplied can be set arbitrarily.

[0080] Furthermore, as described above, instead of performing the process of matching with a plurality of spectral distributions in the environment light source spectral distribution storage unit 113 and the past data history storage unit 117 to extract a reference spectral distribution, a configuration may be adopted in which the spectral distribution of a general light source stored in the environment light source spectral distribution storage unit 113 (the spectral distribution of a general light source for the type of light source estimated from the information on the imaging position) is used as the reference spectral distribution. In this case, the data correcting unit 116 multiplies the estimated spectral distribution and the reference spectral distribution by a weighting coefficient for the spectral intensity of each of them, then adds them together, and calculates the average value (1 / 2) of the addition results. The data correction unit 116 performs a correction to change the calculated average value at the processed wavelength to the spectral intensity of the estimated spectral distribution.

[0081] Furthermore, as described above, a configuration may be adopted in which the spectral distribution of a general light source at the imaging location stored in the environment light source spectral distribution storage unit 113 is used as the reference spectral distribution without performing the process of extracting a reference spectral distribution by matching with a plurality of spectral distributions in the environment light source spectral distribution storage unit 113 and the past data history storage unit 117. Then, the data correction unit 116 determines whether there are wavelengths in the reference spectral distribution that have significantly different spectral intensities, i.e., whether there are any wavelengths that are not included in a preset range of spectral intensities, and extracts them as wavelengths that should or should not be included.

[0082] In this case, the data corrector 116 multiplies the estimated spectral distribution and the reference spectral distribution by the weighting coefficients of the spectral intensity at this wavelength, then adds them together, and calculates the average value (1 / 2) of the addition results. The data correction unit 116 performs a correction to change the calculated average value at the processed wavelength to the spectral intensity of the estimated spectral distribution.

[0083] Furthermore, when correcting the estimated spectral reflectance, the data correction unit 116 performs template matching between each of the spectral images (spectral reflectance information) of the affected area images generated in the past and stored in the remote standard image storage unit 114 and each of the estimated spectral reflectances, and extracts an image set of spectral images having the most similar spectral reflectance information as a reference spectral reflectance. At this time, the data correction unit 116 may perform template matching between the estimated spectral reflectance and the spectral reflectances estimated and corrected in the past and stored in the past data history storage unit 117, and extract the spectral reflectance in the past data history storage unit 117 that is similar to the estimated spectral reflectance as the reference spectral reflectance. Then, the data correction unit 116 compares the spectral intensity for each wavelength between the reference spectral reflectance and the estimated spectral reflectance, and determines whether or not the difference between the spectral intensities exceeds a preset intensity difference.

[0084] At this time, when the data correction unit 116 detects a wavelength where the difference in spectral intensity exceeds the above-mentioned intensity difference, it multiplies the reference spectral reflectance and the estimated spectral reflectance by weighting coefficients for the spectral intensity, adds them together, and calculates the average value (1 / 2) of the addition results. The data correction unit 116 performs correction to change the calculated average value to the spectral intensity of the estimated spectral reflectance at the processed wavelength. The weighting coefficients by which the reference spectral reflectance and the estimated spectral reflectance are multiplied can be set arbitrarily.

[0085] Furthermore, as described above, instead of performing a matching process with a plurality of spectral reflectances in the remote standard image storage unit 114 and the past data history storage unit 117 to extract a reference spectral reflectance, the spectral reflectance of a similar affected area stored in the remote standard image storage unit 114 (the type of condition of the affected area is extracted based on the patient number, and the spectral reflectance of the affected area with the condition corresponding to that type) may be used as the reference spectral reflectance. In this case, the data correction unit 116 multiplies the estimated spectral reflectance and the reference spectral reflectance by the weighting coefficients of the spectral intensities, adds them together, and calculates the average value (1 / 2) of the addition results. The data correction unit 116 performs a correction to change the calculated average value to the spectral intensity of the estimated spectral reflectance at the processed wavelength.

[0086] Furthermore, as described above, a configuration may be adopted in which a general spectral reflectance of an affected area with a similar pathology stored in the remote standard image storage unit 114 is used as the reference spectral reflectance without performing a process of extracting a reference spectral reflectance by matching with a plurality of spectral reflectances in the remote standard image storage unit 114 and the past data history storage unit 117. Then, the data correction unit 116 determines whether there are wavelengths in the reference spectral reflectance that have significantly different spectral intensities, that is, whether there are wavelengths that are not included in a preset range of spectral intensities, and extracts them as wavelengths that should or should not be included.

[0087] In this case, the data correction unit 116 multiplies the estimated spectral reflectance and the reference spectral reflectance at this wavelength by the weighting coefficients of the spectral intensity, and then adds them together, and calculates the average value (1 / 2) of the addition results. The data correction unit 116 performs a correction to change the calculated average value to the spectral intensity of the estimated spectral reflectance at the processed wavelength.

[0088] As described above, according to this embodiment, similarly to the first embodiment, from each of images of a patient's affected area (affected area images) captured in locations with different light source environments, color information that has been removed from the effects of the light source environment, i.e., the spectral intensity of the wavelength in the spectral reflectance, is scaled by the gradation of each of the color components R, G, and B to generate a spectral image (remote standard image). This unifies the characteristics when visually recognizing the color of the affected area image, thereby improving the accuracy of visual examination in remote medical care.

[0089] Furthermore, according to this embodiment, the wavelengths and spectral intensities of the estimated spectral sensitivity, spectral distribution, and spectral reflectance are corrected based on the wavelengths and spectral intensities of previously estimated or general spectral sensitivity, spectral distribution, and spectral reflectance, respectively. This makes it possible to correct abnormal data in the estimation of the relationship between the estimated wavelength and spectral intensity to appropriate numerical values, and makes it possible to estimate the spectral sensitivity, spectral distribution, and spectral reflectance with high accuracy. A spectral image can be generated from the obtained spectral reflectance, and a doctor can accurately diagnose the condition of the affected area from the color state when observing the image of the affected area in the examination room, or from the observed spectral image.

[0090] Furthermore, according to this embodiment, it is possible to observe the spectral intensity of 31-dimensional wavelengths in each spectroscopic image, and subtle color changes that are difficult to see in a color image in which the color components R, G, and B are combined can be detected from the changes in the spectral intensity of 31-dimensional wavelengths, thereby improving the accuracy of examination of the affected area. Furthermore, according to this embodiment, by using each of the spectral images (i.e., the spectral reflectance) and the spectral distribution of the light source in the examination room used by the doctor, it is possible to display the image of the affected area in the color state in which it will be observed in the examination room. This allows the doctor to observe the patient's affected area in the same way as during an examination in a normal environment, and shortens the examination time and enables prompt response without having to worry about color changes due to differences in the spectral distribution of the light source.

[0091] As an application example of the remote standard image generation system according to the first and second embodiments, an online cosmetics counseling system will be described. In the online counseling system, the client consults with a BA (beauty advisor, i.e., observer) online about the selection and use of cosmetics suitable for the color and condition of the client's skin (subject). The online counseling system may have the same configuration as the remote standard image generation system described above, with the patient (subject) being replaced with the counselor, and the doctor (observer) being replaced with the BA. In addition, the past data history storage unit 117 may store information on the spectral reflectance extracted from images of bare skin or with makeup. The online counseling system, with the above-mentioned configuration, allows the BA to accurately grasp the client's skin color and condition.

[0092] In addition, a program for realizing the functions of the remote standard image generating system 100 of FIG. 1 and the remote standard image generating system 100A of FIG. 5 in the present invention may be recorded in a computer-readable recording medium, and the program recorded in the recording medium may be read into a computer system and executed to generate a remote standard image. The term "computer system" as used herein includes hardware such as an OS (Operating System) and peripheral devices. The term "computer system" also includes a WWW (World Wide Web) system equipped with a homepage providing environment (or display environment). The term "computer-readable recording medium" refers to portable media such as a flexible disk, a magneto-optical disk, a ROM (Read Only Memory), and a CD-ROM (Compact Disc - Read Only Memory), and storage devices such as a hard disk built into a computer system. The term "computer-readable recording medium" also refers to a storage device that holds a program for a certain period of time, such as a volatile memory (RAM (Random Access Memory)) inside a computer system that becomes a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line.

[0093] The above program may also be transmitted from a computer system in which the program is stored in a storage device or the like to another computer system via a transmission medium, or by transmission waves in the transmission medium. Here, the "transmission medium" that transmits the program refers to a medium that has the function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The above program may also be for realizing part of the above-mentioned functions. Furthermore, it may be a so-called difference file (difference program) that can realize the above-mentioned functions in combination with a program already recorded in the computer system.

[0094] Although an embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and design changes and the like that do not deviate from the gist of the present invention are also included. [Explanation of symbols]

[0095] 100,100A...Remote standard image generation system 101...Image data input unit 102...observation image correction unit 103...imaging device spectral sensitivity acquisition unit 104...Image capture location estimation unit 105…Environmental light source spectral distribution acquisition section 106...Object spectral reflectance estimation section 107...Standard image generating unit 108...Output image generating unit 109...Display section 110...Image data output unit 111...observation image data storage unit 112...imaging device spectral sensitivity memory unit 113…Environmental light source spectral distribution storage unit 114...Remote standard image storage unit 115...output image storage unit 116...Data correction section 117...Past data history storage section

Claims

1. 1. A remote standard image generating system for generating a remote standard image including information on a spectral reflectance of a subject from an observation image, the observation image being an image of the subject captured at an imaging location different from an observation location where an observer observes the subject, as an image used by the observer to observe the subject, comprising: an imaging environment estimation unit that estimates an imaging location where the observation image was captured; an environment light source spectral distribution acquisition unit that acquires a spectral distribution of a light source corresponding to the environment of the image capture location estimated by the image capture environment estimation unit; a subject spectral reflectance estimating unit that estimates a spectral reflectance of the subject from at least the observed image, a spectral distribution of a light source acquired by an environment light source spectral distribution acquiring unit in accordance with the environment of the imaging location estimated by the imaging environment estimating unit, and a spectral sensitivity of an imaging device that captured the observed image; a standardized image generating unit for generating the remote standard image including information on the estimated spectral reflectance; A remote standard image generating system comprising:

2. The subject spectral reflectance estimating unit estimates the spectral reflectance of the subject, which is composed of the number of color channels of the remote standard image, from the observation image having a number of color channels smaller than the number of color channels of the remote standard image.

2. The remote standard image generating system according to claim 1.

3. The imaging environment estimation unit estimates whether the imaging location is indoors or outdoors based on a position in map data indicated by the position information acquired by a global positioning system (GPS) receiver carried by a person who images the subject at the imaging location, If the imaging location is outdoors, the environmental light source spectral distribution acquisition unit acquires a spectral distribution of a light source corresponding to the weather at the imaging location based on a weather forecast for the area at the imaging location.

3. The remote standard image generating system according to claim 1 or 2.

4. The imaging environment estimation unit estimates whether the imaging location is inside a vehicle from a moving speed based on a time-series change in the position information, the environmental light source spectral distribution acquisition unit acquires a spectral distribution of a light source corresponding to whether the imaging location is inside a vehicle or not; 4. The remote standard image generating system according to claim 3.

5. The imaging environment estimation unit classifies the type of light source at the imaging location based on the color (tone) of the imaging location.

3. The remote standard image generating system according to claim 1 or 2.

6. The imaging environment estimation unit estimates the imaging location where the observation image was captured from a background image of the observation image.

3. The remote standard image generating system according to claim 1 or 2.

7. a data correction unit that, when estimating and acquiring the spectral distribution of the light source, judges the spectral distribution under a predetermined condition, and if it is judged that the estimation accuracy is poor, corrects the spectral distribution using a reference spectral distribution that is a past spectral distribution corresponding to the spectral distribution. Further equipped 7. The remote standard image generating system according to claim 1,

8. When a spectrum of a wavelength that should not be included in the estimated spectral distribution exists, the data correction unit corrects the spectrum of the wavelength.

8. The remote standard image generating system according to claim 7.

9. The data correction unit corrects the spectrum of a wavelength that should be included in the estimated spectral sensitivity when the spectrum of the wavelength does not exist.

9. The remote standard image generating system according to claim 7 or 8.

10. The data correction unit compares each spectrum at each wavelength of the estimated spectral distribution with a spectrum at each wavelength of a reference spectral distribution similar to the estimated spectral distribution, and corrects a spectrum having a difference equal to or greater than a predetermined difference with respect to the reference spectral distribution.

10. The remote standard image generating system according to claim 7, wherein the remote standard image generating system is a remote standard image generating system.

11. 1. A remote standard image generating method for generating a remote standard image including information on a spectral reflectance of a subject from an observation image, the remote standard image including information on a spectral reflectance of the subject, as an image used by an observer for observing the subject, the remote standard image generating method comprising the steps of: an imaging environment estimating step in which an imaging environment estimating unit estimates an imaging location where the observation image was captured; an environment light source spectral distribution acquisition step in which an environment light source spectral distribution acquisition unit acquires a spectral distribution of a light source corresponding to the environment of the image capture location estimated by the image capture environment estimation unit; a subject spectral reflectance estimating step in which a subject spectral reflectance estimating unit estimates a spectral reflectance of the subject from at least the observed image, a spectral distribution of a light source acquired by an environment light source spectral distribution acquiring unit in accordance with the environment of the imaging location estimated by the imaging environment estimating unit, and a spectral sensitivity of an imaging device that captured the observed image; a standardized image generating step in which a standardized image generating unit generates the remote standard image including information of the estimated spectral reflectance; A method for generating a remote standard image, comprising:

12. A program for causing a computer to function as a remote standard image generating system for generating a remote standard image including information on a spectral reflectance of a subject from an observation image, the observation image being an image of the subject captured at an imaging location different from an observation location where an observer observes the subject, as an image used by the observer to observe the subject, the program comprising: The computer, an imaging environment estimation means for estimating an imaging location where the observed image was captured; an environment light source spectral distribution acquisition means for acquiring a spectral distribution of a light source corresponding to the environment of the image capturing location estimated by the image capturing environment estimation means; a subject spectral reflectance estimation means for estimating a spectral reflectance of the subject from at least the observed image, a spectral distribution of a light source acquired by an environment light source spectral distribution acquisition means in accordance with the environment of the imaging location estimated by the imaging environment estimation means, and a spectral sensitivity of an imaging device that captured the observed image; a standardized image generating means for generating the remote standard image including information on the estimated spectral reflectance; A program to function as a

Images