Difference visualization device, difference visualization method, and difference visualization program

Abstract

A difference visualization device according to one embodiment is provided with: a variable calculation unit that calculates at least one coefficient which is used when first spectral intensity data of a first object and second spectral intensity data of a second object are each converted into color data in a first color system, and which maximizes the value of the color difference between the resulting first color data for the first spectral intensity data and the resulting second color data for the second spectral intensity data; and a difference visualization unit that performs the above conversion using the calculated at least one coefficient for each pixel of first data including a plurality of pixels each having spectral intensity data, and generates second data including a plurality of pixels having color data in which the difference between the first object and the second object is enhanced.

Description

Difference visualization device, difference visualization method, and difference visualization program

[0001] The embodiments relate to a difference visualization device, a difference visualization method, and a difference visualization program.

[0002] Unlike RGB cameras, which capture color information of a subject using three parameters (components) R, G, and B (red, green, and blue), hyperspectral cameras are known that can capture hyperspectral images and videos using dozens or even hundreds of parameters representing the spectral intensity of each wavelength of light. Hyperspectral cameras are used, for example, in factory production lines, to detect dissimilar, identically colored objects that are difficult for the human eye to distinguish and label them pixel by pixel using object recognition algorithms. Hyperspectral cameras are also used in tasks such as automatically removing one of dissimilar, identically colored objects. Note that dissimilarly colored objects refer to objects that are perceived as similar colors by humans but have different spectral spectra. Hereinafter, spectral spectra will simply be referred to as spectra.

[0003] Kunihito Kato, “Application of Hyperspectral Imaging to General Object Recognition,” Journal of the Japan Society for Precision Engineering, 2018, vol. 84, No. 12, 958-962.

[0004] On the other hand, unlike the above-described example in which the results of distinguishing between different types of objects of the same color are used as input for robot control, there are cases in which it is desired to highlight different types of objects of the same color for human viewing in the context of education, entertainment, etc. Furthermore, in such applications intended for human viewing, there are cases in which it is expected to display using a difference emphasis filter that naturally emphasizes the differences between objects A and B while retaining information such as the shading and contours of objects A and B, rather than performing a binary classification into different types of objects of the same color A and B, or a ternary classification into objects A, B, and other objects C.

[0005] Non-Patent Document 1 proposes a method for pixel-by-pixel classification of hyperspectral images and videos using a support vector machine (SVM) or regression deep learning to classify disparate, identically colored objects that are difficult to distinguish in RGB images. However, this classification method results in a visualization method that results in a binary classification of disparate, identically colored objects A and B, or a ternary classification of objects A, B, and another object C. This makes it difficult to naturally emphasize the differences between objects A and B while preserving information such as the shading and contours of objects A and B. Furthermore, with the above classification method, when object C is observed using a visualization program that can classify the differences between objects A and B, it is difficult to visualize whether object C has characteristics closer to object A (whether it appears to have a color similar to object A) or closer to object B (whether it appears to have a color similar to object B). In other words, there is room for improvement in the application of visualization to object C, which is different from objects A and B.

[0006] The present invention has been made in light of the above circumstances, and its purpose is to provide a means for making it possible to express the differences between two objects A and B of different types but the same color by natural color differences that retain shadows and contours. Furthermore, it provides a means for making it possible to observe whether an object C, which is different from objects A and B, appears to have a color closer to that of object A or object B.

[0007] In one embodiment, the difference visualization device includes: a calculation unit that calculates at least one coefficient to be used in the conversion such that, when first spectral intensity data of a first object and second spectral intensity data of a second object are converted into color data in a first color system, a color difference between first color data of the first spectral intensity data and second color data of the second spectral intensity data is maximized; and a difference visualization unit that performs the conversion using the at least one calculated coefficient for each pixel of the first data including a plurality of pixels each having spectral intensity data, to generate second data including a plurality of pixels having color data in which a difference between the first object and the second object is emphasized.

[0008] According to the embodiment, it is possible to provide a means for realizing the ability to express the difference between the two objects with natural color differences that retain shadows and contours. Furthermore, it is possible to provide a means for making it possible to observe whether an object C, which has spectral data different from those of objects A and B, appears to have a color closer to that of object A or object B.

[0009] FIG. 1 is a block diagram illustrating an example of a hardware configuration of a difference visualization device according to an embodiment. FIG. 2 is a block diagram illustrating an example of a functional configuration of a difference visualization device according to an embodiment. FIG. 3 is a diagram illustrating an example of a standard white light spectrum acquired by the difference visualization device according to an embodiment. FIG. 4 is a diagram illustrating an example of a standard white light spectrum acquired by the difference visualization device according to an embodiment. FIG. 5 is a diagram illustrating an example of a hyperspectral image including objects A and B acquired by the difference visualization device according to an embodiment. FIG. 6 is a diagram illustrating an example of spectral data of pixels in the hyperspectral image including objects A and B acquired by the difference visualization device according to an embodiment. FIG. 7 is a diagram illustrating an example of spectral data of pixels in the hyperspectral image including objects A and B acquired by the difference visualization device according to an embodiment. FIG. 8 is a diagram illustrating an example of information on maximum and minimum values ​​of variables stored by the difference visualization device according to an embodiment. FIG. 9 is a diagram illustrating an example of color matching function information stored by the difference visualization device according to an embodiment. FIG. 10 is a diagram illustrating an example of color matching function information stored by the difference visualization device according to an embodiment. FIG. 11 is a diagram illustrating an example of variable information stored by the difference visualization device according to an embodiment. FIG. 12 is a flowchart illustrating an example of a process for calculating difference emphasis filter variables using the difference visualization device according to an embodiment. FIG. 13 is a flowchart illustrating an example of a process for generating difference-enhanced data using the difference visualization device according to the embodiment.

[0010] Hereinafter, several embodiments will be described with reference to the drawings. In the following description, components having the same functions and configurations will be given the same reference numerals.

[0011] 1. Embodiment A difference visualization device according to an embodiment will be described.

[0012] 1.1 Hardware Configuration First, the hardware configuration of the difference visualization device 10 according to the embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing an example of the hardware configuration of the difference visualization device according to the embodiment.

[0013] The difference visualization device 10 includes a control circuit 11 , a storage 12 , and a user interface 13 .

[0014] The control circuit 11 is a circuit that controls the overall components of the difference visualization device 10. The control circuit 11 includes a CPU (Central Processing Unit), RAM (Random Access Memory), and ROM (Read Only Memory). The ROM of the control circuit 11 stores programs and the like used in various processes in the difference visualization device 10. The CPU of the control circuit 11 controls the entire difference visualization device 10 in accordance with the programs stored in the ROM of the control circuit 11. The RAM of the control circuit 11 is used as a work area for the CPU of the control circuit 11.

[0015] The storage 12 stores, for example, information used in various processes in the difference visualization device 10 .

[0016] The user interface 13 is an interface that manages communication between a user and the control circuit 11. The user interface 13 includes an input device and an output device. The output device includes, for example, a display.

[0017] 1.2 Functional Configuration The functional configuration of the difference visualization device 10 will be described with reference to Fig. 2. Fig. 2 is a block diagram showing an example of the functional configuration of the difference visualization device according to the embodiment.

[0018] The CPU of the control circuit 11 loads the program stored in the ROM of the control circuit 11 or the storage 12 into the RAM of the control circuit 11. The CPU of the control circuit 11 then interprets and executes the program loaded into the RAM of the control circuit 11. In this way, the functions of the difference emphasis filter variable calculation unit 21, the difference emphasis filter application unit 22, and the storage unit 23 are realized in the difference visualization device 10.

[0019] The storage unit 23 stores, for example, information 231 on maximum and minimum values ​​of variables, color matching function information 232, and variable information 233. The information 231 on maximum and minimum values ​​of variables is, for example, information for specifying the possible ranges of each of a plurality of difference emphasis filter variables (coefficients) for converting spectral data (spectral intensity data) of each pixel in the hyperspectral image or hyperspectral video to be observed into color data (data of a plurality of color values ​​or components) in a certain color system (color space). The certain color system is a color system used when calculating color data that visually emphasizes the difference between objects A and B of different types and the same color. It can also be said that the certain color system is a color system in which the calculated color data visually emphasizes the difference between objects A and B of different types and the same color. Specific examples of spectral data (spectral intensity data) will be described later; the spectral data is data related to spectral intensity. The difference emphasis filter variables include, for example, nine variables. More specifically, the difference emphasis filter variables include a scale coefficient a ｘ , a ｙ , and a ｚ , translation coefficient b ｘ , b ｙ , and b ｚ , and the magnification coefficient c ｘ , c ｙ , and c ｚ The variable maximum and minimum value information 231 includes, for example, maximum and minimum values ​​that each difference emphasis filter variable can have. The color matching function information 232 includes color matching functions that indicate the characteristics of the human eye. The color matching functions are, for example, functions related to the mixing ratio of components X, Y, and Z in the XYZ color system defined by the International Commission on Illumination (CIE) so that a color is perceived as being equivalent to the color of light of a single wavelength. The XYZ color system defined by the International Commission on Illumination (CIE) is known as a common color system that represents color by components X, Y, and Z. The variable information 233 is information related to a plurality of difference emphasis filter variables that are applied to convert spectral data into color data in a certain color system as described above.

[0020] In addition, the scale factor a ｘ , a ｙ , and aｚ , translation coefficient b ｘ , b ｙ , and b ｚ , and the magnification coefficient c ｘ , c ｙ , and c ｚ can be said to be variable parameters for modifying the color matching functions.

[0021] The difference emphasis filter variable calculation unit 21 acquires, from outside the device, a standard white light spectrum Is, a hyperspectral (HS) image IMGr including objects A and B, position information CA of pixels included in object A, and position information CB of pixels included in object B. Objects A and B are objects of different types but the same color, the difference of which is to be visually emphasized. The position information CA specifies the position of the pixel included in object A in the hyperspectral image IMGr. The position information CB specifies the position of the pixel included in object B in the hyperspectral image IMGr. The difference emphasis filter variable calculation unit 21 calculates the above-mentioned multiple difference emphasis filter variables to be applied to conversion into color data that visually emphasizes the difference between objects A and B, using, for example, the hyperspectral image IMGr acquired from outside the device, the position information CA and CB, information 231 on maximum and minimum values ​​of variables, and color matching function information 232. The difference emphasis filter variable calculation unit 21 also transmits the calculated multiple difference emphasis filter variables to the storage unit 23. As a result, the plurality of difference emphasis filter variables calculated so as to visually emphasize the difference between objects A and B are stored in the variable information 233. The difference emphasis filter variable calculation unit 21 is also simply referred to as a variable calculation unit.

[0022] The difference emphasis filter application unit 22 acquires a standard white light spectrum Is and a hyperspectral image or hyperspectral image IMGs of the observation target from outside the device. Note that, hereinafter, the hyperspectral image or hyperspectral image IMGs of the observation target will also be simply referred to as observation target data IMGs. The difference emphasis filter application unit 22 also acquires color matching function information 232 and variable information 233 from the storage unit 23. The difference emphasis filter application unit 22 applies a difference emphasis filter to the observation target data IMGs using the acquired standard white light spectrum Is, observation target data IMGs, color matching function information 232, and variable information 233 (performing processing to emphasize the difference between objects A and B). In this way, the difference emphasis filter application unit 22 generates a difference emphasis image or difference emphasis image IMGo in which the difference emphasis filter has been applied to the observation target data IMGs. Note that, hereinafter, the difference emphasis image or difference emphasis image IMGo will also be simply referred to as difference emphasis data IMGo. Furthermore, when the observation target data IMGs is a hyperspectral image, the difference emphasis data IMGo corresponds to each frame of the observation target data IMGs and includes frames to which a difference emphasis filter has been applied. The difference emphasis filter application unit 22 is also referred to as a difference visualization unit.

[0023] The difference emphasis filter application unit 22 also outputs the generated difference emphasis data IMGo to the outside, and displays it to the user on a display, for example.

[0024] 1.3 Standard White Light Spectrum and Hyperspectral Image Including Objects A and B Examples of a standard white light spectrum Is and a hyperspectral image IMGr including objects A and B will be described with reference to FIGS. 3 , 4 , 5 , 6 , and 7 . FIGS. 3 and 4 are diagrams showing an example of a standard white light spectrum acquired by the difference visualization device according to the embodiment. FIG. 5 is a diagram showing an example of a hyperspectral image including objects A and B acquired by the difference visualization device according to the embodiment. FIGS. 6 and 7 are diagrams showing an example of spectral data of pixels in the hyperspectral image including objects A and B acquired by the difference visualization device according to the embodiment.

[0025] First, the standard white light spectrum Is will be described with reference to Figures 3 and 4. As shown in Figure 3, the standard white light spectrum Is includes each wavelength λ (spectral component) and the spectral intensity Is(λ) corresponding to the wavelength λ. The standard white light spectrum Is can be expressed by a graph such as that shown in Figure 4. In this graph, the horizontal axis represents the wavelength λ, and the vertical axis represents the intensity Is(λ).

[0026] Next, a hyperspectral image IMGr including objects A and B will be described with reference to FIGS. 5 to 7 . As shown in FIG. 5 , the hyperspectral image IMGr is an image including objects A and B. The hyperspectral image IMGr is represented by, for example, a plurality of pixels arranged in a two-dimensional space represented by two coordinate components, similar to a normal image. Furthermore, each pixel of the hyperspectral image IMGr is associated with spectral data S as shown in FIGS. 6 and 7 . As shown in FIG. 6 , the spectral data S includes each wavelength λ and the intensity S(λ) of the spectrum corresponding to the wavelength λ. The spectral data S can be represented by a graph as shown in FIG. 7 . In the graph, the horizontal axis represents the wavelength λ, and the vertical axis represents the intensity S(λ).

[0027] Although not shown, when the observation target data IMGs acquired by the difference emphasis filter application unit 22 is a hyperspectral image, the observation target data IMGs has the same data structure as the hyperspectral image IMGr. Furthermore, when the observation target data IMGs is a hyperspectral video, each frame of the observation target data IMGs has the same data structure as the hyperspectral image IMGr.

[0028] 5 also illustrates position information CA included in object A and position information CB included in object B, which are acquired by the difference emphasis filter variable calculation unit 21. The position information CA is a component X of a two-dimensional coordinate that identifies the position of a pixel included in object A in the hyperspectral image IMGr. Ａ and Y Ａ The position information CB includes two-dimensional coordinate components X that identify the position of a pixel included in the object B in the hyperspectral image IMGr. Ｂand Y Ｂ Includes.

[0029] 1.4 Information Stored in the Storage Unit The information 231 on maximum and minimum values ​​of variables, color matching function information 232, and variable information 233 stored in the storage unit will be described with reference to FIGS. 8, 9, 10, and 11. FIG. 8 is a diagram showing an example of information on maximum and minimum values ​​of variables stored by the difference visualization device according to the embodiment. FIGS. 9 and 10 are diagrams showing an example of color matching function information stored by the difference visualization device according to the embodiment. FIG. 11 is a diagram showing an example of variable information stored by the difference visualization device according to the embodiment.

[0030] First, the information 231 on the maximum and minimum values ​​of variables stored in the storage unit 23 will be described with reference to FIG.

[0031] 8, the variable maximum and minimum value information 231 stores 18 values, including the maximum and minimum values ​​of each of the nine difference emphasis filter variables. The maximum and minimum values ​​of each difference emphasis filter variable are the maximum and minimum values ​​of the range of values ​​that the difference emphasis filter variable can have. In other words, when calculating a plurality of difference emphasis filter variables so as to visually emphasize the difference between objects A and B, the difference emphasis filter variable calculation unit 21 calculates, for each difference emphasis filter variable, a value that is less than the maximum value and greater than or equal to the minimum value of the difference emphasis filter variable. In FIG. 8, the scale factor a ｘ , a ｙ , and a ｚ , translation coefficient b ｘ , b ｙ , and b ｚ , and the magnification coefficient c ｘ , c ｙ , and c ｚ For each difference emphasis filter variable, the maximum and minimum values ​​are respectively added to the end of the difference emphasis filter variable. ＿ｍａｘ " and " ＿ｍｉｎ " is added.

[0032] Next, the color matching function information 232 stored in the storage unit 23 will be described with reference to FIGS. 9 and 10. FIG.

[0033] The color matching function information 232 includes color matching functions fx(λ), fy(λ), and fz(λ) for the components X, Y, and Z in the XYZ color system, respectively. As shown in FIG. 9, each color matching function includes a wavelength λ and a value of the color matching function corresponding to the wavelength λ. Each color matching function can be expressed by a graph such as that shown in FIG. 10. In the graph, the horizontal axis represents the wavelength λ, and the vertical axis represents the value of each color matching function.

[0034] The variable information 233 stored in the storage unit 23 will be described with reference to FIG. 11. The variable information 233 includes the scale coefficient a calculated by the difference emphasis filter variable calculation unit 21. ｘ , a ｙ , and a ｚ , translation coefficient b ｘ , b ｙ , and b ｚ , and the magnification coefficient c ｘ , c ｙ , and c ｚ In the following, the scale coefficient a calculated by the difference emphasis filter variable calculation unit 21 is stored. ｘ , a ｙ , and a ｚ , translation coefficient b ｘ , b ｙ , and b ｚ , and the magnification coefficient c ｘ , c ｙ , and c ｚ are the scale factors a ｘ ', a ｙ ', and a ｚ ', translation coefficient b ｘ ', b ｙ ', and b ｚ ', and the magnification coefficient c ｘ ', c ｙ ', and c ｚ It is written as '.

[0035] 1.5 Operation Next, the operation of the difference visualization device 10 according to the embodiment will be described.

[0036] The operation of the difference visualization device 10 according to the embodiment includes a process for calculating a difference emphasis filter variable and a process for generating difference emphasis data IMGo, which are executed in this order. The process for calculating the difference emphasis filter variable and the process for generating the difference emphasis data IMGo will be described below in this order.

[0037] 1.5.1 Process for Calculating Difference Emphasis Filter Variables The overall flow of the process for calculating difference emphasis filter variables will be described with reference to Fig. 12. Fig. 12 is a flowchart showing an example of the process for calculating difference emphasis filter variables using the difference visualization device according to the embodiment.

[0038] When the process of calculating the difference emphasis filter variables starts (START), the difference emphasis filter variable calculation unit 21 acquires data for calculating the difference emphasis filter variables (S1). Note that the data for calculating the difference emphasis filter variables includes, for example, the standard white light spectrum Is, the hyperspectral image IMGr, and the position information CA and CB. Then, the process proceeds to S2.

[0039] The difference emphasis filter variable calculation unit 21 acquires spectral data of the objects A and B in the hyperspectral image IMGr based on the acquired hyperspectral image IMGr and the position information CA and CB (S2). Note that hereinafter, the spectral data of the objects A and B are respectively referred to as spectral data S Ａ and S Ｂ Then, the process proceeds to S3.

[0040] The difference emphasis filter variable calculation unit 21 calculates the difference emphasis filter variable from the acquired spectral data S Ａ and S Ｂ Based on the above, the difference emphasis filter variable calculation unit 21 calculates a plurality of difference emphasis filter variables that maximize the color difference between the objects A and B (S3). That is, the difference emphasis filter variable calculation unit 21 calculates a plurality of difference emphasis filter variables that maximize the color difference between the objects A and B based on the above (S3). ｘ ', a ｙ ', and a ｚ ', translation coefficient b ｘ ', b ｙ ', and b ｚ ', and the magnification coefficient c ｘ ', c ｙ', and c ｚ Then, the process proceeds to S4. The calculation of the difference emphasis filter variable will be described later.

[0041] The difference emphasis filter variable calculation unit 21 transmits the calculated plurality of difference emphasis filter variables to the variable information 233 in the storage unit 23 (S4). As a result, the difference emphasis filter variables are stored in the variable information 233.

[0042] With the above processing, the process of calculating the difference emphasis filter variables is completed.

[0043] 1.5.1.1 Calculation of Difference Emphasis Filter Variables An example of calculation of difference emphasis filter variables in the process of S3 will be described.

[0044] The difference emphasis filter variable calculation unit 21 calculates the spectral data S Ａ and S Ｂ , the standard white light spectrum Is, and the calculated difference emphasis filter variable, the color difference D expressed by the calculated difference emphasis filter variable is calculated to be the maximum value. The color difference D between the objects A and B is, for example, ＊ a ＊ b ＊ It is calculated using the square sum error of the color system. ＊ a ＊ b ＊ The color system is L ＊ Component a ＊ Component b, and ＊ It is a color system expressed by components. ＊ a ＊ b ＊ The color system is known as a uniform color system, which is defined so that color differences that are perceived as being of equal magnitude have equal distances in the color system.

[0045] First, the difference emphasis filter variable calculation unit 21 calculates the scale coefficient a ｘ , a ｙ , and a ｚ , translation coefficient b ｘ , b ｙ , and b ｚ , and the magnification coefficient c ｘ , c ｙ , and c ｚThe spectral data S in the color system represented by the components X', Y', and Z' is Ａ Component X of Ａ ', Y Ａ ', and Z Ａ ', and spectral data S Ｂ Component X of Ｂ ', Y Ｂ ', and Z Ｂ ' is calculated. Ａ ', Y Ａ ', and Z Ａ ' corresponds to the components X', Y', and Z'. Ｂ ', Y Ｂ ', and Z Ｂ ' corresponds to the components X', Y', and Z'. Ａ and S Ｂ The components X', Y', and Z' are calculated using, for example, the following formulas (1), (2), and (3). In the following formulas (1), (2), and (3), the value N １ is a constant expressed by the following formula (4). In the following formulas (1), (2), and (3), the spectral data S Ａ and S Ｂ The intensity of is simply denoted as S(λ).

[0046]

[0047]

[0048]

[0049]

[0050] The difference emphasis filter variable calculation unit 21 calculates each of the spectral data S Ａ and S Ｂ The components X', Y', and Z' of L ＊ a ＊ b ＊ L in color system ＊ Component a ＊ Component b, and ＊ That is, the difference emphasis filter variable calculation unit 21 converts the spectral data S Ａ Component L ＊Ａ , a ＊ Ａ , and b ＊ Ａ , and the spectral data S Ｂ Component L Ｂ ＊ , a Ｂ ＊ , and b Ｂ ＊ Calculate the component L ＊ Ａ , a ＊ Ａ , and b ＊ Ａ Is, L ＊ a ＊ b ＊ L in color system ＊ Component a ＊ Component b, and ＊ Component L ＊ Ｂ , a ＊ Ｂ , and b ＊ Ｂ Is, L ＊ a ＊ b ＊ L in color system ＊ Component a ＊ Component b, and ＊ Each spectral data S Ａ and S Ｂ L ＊ Component a ＊ Component b, and ＊ The component is calculated using, for example, the following formulas (5), (6), and (7). In the following formulas (5), (6), and (7), the value X ｎ , Y ｎ , Z ｎ are constants relating to the components X, Y, and Z of the XYZ color system for a perfect diffuse reflecting surface (reference white surface). ｎ , Y ｎ , Z ｎ is calculated using the following formulas (8), (9), and (10). In the following formulas (8), (9), and (10), the value N ２ is a constant calculated using the following formula (11).

[0051]

[0052]

[0053]

[0054]

[0055]

[0056]

[0057]

[0058] Then, the difference emphasis filter variable calculation unit 21 calculates each of the spectral data S Ａ and S Ｂ L ＊ a ＊ b ＊ L in color system ＊ Component a ＊ Component b, and ＊ The difference emphasis filter variable calculation unit 21 calculates the color difference D based on the components. For example, the color difference D is calculated using the following formula (12): Ａ Component L Ａ ＊ , a Ａ ＊ , and b Ａ ＊ and the spectral data S Ｂ Component L Ｂ ＊ , a Ｂ ＊ , and b Ｂ ＊ Calculate the sum of squares error with

[0059]

[0060] The difference emphasis filter variable calculation unit 21 also calculates a plurality of difference emphasis filter variables that are included in the range that each difference emphasis filter variable can have and that maximize the color difference D expressed by the above formula (12). ｘ ', a ｙ ', and a ｚ ', translation coefficient b ｘ ', b ｙ', and b ｚ ', and the magnification coefficient c ｘ ', c ｙ ', and c ｚ ' is calculated.

[0061] The difference emphasis filter variable calculation unit 21 calculates the scale coefficient a at which the color difference D becomes the maximum value. ｘ ', a ｙ ', and a ｚ ', translation coefficient b ｘ ', b ｙ ', and b ｚ ', and the magnification coefficient c ｘ ', c ｙ ', and c ｚ When calculating ', for example, for each difference emphasis filter variable, the color difference D is calculated in a round-robin manner while changing the value of the difference emphasis filter variable in a sufficiently small range from the maximum value to the minimum value. Then, the difference emphasis filter variable calculation unit 21 determines the difference emphasis filter variable that gives the maximum color difference D.

[0062] Then, the difference emphasis filter variable calculation unit 21 calculates the scale coefficient a ｘ ', a ｙ ', and a ｚ ', translation coefficient b ｘ ', b ｙ ', and b ｚ ', and the magnification coefficient c ｘ ', c ｙ ', and c ｚ ' is stored in the variable information 233. As will be described later, the scale coefficient a ｘ ', a ｙ ', and a ｚ ', translation coefficient b ｘ ', b ｙ ', and b ｚ ', and the magnification coefficient c ｘ ', c ｙ ', and c ｚ ' is used in the process of generating the difference-enhanced data.

[0063] In this manner, the difference emphasis filter variables are calculated.

[0064] In addition, the scale coefficient a calculated as above ｘ ', aｙ ', and a ｚ ', translation coefficient b ｘ ', b ｙ ', and b ｚ ', and the magnification coefficient c ｘ ', c ｙ ', and c ｚ ' are the scale coefficients a in the above equations (1) to (4), respectively. ｘ , a ｙ , and a ｚ , translation coefficient b ｘ , b ｙ , and b ｚ , and the magnification coefficient c ｘ , c ｙ , and c ｚ The coordinates (color data) calculated by substituting the above can be said to be coordinates of the XYZ color system that emphasize the difference between objects A and B of the same color but of different kinds based on the color matching function.

[0065] 1.5.2 Process for Generating Difference-Enhanced Data The overall flow of the process for generating difference-enhanced data will be described with reference to Fig. 13. Fig. 13 is a flowchart showing an example of the process for generating difference-enhanced data using the difference visualization device according to the embodiment.

[0066] When the process of generating difference-enhanced data starts (START), the difference-enhancement filter application unit 22 acquires external data for generating the difference-enhanced data from outside the device (S11). Note that the external data includes, for example, a standard white light spectrum Is and observation object data IMGs. The process then proceeds to S12.

[0067] The difference emphasis filter application unit 22 refers to the color matching function information 232 and the variable information 233 (S12). The difference emphasis filter application unit 22 acquires this information from, for example, the storage unit 23. Then, the process proceeds to S13.

[0068] The difference emphasis filter application unit 22 generates difference emphasis data IMGo based on the acquired standard white light spectrum Is, the observation object data IMGs, the color matching function information 232, and the variable information 233 (S13). That is, the difference emphasis filter application unit 22 generates the difference emphasis data IMGo by applying the difference emphasis filter to the observation object data IMGs. Note that if the observation object data IMGs is a hyperspectral image, frames to which the difference emphasis filter has been applied are generated for each frame of the observation object data IMGs. Then, the process proceeds to S14. The generation of the difference emphasis data IMGo will be described later.

[0069] The difference emphasis filter application unit 22 outputs the generated difference emphasis data IMGo to the outside (S14). As described above, the difference emphasis data IMGo is output, for example, by a display.

[0070] With the above processing, the processing for generating difference-enhanced data is completed.

[0071] 1.5.2.1 Generation of Difference-Emphasis Data An example of generation of difference-emphasis data in the process of S13 will be described.

[0072] The difference emphasis filter application unit 22 applies the scale coefficient a calculated by the difference emphasis filter variable calculation unit 21 to the difference emphasis filter application unit 22. ｘ ', a ｙ ', and a ｚ ', translation coefficient b ｘ ', b ｙ ', and b ｚ ', and the magnification coefficient c ｘ ', c ｙ ', and c ｚ ' is the scale factor a in the above equations (1) to (4). ｘ , a ｙ , and a ｚ , translation coefficient b ｘ , b ｙ , and b ｚ , and the magnification coefficient c ｘ , c ｙ , and c ｚto calculate the components X', Y', and Z' of the spectral data S of each pixel of the observation object data IMGs acquired in the process of S11. That is, for the spectral data S of each pixel of the observation object data IMGs, the components X', Y', and Z' in the XYZ color system that emphasizes the difference between objects A and B of different types but the same color are calculated.

[0073] Further, the difference emphasis filter application unit 22 applies the spectral data S in the XYZ color system calculated as described above to the Ａ and S Ｂ The components X', Y', and Z' in the above formula are converted into the components R, G, and B in the sRGB color system defined by the International Electrotechnical Commission (IEC) and used for display on a display device, for example, using the following formulas (13), (14), (15), and (16). Ｌ , G Ｌ , and B Ｌ are respectively associated with the components R, G, and B. Ｌ , G Ｌ , and B Ｌ is a component that has not been corrected for brightness, etc., when displayed on a display, etc. In addition, in equation (16), the component R Ｌ , G Ｌ , and B Ｌ Each of these is represented as a variable u. In addition, in equation (16), the variable u (component R Ｌ , G Ｌ , and B Ｌ γ(u) is calculated according to the magnitude of each of the

[0074]

[0075]

[0076]

[0077]

[0078] Then, the difference emphasis filter application unit 22 applies the spectral data S calculated as described above to the Ａ and S ＢThe components R, G, and B in the sRGB color system are used as data for each pixel of an image or video frame to be output as difference-enhanced data.

[0079] In this way, the difference emphasis filter application unit 22 generates coordinates in the sRGB color system that emphasize the difference between objects A and B for the spectral data S of each pixel of the observation object data IMGs. In this way, the difference emphasis filter application unit 22 applies the difference emphasis filter to the observation object data IMGs, thereby generating difference emphasis data IMGo from the observation object data IMGs. Furthermore, the difference emphasis data IMGo generated in this way can be said to be an RGB image that emphasizes the difference between objects A and B.

[0080] 1.6 Effects of the Embodiments The embodiments provide a means for realizing the representation of the differences between two objects A and B of different types but the same color using natural color differences that retain shadows and contours. Furthermore, the embodiments provide a means for making it possible to observe whether an object C, which has spectral data different from those of objects A and B, appears to have a color closer to that of object A or object B.

[0081] The difference visualization device 10 according to the embodiment includes a difference emphasis filter variable calculation unit 21 and a difference emphasis filter application unit 22. The difference emphasis filter variable calculation unit 21 calculates spectral data S of an object A acquired from a hyperspectral image IMGr. Ａ (λ), and the spectral data S of object B Ｂ(λ) calculates coefficients used to calculate color data in the XYZ color system that emphasizes the difference between objects A and B so that the color difference D is maximized when each of the data is converted to color data. The coefficients include a scale coefficient, a translation coefficient, and a magnification coefficient. The difference emphasis filter application unit 22 uses the calculated coefficients to calculate color data in the XYZ color system that emphasizes the difference between objects A and B for each pixel of the observation target data IMGs. Then, the difference emphasis filter application unit 22 generates difference emphasis data IMGo that emphasizes the difference between objects A and B, based on the color data in the XYZ color system that emphasizes the difference between objects A and B. With the above-described configuration, the difference visualization device 10 according to the embodiment applies a difference emphasis filter in accordance with the spectral data of each pixel of the observation target data IMGs. As a result, the difference visualization device 10 according to the embodiment can express the differences between two objects A and B of different types but the same color with natural color differences that preserve shadows and contours. Furthermore, the difference visualization device 10 according to the embodiment makes it possible to observe whether an object C having spectral data different from those of objects A and B appears to have a color closer to that of object A or object B. Therefore, the difference visualization device 10 according to the embodiment applies a difference emphasis filter in which the scale coefficient, translation coefficient, and magnification coefficient of the color matching function are controlled, simulating the process of visual evolution in living organisms, thereby making it possible to generate an RGB image that naturally emphasizes the differences between different types of objects of the same color that are difficult for the human eye to distinguish, which appear in hyperspectral images and videos.

[0082] 2. Others In the above embodiment, the difference emphasis filter variable calculation unit 21 calculates the scale coefficient a ｘ ', a ｙ ', and a ｚ ', translation coefficient b ｘ ', b ｙ ', and b ｚ ', and the magnification coefficient c ｘ ', c ｙ ', and c ｚ Although an example in which all of the coefficients a′ are calculated has been shown, the present invention is not limited to this. The difference emphasis filter variable calculation unit 21 may calculate at least one of these coefficients. In this case, the scale coefficient a ｘ ', a ｙ', and a ｚ ', translation coefficient b ｘ ', b ｙ ', and b ｚ ', and the magnification coefficient c ｘ ', c ｙ ', and c ｚ The coefficients of "a" and "b" except for the at least one coefficient are determined in advance. With this configuration, the same effects as those of the embodiment can be achieved.

[0083] The present invention is not limited to the above-described embodiments, and various modifications can be made in the implementation stage without departing from the spirit of the invention. Furthermore, the embodiments may be implemented in appropriate combinations, in which case the combined effects can be obtained. Furthermore, the above-described embodiments include various inventions, and various inventions can be extracted by combining selected elements from the disclosed elements. For example, if the problem can be solved and the desired effect can be obtained even if some elements are deleted from all elements shown in the embodiments, the configuration from which these elements are deleted can be extracted as an invention.

[0084] REFERENCE SIGNS LIST 10... Difference visualization device 11... Control circuit 12... Storage 13... User interface 21... Difference emphasis filter variable calculation unit 22... Difference emphasis filter application unit 23... Storage unit 231... Information on maximum and minimum values ​​of variables 232... Color matching function information 233... Variable information

Claims

1. A difference visualization device comprising: a variable calculation unit that calculates at least one coefficient used in the conversion of first spectral intensity data of a first object and second spectral intensity data of a second object into color data in a first color system, such that a color difference between the first color data of the first spectral intensity data and the second color data of the second spectral intensity data is maximized when the first spectral intensity data and the second spectral intensity data are converted into color data in a first color system; and a difference visualization unit that performs the conversion using the at least one calculated coefficient for each pixel of the first data, which includes a plurality of pixels each having spectral intensity data, to generate second data, which includes a plurality of pixels having color data in which differences between the first object and the second object are emphasized.

2. The difference visualization device according to claim 1, wherein the variable calculation unit calculates the at least one coefficient based on a color matching function, and each of the at least one coefficient corresponds to one of a scale coefficient, a translation coefficient, and a magnification coefficient for modifying the color matching function.

3. The difference visualization device according to claim 1, wherein the first spectral intensity data and the second spectral intensity data are spectral intensity data of a first pixel and a second pixel of third data including a plurality of pixels, and the first data and the third data are images acquired by a hyperspectral camera.

4. A difference visualization method comprising: calculating at least one coefficient used in the conversion such that, when first spectral intensity data of a first object and second spectral intensity data of a second object are converted into color data in a first color system, a color difference between first color data of the first spectral intensity data and second color data of the second spectral intensity data is maximized; and performing the conversion using the at least one coefficient calculated for each pixel of first data including a plurality of pixels each having spectral intensity data, to generate second data including a plurality of pixels having color data in which differences between the first object and the second object are emphasized.

5. A program for causing each unit of the difference visualization device according to any one of claims 1 to 3 to function.

Images