Information processing device, information processing system, and information processing method

Abstract

To provide an information processing device, an information processing system and an information processing method with which it is possible to easily detect the features of a substance even when the optical characteristics of the substance are unknown.SOLUTION: An information processing device comprises: an image acquisition unit for acquiring a spectral image for a plurality of spectral wavelengths as a captured image for a subject to be imaged; a spectrum calculation unit for calculating the spectroscopic spectrum of each pixel on the basis of the plurality of spectral images; a feature wavelength detection unit for detecting the feature wavelength of the spectroscopic spectrum of each pixel; a color setting unit for calculating a feature angle that corresponds to the feature wavelength of each pixel and setting the color of a hue circle that corresponds to the feature angle as the feature color of the pixel; and an image generation unit for generating a feature detection image in which each pixel of the captured image is converted to a feature color that corresponds to the pixel.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing apparatus, an information processing system, and an information processing method for processing images. [Background technology]

[0002] Conventionally, a method for detecting foreign matter other than the target object from a captured image of the target object is known (see, for example, Non-Patent Document 1). The method described in Non-Patent Document 1 is a method for detecting leaves, branches, and the like mixed in with blueberries, and detects the leaves and branches by detecting an absorption peak wavelength of 680 nm from a captured image of blueberries. In other words, it is possible to detect foreign matter present in a captured image based on the absorption peak wavelength of the foreign matter to be detected. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Mizuki TSUTA, Tomohiro TAKAO, Junichi SUGIYAMA, Yukihiro WADA, Yasuyuki SAGARA, "Foreign Substance Detection in Blueberry Fruits by Spectral Imaging," Food Science and Technology Research, Vol. 12, No. 2, Japan Society for Food Science and Technology, 2006, published May 25, 2007, pp. 96-100 Summary of the Invention [Problem to be solved by the invention]

[0004] The conventional foreign matter detection method described above is effective when the absorption peak wavelength of the detected foreign matter is known. However, when the substance of the foreign matter or other substance is unknown and its optical characteristics, such as its absorption peak wavelength, are unknown, it is difficult to detect the characteristics of the substance. [Means for solving the problem]

[0005] An information processing device according to a first aspect of the present disclosure includes: an image acquisition unit that acquires spectral images for a plurality of spectral wavelengths as an image of an imaging target; a spectrum calculation unit that calculates the spectral spectrum of each pixel based on the plurality of spectral images; a characteristic wavelength detection unit that detects a characteristic wavelength corresponding to a predetermined characteristic condition in the spectral spectrum of each of the pixels; a color setting unit that, when a spectral wavelength range including the plurality of spectral wavelengths is assigned to a predetermined angle range of a hue wheel, calculates a characteristic angle corresponding to the characteristic wavelength of each of the pixels and sets a color of the hue wheel according to the characteristic angle as a characteristic color of that pixel; and an image generation unit that generates a feature detection image in which each of the pixels of the captured image is converted into the characteristic color corresponding to that pixel.

[0006] In the information processing device of this aspect, it is preferable that the angle range of the hue circle is 270 degrees or less.

[0007] In the information processing device of this aspect, it is preferable that the color setting unit sets the characteristic color by setting the shortest wavelength in the spectral wavelength range as the minimum angle in the angle range and the longest wavelength in the spectral wavelength range as the maximum angle in the angle range.

[0008] In the information processing device of this aspect, the color setting unit may set the characteristic color by setting the shortest characteristic wavelength among the characteristic wavelengths detected at the multiple pixels as the minimum angle of the angle range, and by setting the longest characteristic wavelength as the maximum angle of the angle range.

[0009] In the information processing device of this aspect, the characteristic wavelength detection unit may detect, for each of the pixels, n characteristic wavelengths corresponding to the n characteristic conditions, and the color setting unit may calculate the characteristic angle based on each projection point when the characteristic points are projected onto a straight line based on the characteristic points corresponding to the characteristic wavelengths of each of the pixels plotted in an n-dimensional space with the n characteristic conditions as axes, respectively.

[0010] In the information processing device of the above aspect, it is preferable that the straight line is a straight line that maximizes the variance of the projection points corresponding to the plurality of feature points.

[0011] In the information processing device of the above aspect, it is preferable that the color setting unit sets the characteristic color by setting one of the two points on the straight line that are farthest apart as the minimum angle of the angle range and the other as the maximum angle of the angle range.

[0012] In the information processing device according to this aspect, it is preferable that the characteristic wavelength detection section detects the characteristic wavelength based on a second derivative waveform of the optical spectrum of each of the pixels.

[0013] An information processing system according to a second aspect of the present disclosure includes the information processing device according to the first aspect described above, and a spectroscopic camera configured to capture the captured image including a plurality of spectroscopic images of the imaging target.

[0014] An information processing method of a second aspect of the present disclosure is an information processing method that causes one or more processors to process image information, and causes the one or more processors to perform the following operations: acquire spectral images for a plurality of spectral wavelengths as an image of an image capture target; calculate a spectral spectrum of each pixel based on the plurality of spectral images; detect a characteristic wavelength corresponding to a predetermined characteristic condition in the spectral spectrum of each pixel; when a spectral wavelength range including the plurality of spectral wavelengths is assigned to a predetermined angle range of a hue wheel, calculate a characteristic angle corresponding to the characteristic wavelength of each pixel, and set a color on the hue wheel corresponding to the characteristic angle as a characteristic color of that pixel; and generate a feature detection image in which each pixel of the image capture target is converted into the characteristic color corresponding to that pixel. [Brief explanation of the drawings]

[0015] [Figure 1] FIG. 1 is a diagram showing a schematic configuration of an information processing system according to a first embodiment. [Figure 2] FIG. 1 is a diagram showing a schematic configuration of a spectroscopic camera according to an embodiment of the present invention. [Figure 3] 3 is a flowchart showing an information processing method according to the present embodiment. [Figure 4] 10 is a diagram showing an example of a captured image of an object in which wheat flour as an inspection object and polyethylene as a foreign substance are mixed in the present embodiment; [Figure 5] 5 is a diagram showing an example of optical spectra at a plurality of pixels of the captured image of FIG. 4. [Figure 6] FIG. 6 is a diagram showing the second derivative waveform of the spectrum of FIG. 5. [Figure 7] FIG. 10 is a diagram showing the angle range for setting a color on a color wheel. [Figure 8] 5A and 5B are diagrams illustrating a method for determining a characteristic color of each pixel in the embodiment. [Figure 9] FIG. 5 is a diagram showing an example of a feature detection image for the captured image of FIG. 4. [Figure 10] 10A and 10B are diagrams illustrating a method for determining a characteristic color of each pixel in the second embodiment. [Figure 11] FIG. 11 is an explanatory diagram of a case where the characteristic wavelength λd of the foreign substance in FIG. 10 satisfies λd>λM1. [Figure 12] FIG. 10 is a diagram illustrating a method for determining the color of each pixel in the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0016] [First embodiment] An information processing system according to a first embodiment of the present invention will be described below. (Outline of information processing system configuration) FIG. 1 is a diagram showing a schematic configuration of an information processing system according to this embodiment. The image processing system of this embodiment is configured to include a spectroscopic camera 1A and a terminal device 1B, which is an information processing device, as shown in Fig. 1. The spectroscopic camera 1A and the terminal device 1B may be configured integrally, for example, a smartphone equipped with the spectroscopic camera 1A.

[0017] (Outline of spectroscopic camera configuration) Fig. 2 is a diagram showing a schematic configuration of the spectroscopic camera 1A of this embodiment. As shown in Fig. 2, the spectroscopic camera 1A includes a spectroscopic element 11, an image sensor 12 that captures image light dispersed by the spectroscopic element 11, and a camera control unit 13 that controls the operation of the spectroscopic camera 1A.

[0018] For example, an interference filter (Fabry-Perot etalon) in which a pair of reflective films are arranged opposite each other can be used as the spectroscopic element 11. In such an interference filter, the wavelength of the light to be separated can be changed by changing the size of the gap between the pair of reflective films. The image pickup element 12 is configured by, for example, a charge-coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS), and picks up an image of the light dispersed by the dispersing element 11.

[0019] Based on an imaging command from terminal device 1B, camera control unit 13 controls spectroscopic element 11 and image sensor 12 to capture an image of imaging target W. In this embodiment, camera control unit 13 sequentially switches the spectral wavelength of light transmitted through spectroscopic element 11, and causes image sensor 12 to capture a spectral image each time the spectral wavelength is switched. Then, the camera control unit 13 transmits the obtained plurality of spectral images as a captured image of the imaging target W to the terminal device 1B.

[0020] [Terminal device configuration] 1, terminal device 1B may be configured by a general computer such as a smartphone, a tablet terminal, or a personal computer, and constitutes an information processing device of the present disclosure. Specifically, terminal device 1B includes a display unit 21, an operation unit 22, a storage unit 23, and a control unit 24. Terminal device 1B may also include a communication device or the like capable of communicating with external devices via the Internet or the like.

[0021] The display unit 21 is a display, and displays an image under the control of the control unit 24 . The operation unit 22 is configured with, for example, a keyboard, a mouse, a touch panel, etc., and receives input operations from the user and inputs operation signals corresponding to the input operations to the control unit 24. Furthermore, the display unit 21 and the operation unit 22 may be configured integrally with the terminal device 1B, or may be configured as separate entities from the terminal device 1B and connected to the terminal device 1B so as to be able to communicate with it.

[0022] The storage unit 23 is an information storage device such as a semiconductor memory or a hard disk, and stores various programs and data for controlling the terminal device 1B and the information processing system 1. For example, the storage unit 23 stores an information processing program for detecting characteristics of the imaging target W from a plurality of spectral images of the imaging target W obtained as captured images. The storage unit 23 also stores color wheel information indicating a color wheel used in the information processing program.

[0023] The control unit 24 is configured with an arithmetic circuit such as a CPU (Central Processing Unit) and can be realized as one or more processors, and reads and executes various programs stored in the storage unit 23. For example, in this embodiment, by reading and executing the information processing programs stored in the storage unit 23, the control unit 24 functions as an image acquisition unit 241, a spectrum calculation unit 242, a characteristic wavelength detection unit 243, a color setting unit 244, and an image generation unit 245.

[0024] The image acquisition unit 241 acquires, as a captured image, spectral images for a plurality of spectral wavelengths input from the spectroscopic camera 1A. That is, the plurality of spectral images included in the captured image are captured for the same imaging target under the same imaging conditions, and each pixel of each spectral image has information for the same position on the imaging target. In this embodiment, the image acquisition unit 241 acquires, as captured images, spectral images of the imaging target W captured at a plurality of spectral wavelengths by the spectroscopic camera 1A, but is not limited to this. For example, the image acquisition unit 241 may acquire, as captured images, a plurality of spectral images transmitted via the Internet or the like, or may acquire, as captured images, a plurality of spectral images stored in the storage unit 23.

[0025] The spectrum calculation unit 242 calculates the spectrum for each pixel in the captured image, that is, calculates the spectrum for pixel (i, j) based on the signal value for that pixel (i, j) in each spectral image. The characteristic wavelength detection unit 243 detects, for each pixel, a characteristic wavelength corresponding to a predetermined characteristic condition based on the optical spectrum. The predetermined characteristic condition is a condition that indicates the characteristics of the optical spectrum, and may include, for example, the peak wavelength with the maximum peak height, the peak wavelength with the second largest peak height, the bottom wavelength with the maximum bottom depth (i.e., the peak wavelength that is downwardly convex in the optical spectrum), the zero-crossing points, peak wavelength, and bottom wavelength of the first-order differential waveform of the optical spectrum, and the zero-crossing points, peak wavelength, and bottom wavelength of the second-order differential waveform of the optical spectrum. In this embodiment, the peak wavelength with the highest peak height among the peak wavelengths included in the second-order differential waveform of the optical spectrum is used as the characteristic condition.

[0026] The color setting unit 244 sets a color that indicates the characteristics of each pixel based on the calculated spectral spectrum. Specifically, the color setting unit 244 calculates the angle on the hue wheel that corresponds to the characteristic wavelength of each pixel when the spectral wavelength range that includes the spectral wavelengths of the multiple acquired spectral images is assigned to a predetermined angle range on the hue wheel, and sets the color on the hue wheel corresponding to the angle as the color for that pixel. Therefore, in this embodiment, the color of each pixel is set to a color that is unrelated to the color of the actual image capture target W.

[0027] The image generating unit 245 generates a feature detection image by converting each pixel of the captured image into the color of the corresponding pixel set by the color setting unit 244, and causes the display unit 21 to display the image.

[0028] (Information processing method) Next, the information processing method by the information processing system 1 in this embodiment will be described in more detail. FIG. 3 is a flowchart showing the information processing method of this embodiment. In this embodiment, when a user performs an input operation on the terminal device 1B to perform a detection process for detecting features of the imaging target W, the image acquisition unit 241 acquires a captured image of the imaging target (step S1: image acquisition step). For example, the image acquisition unit 241 outputs an imaging command to the spectroscopic camera 1A to capture spectral images for a plurality of spectral wavelengths. As a result, the spectroscopic camera 1A captures spectral images for each of the plurality of spectral wavelengths of the imaging target and returns these spectral images to the terminal device 1B as captured images of the imaging target. As a result, the image acquisition unit 241 acquires a captured image including the plurality of spectral images. Examples of detection processes for detecting the characteristics of the imaging target W include a process for detecting the position of the test object and the position of any foreign matter other than the test object when the test target W is an inspection sample containing the test object, and a process for detecting the position of each test object in an inspection sample containing multiple test objects.

[0029] Next, the spectrum calculation unit 242 calculates the spectrum of each pixel of the captured image based on the multiple spectral images included in the imaging target W obtained in step S1 (step S2: spectrum calculation step). That is, the spectrum calculation unit 242 calculates the relationship between the wavelength and signal value of pixel (i, j) based on the signal value of pixel (i, j) in each spectral image. Note that in this embodiment, the spectrum is calculated based on the signal value for each wavelength of each pixel (luminance value in the captured image). However, for example, the spectrum of each pixel and the emission spectrum of the light source may be used to calculate the reflectance spectrum, absorption spectrum, etc. of each pixel.

[0030] Fig. 4 is an example of an image captured of an object W, which is a mixture of wheat flour as an inspection object and polyethylene as a foreign substance. Fig. 5 is a diagram showing an example of the optical spectrum of a plurality of pixels in the captured image. In Fig. 5, the solid line indicates the optical spectrum of wheat flour, and the dashed line indicates the optical spectrum of polyethylene.

[0031] After step S2, the characteristic wavelength detection unit 243 detects a characteristic wavelength from the spectrum calculated in step S2 (step S3: characteristic wavelength detection step). Fig. 6 shows the second derivative waveform of the spectrum shown in Fig. 5. In Fig. 6, the solid line shows the spectrum of wheat flour, and the dashed line shows the spectrum of polyethylene. In this embodiment, the characteristic wavelength detection unit 243 detects the peak wavelength of the second-order derivative waveform of the optical spectrum obtained in step S2 as the characteristic wavelength. Since the second-order derivative waveform may also emphasize noise components, it is preferable to reduce the noise components by smoothing. By using such a second-order derivative waveform, it is possible to detect characteristic wavelengths such as peak wavelengths and bottom wavelengths in the optical spectrum with high accuracy, even if the spectral shape makes it difficult to detect the peak wavelength, as shown in FIG. 5 .

[0032] Next, the color setting unit 244 reads out the hue circle information from the storage unit 23, identifies the angle on the hue circle for the peak wavelength of each pixel, and sets the color corresponding to the angle as the characteristic color for that pixel (step S4: color setting step). 7 is a diagram showing an angle range Cr for setting a color on a color wheel. This angle range Cr is a preset range of 270 degrees or less on the color wheel, and its minimum angle c m For example, in this embodiment, as shown in FIG. 7, the range of angles Cr in the hue circle is from 0 degrees to 270 degrees, and the minimum angle c m is 0 degrees, and the maximum angle c M is set to 270 degrees. Note that these angle ranges Cr and the minimum angle c m may be arbitrarily set by the user. FIG. 8 is a diagram illustrating a method for determining the characteristic color of each pixel in this embodiment. The color setting unit 244 specifies the angle of the hue circle corresponding to the characteristic wavelength (peak wavelength in this embodiment) of each pixel of the captured image. That is, as shown in FIG. 8, the color setting unit 244 sets the shortest wavelength λ m The minimum angle c m and the longest wavelength λ of the spectral image M The maximum angle c M By assigning the angle (characteristic angle c1) to the characteristic wavelength λ1, c1=c m +(c M -c m )×(λ1-λ m ) / (λ M -λ m For example, when spectral images from 850 nm to 1050 nm are acquired at predetermined wavelength intervals, the spectral wavelength range of the captured image is 850 nm to 1050 nm. Also, the angle range in the hue circle is set to 0 degrees to 270 degrees. In this case, the shortest wavelength λ in the spectral wavelength range is m The minimum angle c m The longest wavelength in the spectral wavelength range, λ, is assigned to 0 degrees. M The maximum angle c M Therefore, in this case, the characteristic angle c1 for the characteristic wavelength λ1 is calculated as c1=1.35λ1−1147.5. The color setting unit 244 then sets the characteristic color corresponding to the characteristic angle c1 calculated as above based on the hue circle. At this time, since the angle range Cr is 270 degrees or less, the shortest wavelength λ m and the longest wavelength λ M The two have different hues, allowing them to be properly distinguished from each other.

[0033] After the above, the image generating unit 245 generates a feature detection image in which each pixel of the captured image is replaced with the color of the corresponding pixel set in step S4 (step S5). By using such an information processing method, it is possible to detect substances contained in the imaging target W and foreign substances mixed in the substances even when the absorption wavelengths of these substances are unknown. For example, Fig. 9 is a diagram showing an example of a feature detection image for the captured image of Fig. 4. Note that Fig. 9 displays a grayscale image, but in reality, the black parts of Fig. 9 are displayed in greenish yellow, and the gray parts of Fig. 9 are displayed in blue-green. In this embodiment, the characteristic wavelength of wheat flour, which is the inspection object, is 985 nm, and the characteristic wavelength of polyethylene, which is the foreign matter, is 930 nm. In this case, the feature detection image displays blue-green, which corresponds to a hue angle of 182 degrees, at the pixel position of wheat flour (inspection object), and displays greenish yellow, which corresponds to a hue angle of 108 degrees, at the pixel position of polyethylene (foreign matter).

[0034] [Effects of this embodiment] The information processing system 1 of this embodiment includes a spectroscopic camera 1A that captures an image including a plurality of spectroscopic images of an imaging target W, and a terminal device 1B that is an information processing device. The terminal device 1B includes a storage unit 23 and a control unit 24. The control unit 24 loads and executes an information processing program stored in the storage unit 23, thereby functioning as an image acquisition unit 241, a spectrum calculation unit 242, a characteristic wavelength detection unit 243, a color setting unit 244, and an image generation unit 245. The image acquisition unit 241 acquires spectral images for multiple spectral wavelengths as captured images of the imaging target W. The spectrum calculation unit 242 calculates the spectral spectrum of each pixel based on the multiple spectral images. The characteristic wavelength detection unit 243 detects a characteristic wavelength corresponding to a predetermined characteristic condition in the spectral spectrum of each pixel. The color setting unit 244 calculates a characteristic angle c1 corresponding to the characteristic wavelength of each pixel when the spectral wavelength range is assigned to a predetermined angle range Cr of the hue wheel, and sets the color of the hue wheel corresponding to the characteristic angle c1 as the characteristic color. The image generation unit 245 generates a feature detection image in which each pixel of the captured image is converted into the characteristic color corresponding to that pixel.

[0035] As a result, in this embodiment, even if the substances such as the test object and foreign matter contained in the image capture target W are unknown and the optical properties such as the absorption wavelength of these substances are unknown, the characteristics of the substances can be represented by color, making it easy to detect the properties. For example, in this embodiment, even if the optical properties of the test object and foreign matter are unknown, the test object and foreign matter can be appropriately distinguished and the two can be clearly represented by different colors.

[0036] In this embodiment, the angle range Cr of the hue circle is set to 270 degrees or less. This allows different colors to be set for different wavelengths when setting characteristic colors for characteristic wavelengths in the color setting unit 244. In other words, if the angle range Cr is set to 360 degrees, the characteristic color for the shortest wavelength in the spectral wavelength range and the characteristic color for the longest wavelength will be approximately the same color, making it difficult to distinguish between them. In contrast, by narrowing the angle range Cr as described above, it is possible to set different colors for the characteristic color for the shortest wavelength and the characteristic color for the longest wavelength.

[0037] In this embodiment, the color setting unit 244 sets the shortest wavelength λ m The minimum angle c of the angle range Cr in the color wheel m and the longest wavelength λ in the spectral wavelength range M The maximum angle c of the angle range Cr in the color wheel M The characteristic color is set as follows. This makes it possible to cover all the characteristic wavelengths included in the spectral wavelength range, and in the characteristic detection image, pixels corresponding to each characteristic wavelength can be displayed in different colors.

[0038] In addition, in this embodiment, a peak wavelength (bottom wavelength) that is convex downward in the optical spectrum is used as the characteristic wavelength. That is, the characteristic wavelength used in this embodiment is the peak absorption wavelength in the optical spectrum, and this peak absorption wavelength is a characteristic wavelength that indicates the characteristics of the substance contained in the imaged object W. Therefore, by detecting such a peak absorption wavelength, the substance of the imaged object W can be suitably characterized, and it can be suitably distinguished from other substances such as foreign matter.

[0039] In this embodiment, the characteristic wavelength detection unit 243 identifies the characteristic wavelength based on the second derivative waveform of the optical spectrum of each pixel. As a result, even if the optical spectrum has a relatively gentle shape that makes it difficult to detect the peak wavelength or bottom wavelength, it is possible to emphasize the unevenness in the spectral shape and accurately detect the characteristic wavelengths such as the peak wavelength and bottom wavelength of the optical spectrum.

[0040] [Second embodiment] In the first embodiment, the color setting unit 244 sets the shortest wavelength λ m the minimum angle c of the color wheel m and the longest wavelength λ in the spectral wavelength range M the maximum angle c of the color wheel M The characteristic angle c1 for each characteristic wavelength was calculated as follows. However, in this case, if the characteristic wavelengths of the inspection object and the foreign substance contained in the imaging target W are close to each other, the difference in color between them may be difficult to discern in the feature detection image. In contrast, in the second embodiment, the method of setting the characteristic color by the color setting unit 244 is different from that in the first embodiment. In the following description, the same reference numerals will be used to designate items that have already been described, and their description will be omitted or simplified.

[0041] The information processing system of the second embodiment has the same configuration as that of the first embodiment, and includes a spectroscopic camera 1A and a terminal device 1B. The terminal device 1B includes a storage unit 23 and a control unit 24. The control unit 24 reads and executes an information processing program stored in the storage unit, thereby functioning as an image acquisition unit 241, a spectrum calculation unit 242, a characteristic wavelength detection unit 243, a color setting unit 244, and an image generation unit 245. These components of the present embodiment are the same as those of the first embodiment, but differ from those of the first embodiment in the processing performed by the color setting unit 244. Therefore, the processing performed by the color setting unit 244 will be described here, and descriptions of the other components will be omitted.

[0042] FIG. 10 is a diagram illustrating a method for determining the color of each pixel in the second embodiment. In the information processing system of this embodiment, similarly to the first embodiment, as shown in FIG. 3, the processes from step S1 to step S3 are performed to detect the characteristic wavelength of each pixel. In the present embodiment, in the subsequent step S4, if a plurality of characteristic wavelengths are detected in step S3, the color setting unit 244 sets the shortest characteristic wavelength λ 1 among the detected characteristic wavelengths. m1 The minimum angle c m The longest characteristic wavelength λ M1 The maximum angle c M And assign other characteristic wavelengths λ 1i Feature angle to ci , c 1i =c m +(c M -c m )×(λ 1i -λ m1 ) / (λ M1 -λ m1 For example, in the spectral wavelength range of 850 nm to 1050 nm, the angle range Cr in the hue circle is set to 0 degrees to 270 degrees, and the shortest wavelength λ m1 is 900 nm, the longest wavelength in the characteristic wavelength λ M1 When the characteristic wavelength λ is 1000 nm, 1i characteristic angle c 1i is c 1i =2.7λ 1i -2430.

[0043] In particular, when the imaging target W contains two or more types of inspection objects, the shortest wavelength among the characteristic wavelengths of these inspection objects is λ m1 , the longest wavelength is λ M1 This allows for suitable detection of foreign matter. That is, if the inspection sample contains a foreign substance, the characteristic wavelength λ of the foreign substance d is λ m1 <λ d <λ M1 If so, the characteristic wavelength λ of the foreign substance d The characteristic color corresponding to the characteristic wavelength λ is displayed in the characteristic detection image. d is λ m1 or λ M1Even when the hue of the characteristic color of the foreign substance is close to , the angle difference between the hue of the characteristic color of the foreign substance and the hue of the characteristic color of the test object is larger than in the first embodiment, so that the position where the foreign substance exists can be easily determined in the feature detection image.

[0044] Figure 11 shows the characteristic wavelength λ of the foreign substance. d is λ d >λ M1 FIG. Characteristic wavelength λ of foreign matter d is λ d >λ M1 If λ d is the maximum angle c in the angle range Cr M In this case, the characteristic wavelength is assigned to λ M1 A feature detection image is generated in which the hue of the test object is significantly different from that of a test sample that does not contain any foreign matter. Although not shown in the figure, the characteristic wavelength λ of the foreign matter d is λ d <λ m1 The same is true for the case where λ d is the minimum angle c in the angle range Cr m Therefore, the characteristic wavelength is assigned to λ m1 A feature detection image is generated in which the hue of the test object is significantly different from that of a test sample that does not contain any foreign matter. In this way, when a foreign substance is present, a feature detection image with a significantly different overall hue is generated compared to when no foreign substance is present, and the presence or absence of foreign substance can be accurately determined even when the foreign substance content is small. In particular, when consecutive images of test samples are taken and each test sample is sequentially inspected for the presence or absence of foreign substance, test samples containing foreign substance can be easily found.

[0045] In addition, when the imaging target W is an inspection sample containing one inspection object, if no foreign matter is mixed in, the characteristic wavelength will be one. In this case, the characteristic color may be set by the same method as in the first embodiment. When such test samples are successively imaged and each test sample is successively inspected for the presence or absence of foreign matter, foreign matter can be detected more accurately by switching between the method of the first embodiment and the method of the second embodiment. That is, when the test sample does not contain a foreign substance, only the characteristic wavelength for the test object is acquired, and the characteristic wavelength is assigned a characteristic color by the method of the first embodiment. On the other hand, when the test sample contains a foreign substance, the characteristic wavelength of the test object and the characteristic wavelength of the foreign substance are detected, and the characteristic color is assigned to each characteristic wavelength by the method of the second embodiment. In this case, the hue of the test object in the feature detection image changes significantly depending on whether the test object sample contains a foreign object or not. Therefore, even if the amount of foreign object in the test object sample is small, the presence of a foreign object can be properly notified.

[0046] [Effects of this embodiment] In the terminal device 1B of this embodiment, the color setting unit 244 sets the shortest characteristic wavelength λ among the characteristic wavelengths detected in the plurality of pixels. m1 the minimum angle c in the color wheel m and the longest characteristic wavelength λ M1 The maximum angle c in the color wheel M The characteristic color is set as follows. As a result, even when there are two or more characteristic wavelengths and the difference between these characteristic wavelengths is small, the difference in the characteristic angle for each characteristic wavelength can be made wider than in the first embodiment, as shown in Fig. 10. Therefore, in the feature detection image, the hues of pixel positions where different substances exist will differ greatly, making it easier to determine the positions where each substance exists.

[0047] [Third embodiment] In the first and second embodiments, the characteristic wavelength detection unit 243 detects one characteristic wavelength from the optical spectrum of each pixel, and the color setting unit 244 sets a characteristic color based on the characteristic wavelength. On the other hand, there are substances that have similar maximum peak wavelengths but different second peak wavelengths, or substances that have similar maximum peak wavelengths but different bottom wavelengths, and it can be difficult to distinguish between these substances. In the third embodiment, in order to solve this problem, a plurality of characteristic wavelengths are detected from the optical spectrum of each pixel, and a characteristic color is set from these characteristic wavelengths.

[0048] The information processing system of the third embodiment has the same configuration as that of the first embodiment, and includes a spectroscopic camera 1A and a terminal device 1B. The terminal device 1B includes a storage unit 23 and a control unit 24. The control unit 24 reads and executes an information processing program stored in the storage unit, thereby functioning as an image acquisition unit 241, a spectrum calculation unit 242, a characteristic wavelength detection unit 243, a color setting unit 244, and an image generation unit 245. These components of the present embodiment are similar to those of the first embodiment, but differ from those of the first embodiment in the processing performed by the characteristic wavelength detection unit 243 and the color setting unit 244. Therefore, the processing performed by the characteristic wavelength detection unit 243 and the color setting unit 244 will be described here, and a description of the other components will be omitted.

[0049] FIG. 12 is a diagram illustrating a method for determining the color of each pixel in the third embodiment. In the information processing system of this embodiment, as in the first embodiment, after steps S1 and S2 are performed, the characteristic wavelength detection unit 243 detects a characteristic wavelength in step S3, as shown in FIG. In this embodiment, the characteristic wavelength detection unit 243 selects n characteristic wavelengths having predetermined characteristics from each pixel. The n characteristic wavelengths each have different characteristics, and can be selected from, for example, the peak wavelength with the maximum peak height, the bottom wavelength with the maximum bottom depth, the peak wavelength with the second-highest peak height, the zero-crossing points, peak wavelength, and bottom wavelength of the first-order derivative waveform of the optical spectrum, and the zero-crossing points, peak wavelength, and bottom wavelength of the second-order derivative waveform. For simplicity of explanation, n=2 is used here, and an example is shown in which two characteristic wavelengths are selected from each pixel, with the peak wavelength with the maximum peak height in the second-order derivative waveform of the optical spectrum being the first characteristic wavelength λa and the peak wavelength with the second-highest peak height being the second characteristic wavelength λb.

[0050] Then, in step S4, the color setting unit 244 calculates a characteristic angle based on a straight line l based on the characteristic point P obtained when the n characteristic wavelengths of each pixel detected in step S3 are plotted as the characteristic point P in n-dimensional space. For example, in this embodiment, n=2, and the feature point P is plotted in a two-dimensional space having a first axis representing the first characteristic wavelength λa and a second axis representing the second characteristic wavelength λb, as shown in Fig. 12. In the example of Fig. 12, a1 and the second characteristic wavelength λ b1 The pixel feature point P1 has the first feature wavelength λ a2 and the second characteristic wavelength λ b2 The pixel feature point P2 has the first feature wavelength λ ad and the second characteristic wavelength λ bd Feature point P of the pixel having d An example is shown below. The test sample contains two types of test objects and a foreign object, and the feature points P corresponding to one of the test objects are plotted so as to be close to each other, and the feature points P corresponding to the other test object are plotted so as to be close to each other.

[0051] Furthermore, the straight line l based on the feature points P is preferably a straight line that maximizes the variance of each projection point Q, where the points obtained by projecting each feature point P onto the straight line l are set as projection points Q. The color setting unit 244 calculates the principal component value of each projection point Q using a principal component analysis method and converts it into an angle on the hue circle. For example, the color setting unit 244 calculates the element vector (first characteristic wavelength λ ai , second characteristic wavelength λ bi ) to calculate the variance-covariance matrix, and then calculate the eigenvalues ​​and eigenvectors of the variance-covariance matrix. When n characteristic wavelengths are selected from each pixel, n eigenvalues ​​and eigenvectors are obtained, and the largest eigenvalue among these eigenvalues ​​is designated as the first eigenvalue, and the eigenvector corresponding to the first eigenvalue is designated as the first eigenvector. Then, the color setting unit 244 calculates the principal component value of each pixel from the inner product of the element vector of each pixel and the first eigenvector. This principal component value is a parameter indicating the position of each pixel on the line l where the variance of the projection point Q is maximized, and is a parameter obtained by converting n-dimensional data into one-dimensional data.

[0052] After the above, the color setting unit 244 converts the principal component value of each pixel into a characteristic angle in the same manner as in the second embodiment, and sets a characteristic color corresponding to the characteristic angle. That is, on the line l, the projection point Q with the smallest principal component value is located at the smallest angle c m , and the projection point Q with the largest principal component value is assigned to the maximum angle c M , and calculate the feature angle from the principal component value of each pixel. That is, for each projection point Q on the line l, the two projection points Q with the greatest distance are selected, and the one with the smallest principal component value (projection point Q1 in Figure 12) is assigned the minimum angle c m , and the other principal component with a larger value (projection point Q2 in Fig. 12) is assigned to the maximum angle c M Assign to.

[0053] [Effects of this embodiment] In the terminal device 1B of this embodiment, the characteristic wavelength detection unit 243 selects n wavelengths having predetermined characteristics as characteristic wavelengths for each pixel. Then, the color setting unit 244 plots a characteristic point P corresponding to the characteristic wavelength of each pixel in an n-dimensional space with each of the n characteristics as an axis, and calculates the characteristic angle of each pixel based on each projection point Q when the characteristic point P is projected onto a straight line l based on the characteristic point P. This allows the setting of a characteristic color based on not just one characteristic but multiple characteristics from the optical spectrum of each pixel. For example, even if the imaged object W includes many different types of inspection objects, each inspection object can be distinguished based on multiple characteristic wavelengths. Furthermore, even if there is a foreign object with a peak wavelength similar to the inspection object, the inspection object can be distinguished from the foreign object based on the other characteristic wavelengths.

[0054] [Variations] The present invention is not limited to the above-described embodiment, and modifications and improvements within the scope of achieving the object of the present invention are included in the present invention.

[0055] (Variation 1) In the above embodiment, the spectrum calculation unit 422 calculates the signal value for each wavelength of each pixel (each region) as an optical spectrum, but is not limited to this. For example, it may calculate an absorption spectrum indicating the light absorption rate for each wavelength of each pixel, or a reflectance spectrum.

[0056] (Variation 2) The color setting unit 244 converts the characteristic wavelength of each pixel into an angle within the angle range Cr of 270 degrees or less on the hue circle, but is not limited to this. That is, the angle range Cr should be set so that in the feature detection image, the pixel with the maximum characteristic wavelength and the pixel with the minimum characteristic wavelength are clearly distinguishable in color. For example, in the color wheel of the CIE1976 L*a*b* color system, the hue angle of 0 degree and the hue angle of 360 degrees are almost the same color red, but the hue angle of 30 degrees is orange, which can be distinguished from red. Therefore, the angle range Cr may be a range of 330 degrees from the minimum angle of 30 degrees to the maximum angle of 360 degrees. Note that by setting the angle range to 270 degrees or less as in the above embodiment, the minimum angle c m and the maximum angle c M This makes it possible for the human eye to more reliably distinguish the hue of the pixel having the maximum characteristic wavelength from the hue of the pixel having the minimum characteristic wavelength, thereby further reducing the inconvenience of the pixel having the maximum characteristic wavelength and the pixel having the minimum characteristic wavelength being regarded as having the same characteristic.

[0057] (Variation 3) In the third embodiment, the characteristic wavelength detecting unit 243 detects two characteristic wavelengths from the optical spectrum of each pixel, but three or more characteristic wavelengths may be detected. For example, when the characteristic wavelength detecting unit 243 detects n characteristic wavelengths based on three characteristics, the color setting unit 244 calculates a straight line l based on the characteristic point P corresponding to each pixel plotted in three-dimensional space, and converts each projection point Q into an angle on the hue circle according to the position of the projection point Q obtained by projecting the feature point P onto the straight line l.

[0058] (Variation 4) In the third embodiment, an example has been shown in which the color setting unit 244 calculates, as the straight line l based on the feature points P, a straight line that maximizes the variance of the projection points Q using a principal component analysis technique, but this is not limiting. The straight line l based on the feature points P may be, for example, a regression line calculated by the least squares method based on each feature point P, or may be a straight line parallel to the straight line that maximizes the variance of the projection points Q, or a straight line parallel to the regression line.

[0059] (Variation 5) In the above embodiment, the characteristic wavelength detection unit 243 detects characteristic wavelengths based on the second-order derivative waveform of the optical spectrum, but this is not limiting. For example, the zero-crossing points may be detected using the first-order derivative waveform of the optical spectrum. Furthermore, more detailed characteristics may be detected using a higher-order derivative waveform, such as a third-order derivative waveform or higher. Furthermore, if the characteristic wavelength can be detected from the shape of the optical spectrum, it is not necessary to use a derivative waveform. [Explanation of symbols]

[0060] 1...information processing system, 1A...spectroscopic camera, 1B...terminal device (information processing device), 11...spectroscopic element, 12...imaging element, 13...camera control unit, 21...display unit, 22...operation unit, 23...memory unit, 24...control unit, 241...image acquisition unit, 242...spectrum calculation unit, 243...characteristic wavelength detection unit, 244...color setting unit, 245...image generation unit, 422...spectrum calculation unit, Cr...angle range, P1...feature point, P2...feature point, Pd...feature point, Pi...feature point, Q1...projection point, Q2...projection point, W...image target.

Claims

1. an image acquisition unit that acquires spectral images for a plurality of spectral wavelengths as captured images of an imaging target; a spectrum calculation unit that calculates a spectrum of each pixel based on the plurality of spectral images; a characteristic wavelength detection unit that detects a characteristic wavelength corresponding to a predetermined characteristic condition in the optical spectrum of each of the pixels; a color setting unit that, when a spectral wavelength range including the plurality of spectral wavelengths is assigned to a predetermined angle range of a hue wheel, calculates a characteristic angle corresponding to the characteristic wavelength of each pixel, and sets a color on the hue wheel corresponding to the characteristic angle as a characteristic color of the pixel; an image generating unit that generates a feature detection image by converting each pixel of the captured image into the feature color corresponding to the pixel; Equipped with An information processing device, wherein the angle range of the color wheel is equal to or greater than 30 degrees and equal to or less than 270 degrees.

2. the color setting unit sets the characteristic color by setting the shortest wavelength in the spectral wavelength range as the minimum angle of the angle range and the longest wavelength in the spectral wavelength range as the maximum angle of the angle range. The information processing device according to claim 1 .

3. the color setting unit sets the characteristic color by setting the shortest characteristic wavelength among the characteristic wavelengths detected by the plurality of pixels as the minimum angle of the angle range and the longest characteristic wavelength as the maximum angle of the angle range. The information processing device according to claim 1 .

4. the characteristic wavelength detection unit detects n characteristic wavelengths corresponding to the n characteristic conditions for each pixel; the color setting unit calculates the feature angle based on each projection point when the feature points are projected onto a straight line based on feature points corresponding to the feature wavelengths of the pixels plotted in an n-dimensional space with the n feature conditions as axes, respectively. The information processing device according to claim 1 .

5. the straight line is a straight line in which the variance of the projection points corresponding to the plurality of feature points is maximized; The information processing device according to claim 4 .

6. the color setting unit sets the characteristic color by setting one of the two points on the straight line that are farthest apart as the minimum angle of the angle range and the other as the maximum angle of the angle range.

6. The information processing device according to claim 4.

7. the characteristic wavelength detection unit detects the characteristic wavelength based on a second derivative waveform of the optical spectrum of each pixel. The information processing device according to claim 1 .

8. An information processing device according to any one of claims 1 to 7; a spectroscopic camera configured to capture the captured image including a plurality of spectroscopic images of the imaging target; An information processing system comprising:

9. 1. A method of processing image information by one or more processors, comprising: the one or more processors; acquiring spectral images for a plurality of spectral wavelengths as captured images of an imaging target; calculating a spectrum of each pixel based on the plurality of spectral images; detecting a characteristic wavelength corresponding to a predetermined characteristic condition in the optical spectrum of each of the pixels; When a spectral wavelength range including the plurality of spectral wavelengths is assigned to a predetermined angle range of a hue circle, a characteristic angle corresponding to the characteristic wavelength of each pixel is calculated, and a color on the hue circle corresponding to the characteristic angle is set as a characteristic color of the pixel; generating a feature detection image in which each pixel of the captured image is converted into the feature color corresponding to the pixel; and An information processing method, wherein the angle range of the color wheel is 30 degrees or more and 270 degrees or less.

Images